Image processing method and device

Abstract

The invention discloses an image processing method and device, and relates to the technical field of image processing. The method comprises the following steps: generating a preview RAW graph according to preset preview image parameters; splitting the preview RAW graph in a downsampling mode to obtain sub-graphs and original image data corresponding to each sub-graph, and storing the original image data based on a universal flash memory storage mode; analyzing the current performance index to determine the number of subgraphs which can be processed in parallel; and reading the original image data, and performing parallel processing and fusion on the sub-images according to the number of the sub-images which can be processed in parallel to obtain a preview image. Compared with the prior art, after the preview RAW graph is generated according to the preset preview image parameters, through the UFS data storage strategy based on the down-sampling mode, the memory and performance requirements are reduced, and meanwhile through the parallel processing strategy, the processing efficiency is improved. According to the invention, high resolution can be ensured under the condition of keeping the size of the preview image, so that the preview image is clearer.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image processing method and device. Background Art

[0002] With the rapid development of camera photography on mobile devices, users are using cameras more and more frequently. Currently, large-scale conventional previews based on cameras are divided into two resolutions: preview and photo. The hardware responsible for image processing divides the original RAW image into two data streams, preview and photo, and the clarity of the two is also different. As screen sizes become larger and larger, in order to ensure a higher resolution during preview, the memory and power consumption of the device will increase, and when the hardware capabilities of image processing are insufficient, anti-frame delays will occur, resulting in image freezes. In the related art, according to the preset preview output size, the final image is formed according to the preset size after the camera is started, thereby discarding part of the preview size to improve display accuracy. This method is equivalent to making a trade-off between image size and resolution, and does not guarantee both preview size and image quality resolution. Therefore, the current large-size preview display solution cannot ensure a high resolution while maintaining the preview image size, and there is a problem that the preview image is not clear enough. Summary of the Invention

[0003] In view of this, the present application provides an image processing method and device, the main purpose of which is to improve the current large-size preview display solution that cannot ensure a high resolution while maintaining the preview image size, and the problem that the preview image is not clear enough.

[0004] In a first aspect, the present application provides an image processing method, comprising:

[0005] Generate a preview RAW image according to the preset preview image parameters;

[0006] Splitting the preview RAW image into sub-images and original image data corresponding to each sub-image by downsampling, and storing the original image data based on a general flash memory storage method;

[0007] Analyze the current performance indicators to determine the number of subgraphs that can be processed in parallel;

[0008] The original image data is read, and the sub-images are processed and merged in parallel according to the number of sub-images that can be processed in parallel to obtain a preview image.

[0009] Optionally, the splitting the preview RAW image by down-sampling to obtain sub-images and corresponding original image data of each sub-image, and storing the original image data based on a general flash storage manner, comprises: performing down-sampling processing on the preview RAW image, and splitting to obtain a plurality of sub-images and corresponding original image data of each sub-image; performing serial number marking on the original image data according to the area order of the sub-images; and storing according to the serial number marking by a general flash storage manner.

[0010] Optionally, the original image data is read and the sub-images are processed in parallel according to the number of sub-images capable of being processed in parallel, which comprises: reading the original image data according to the serial number marking, and processing the sub-images in parallel by using a multi-state machine; the multi-state machine comprises an initialization state machine, an algorithm processing state machine and an end processing state machine.

[0011] In the case that the number of sub-images capable of being processed in parallel is 3, the processing of the sub-images in parallel by using a multi-state machine comprises: processing a first sub-image by the initialization state machine; after the first sub-image is processed by the initialization state machine, processing the first sub-image by the algorithm processing state machine and processing a second sub-image by the initialization state machine; after the first sub-image is processed by the algorithm processing state machine and the second sub-image is processed by the initialization state machine, processing the first sub-image by the end processing state machine, processing the second sub-image by the algorithm processing state machine and processing a third sub-image by the initialization state machine.

[0012] Fusing the sub-images to obtain a preview image, which comprises: calculating the brightness proportion of each pixel point in the original RAW image; and performing convolution fusion on the sub-images by a proportional window manner based on the brightness proportion to obtain the preview image.

[0013] Optionally, the performance indicators comprise at least one of device temperature, remaining memory information, CPU processing speed.

[0014] The analysis of the current performance indicators to determine the number of sub-images capable of being processed in parallel comprises: comparing the current device temperature with a preset temperature threshold to determine a temperature influence operator; and / or comparing the remaining memory information with a preset memory threshold to determine a memory influence operator; and / or comparing the CPU processing speed with a preset speed threshold to determine a performance influence operator; and determining the number of sub-images capable of being processed in parallel according to the temperature influence operator, the memory influence operator and the performance influence operator.

[0015] Optionally, the parallel processing of the subgraphs by the multi-state machine further comprises: in the case that the current performance index is abnormal, analyzing the current performance index to determine a processing duration threshold; and in the case that the processing duration of the current subgraph is greater than the processing duration threshold, discarding the current subgraph.

[0016] Optionally, after discarding the current subgraph, the method further comprises: counting the subgraphs that complete the parallel processing; and in the case that the number of the subgraphs that complete the parallel processing is less than the total number of the subgraphs, performing difference processing by using a convolution kernel to obtain the preview image.

[0017] In a second aspect, the present application provides an image processing device, comprising:

[0018] a generating unit configured to generate a preview RAW graph according to a preset preview image parameter;

[0019] a storage unit configured to split the preview RAW graph by a down-sampling manner to obtain subgraphs and corresponding original image data of each subgraph, and store the original image data based on a general flash storage manner;

[0020] an analyzing unit configured to analyze a current performance index to determine a number of subgraphs that can be processed in parallel;

[0021] a processing unit configured to read the original image data, and perform parallel processing and fusion of the subgraphs according to the number of subgraphs that can be processed in parallel to obtain a preview image.

[0022] In a third aspect, the present application provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the image processing method of the first aspect.

[0023] In a fourth aspect, the present application provides an electronic device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, wherein the processor executes the computer program to implement the image processing method of the first aspect.

[0024] In a fifth aspect, the present application provides a chip comprising one or more interface circuits and one or more processors; the interface circuit is configured to receive a signal from a memory of an electronic device and send the signal to the processor, the signal comprising computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device executes the image processing method of the first aspect.

[0025] By the technical solution, the image processing method and device provided by the application first generate a preview RAW image according to a preset preview image parameter, then split the preview RAW image to obtain sub-images and original image data corresponding to each sub-image through a downsampling manner, and store the original image data based on a general flash memory storage manner. The number of sub-images capable of being processed in parallel is determined by analyzing a current performance index, then the original image data is read, and the sub-images are processed and fused in parallel according to the number of sub-images capable of being processed in parallel to obtain a preview image. Compared with related technologies, after the preview RAW image is generated according to the preset preview image parameter, the UFS (general flash memory storage manner) data storage strategy based on the downsampling manner is used to reduce the memory and performance requirements, and the processing efficiency is improved through the parallel processing strategy. The high resolution can be ensured while the size of the preview image is maintained, so that the preview image is clearer.

[0026] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0027] The accompanying drawings incorporated in the specification and forming a part of the specification illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application.

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings required to be used in the embodiments or the prior art description will be briefly introduced. Obviously, for those skilled in the art, other drawings can also be obtained based on these drawings without creative labor.

[0029] Figure 1 A flowchart of an image processing method provided by an embodiment of the present application is shown;

[0030] Figure 2 A schematic diagram of data storage in a DDR data area provided by an embodiment of the present application is shown;

[0031] Figure 3 A schematic diagram of splitting an image through downsampling processing provided by an embodiment of the present application is shown;

[0032] Figure 4 A structural schematic diagram of an image processing device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0033] Some embodiments of the present disclosure will be described in detail herein with reference to the drawings, in which some embodiments of the present disclosure are shown by way of illustration. The following description is, therefore, not to be taken in a limiting sense, but is made merely for the purpose of describing the various illustrative embodiments of the present disclosure. The order in which the method, apparatus and / or system described herein are presented is merely for illustrative purposes and is not meant to limit the order in which the method, apparatus and / or system are performed. For example, the order in which the operations are described is not meant to limit the order in which the operations are performed, but is merely an example. Additionally, the description of the features of the present disclosure can be omitted for the sake of brevity and clarity.

[0034] The implementations described in some embodiments of the present disclosure are not meant to represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0035] In the field of camera display, in view of the current trend of frequent use of cameras, especially in the context of the continuous popularization of high-definition screens and the significant increase in PPI (pixels per inch), it is particularly important to improve the high-quality display of camera preview images. Existing camera systems usually adopt a dual-resolution strategy, that is, after the raw RAW image output by the sensing device, the hardware image processor distinguishes two independent data streams of preview and shooting. The preview data stream is obtained by downsampling to obtain a lower resolution to meet the real-time display requirement, while the shooting data is kept at a high resolution to ensure the imaging quality. However, with the continuous increase in the resolution of large-screen devices, high-resolution preview not only increases memory consumption and power consumption, but also may cause frame rate to drop and preview to lag due to hardware processing capacity bottleneck.

[0036] In the related art, it is proposed to prioritize the smoothness of the preview when setting the preview and shooting resolution, and to control the memory occupation and reduce the power consumption by reducing the preview size, but this inevitably sacrifices the clarity of the preview, thereby affecting the user's visual experience.

[0037] Therefore, in order to improve the current large-size preview display scheme which cannot guarantee a high resolution while maintaining the size of the preview image, there is a problem that the preview image is not clear enough. The present embodiment provides an image processing method as shown in Figure 1 The method comprises:

[0038] S101, generating a preview RAW image according to a preset preview image parameter.

[0039] Firstly, the execution subject of the image processing method proposed in the embodiment includes a mobile device with a camera function or application, including a smart phone, a smart watch, a camera, a tablet computer and other smart terminals with a camera function. More specifically, it can be applied to an image signal processor (ISP) in a mobile device or a device or chip that controls the image signal processor to perform image processing work. Such an image signal processor can include a hardware processing module and a software processing module for processing images.

[0040] The preview RAW image is generated according to the preset preview image parameters. The preset preview image parameters refer to higher preset values of the image parameters generated in the preview stage. The purpose is to break through the traditional practice of separating preview and shooting resolution, and to set the same high standard for both sizes. For example, the original preview resolution is 1920x1080, and the shooting resolution is 4096x3072. The high resolution of 4096x3072 is unified. In this way, the user can experience the same clarity as actual shooting in the preview stage, greatly improving the user experience. In addition, the parameters also include other camera configuration initialization parameters such as customized parameters, starting data formats YUV / MIPIRaw / UnpackedRaw, etc., which are all set according to the preset parameters.

[0041] The preview RAW image is the original image of the preview. When the user opens the camera application and starts previewing, the system initializes the camera function according to the pre-set parameter information. The application layer initiates a preview request, which will be passed down to the driver program at the bottom layer of the operating system, and then converted into specific control instructions for the image sensor (Sensor). For example, the automatic exposure (AE) module will calculate the appropriate exposure value (EV) according to the ambient light conditions, and then generate the corresponding gain information (Gain) and exposure time (Exposure Time). According to these information, the frame length (Frame Length) of each frame of image and the exposure time (Line Count) of each row of pixels are determined to ensure accurate exposure process and output image. This process can be referred to as shown in Figure 2 As shown in the figure, a preview RAW image is divided into RAW#1, RAW#2 and other cache streams in the DDR data area. As the resolution increases, the memory increases and the overall performance deteriorates.

[0042] S102, the preview RAW image is split into subgraphs and corresponding raw image data of each subgraph by downsampling, and the raw image data is stored based on a general flash storage method.

[0043] In S101, since the preview image adopts a large resolution, the demand for memory and processing power consumption is also large. In the related art, the data is usually split into a preview stream and a photographing stream. The preview stream needs to be displayed in real time, and its size is usually smaller than that of the photographing stream to ensure the fluency of the preview. However, even so, if the large-size original data is directly processed and temporarily stored in the DDR (Double Data Rate) memory, it will still cause excessive memory occupation, affecting the system performance and the real-time performance of the preview.

[0044] To solve this problem, the embodiment generates a plurality of smaller-size image data, called subgraphs or subarea images, and the original image data corresponding to each subgraph, by downsampling processing, that is, reducing the size of a large-size image by a certain ratio, after outputting the preview RAW graph. Figure 3 As shown, two kinds of downsampling image splitting methods are shown, the first kind splits the preview RAW graph into four subgraphs by downsampling, and the second kind splits it into nine subgraphs. Generally, it is split into 2x2 or 3x3 format, and of course, the splitting ratio is not required to be changed with different image formats and sizes. Then, the original image data is stored in the form of a file in the UFS storage unit with high speed and low power consumption by the UFS (Universal Flash Storage) storage method, instead of being entirely retained in the DDR memory, thereby effectively relieving the memory pressure.

[0045] S103, analyzing the current performance index to determine the number of subgraphs that can be processed in parallel.

[0046] The storage is only for caching, and the preview image needs to be further processed subsequently. Therefore, the performance index of the current device needs to be analyzed to determine the number of subgraphs that can be processed in parallel. The number of subgraphs that can be processed in parallel is the maximum number of subgraphs that can be processed simultaneously according to the performance of the current device. The performance index refers to the performance of the current device or the image processing module, including temperature, remaining memory, and CPU processing speed, etc. By monitoring the state of the core hardware resources, including but not limited to the processing capacity of the CPU, the temperature of the current device, and the memory capacity available for allocation, and other key indicators. Specifically, according to a series of preset thresholds, the system can dynamically adjust the number of image processing tasks being executed, for example, reduce or increase the number of subgraphs processed simultaneously, to ensure the effective use of resources and the stability of the system.

[0047] S104, reading the original image data, and processing and fusing the subgraphs in parallel according to the number of subgraphs that can be processed in parallel, to obtain the preview image.

[0048] In the embodiment, first, a preview RAW image is generated according to preset preview image parameters, then the preview RAW image is split into subgraphs and corresponding original image data of each subgraph through a downsampling manner, and the original image data is stored based on a general flash storage manner. The number of subgraphs capable of being processed in parallel is determined by analyzing the current performance index, then the original image data is read, and the subgraphs are processed and fused in parallel according to the number of subgraphs capable of being processed in parallel, to obtain a preview image. Compared with related technologies, after the preview RAW image is generated according to the preset preview image parameters, the UFS (general flash storage manner) data storage strategy based on the downsampling manner is used to reduce the memory and performance requirements, and the processing efficiency is improved through the parallel processing strategy. The high resolution can be ensured while the size of the preview image is maintained, so that the preview image is clearer.

[0049] Optionally, the preview RAW image is split into subgraphs and corresponding original image data of each subgraph through a downsampling manner, and the original image data is stored based on a general flash storage manner, including: the preview RAW image is downsampled and split into a plurality of subgraphs and corresponding original image data of each subgraph; the original image data is sequentially marked according to the region order of the subgraphs; and the original image data is stored according to the sequential marking through the general flash storage manner.

[0050] In the embodiment, the original preview RAW image is processed by using a downsampling manner, which greatly reduces the image data amount while retaining the main feature information of the image. Then, the image is divided into a plurality of smaller regions, and each region is sequentially marked to establish a corresponding relationship between the region and the original image data block. Such sequential marking helps to quickly locate and organize the image data in the subsequent reading and processing process. Finally, the original image data is stored in the UFS storage unit in the form of a file through the UFS, instead of being entirely retained in the DDR memory, thereby effectively relieving the memory pressure. The downsampling strategy not only reduces the occupation space of the RAW data in the DDR memory, but also reduces the complexity and time consumption of image processing during preview, thereby improving the smoothness of preview and the overall performance of the system. Through reasonable management and processing of the original data, both high-quality preview effect and real-time preview experience demand can be ensured.

[0051] Optionally, the original image data is read, and the subgraphs are processed in parallel according to the number of subgraphs capable of being processed in parallel, including: the original image data is read according to the sequential marking, and the subgraphs are processed in parallel by using a plurality of state machines; the plurality of state machines include an initialization state machine, an algorithm processing state machine, and an end processing state machine.

[0052] In the embodiment, since the serial number mark is sequentially performed in the down-sampling based UFS storage process, reading is performed according to the serial number, so as to realize fast positioning and organization of image data. Three state machines are set, and each state machine is used for different processing of the image, for example, the initialization state machine can be used for initialization processing of each subgraph.

[0053] In a feasible embodiment, in the case that the number of subgraphs capable of being processed in parallel is 3, the subgraphs are processed in parallel by using multiple state machines, including: processing the first subgraph by the initialization state machine; after the first subgraph is processed by the initialization state machine, processing the first subgraph by the algorithm processing state machine and processing the second subgraph by the initialization state machine; after the first subgraph is processed by the algorithm processing state machine and the second subgraph is processed by the initialization state machine, processing the first subgraph by the end processing state machine, processing the second subgraph by the algorithm processing state machine, and processing the third subgraph by the initialization state machine.

[0054] The parallel processing process mentioned in the embodiment is described. First, the multiple state machines include the initialization state machine, the algorithm processing state machine and the end processing state machine, and the initialization, the algorithm processing and the end processing need to be sequentially performed. The subgraphs are processed in parallel by using the multiple state machines. The parallel processing is not to simultaneously perform the same processing on the three subgraphs, but to take a certain state machine as the main machine, and when the first subgraph is processed by the certain state machine, the next subgraph is processed, and at the same time, the next state machine processes the first subgraph. That is, the multiple state machines simultaneously process multiple subgraphs, but the processing stages of each subgraph are different.

[0055] Further, the subgraphs are fused to obtain a preview image, including: calculating the brightness proportion of each pixel point in the original RAW graph; and performing convolution fusion on the subgraphs by the proportional window method based on the brightness proportion to obtain the preview image.

[0056] In the embodiment, the assumption premise of the split data fusion part is that the brightness information of the subgraph part of the image before splitting is basically consistent. That is, in order to ensure that the processed image will not have a large difference in brightness before and after processing. Therefore, the proportional window method is used for fusion in this part. The specific idea is to calculate the pixel brightness proportion of each pixel point of the subgraph before splitting, and then perform convolution calculation and fusion by the proportional window according to the brightness proportion of the subgraph part before splitting when the subgraph is fused, so that the preview image after fusion naturally connects in brightness, color and the like, so as to finally generate a high-quality preview image.

[0057] It should be noted that since the sub-graphs are obtained by splitting the original image by down-sampling, the luminance information of the sub-graphs of the original image is determined according to the sub-graphs after splitting, that is, the luminance proportion of each pixel in the corresponding sub-graph region in the original image is determined.

[0058] Optionally, the performance indicators include at least one of device temperature, remaining memory information, and CPU processing speed; the number of sub-graphs capable of being processed in parallel is determined by analyzing the current performance indicators, including: comparing the current device temperature with a preset temperature threshold to determine a temperature influence operator; and / or comparing the remaining memory information with a preset memory threshold to determine a memory influence operator; and / or comparing the CPU processing speed with a preset speed threshold to determine a performance influence operator; and determining the number of sub-graphs capable of being processed in parallel according to the temperature influence operator, the memory influence operator, and the performance influence operator.

[0059] In this embodiment, the number of sub-graphs capable of being processed in parallel is determined by monitoring key performance indicators, for example, by device temperature monitoring, memory usage monitoring, and CPU processing speed monitoring, to obtain three influence operators respectively, and the number of sub-graphs capable of being processed in parallel is determined by comprehensively analyzing the above three aspects to adapt to the optimal performance configuration under the current environment. For example, if the temperature is normal, the memory is sufficient, and the CPU processing speed is fast, a higher number of parallel sub-graphs can be set; otherwise, when any one of the performance indicators does not meet the condition, the number of sub-graphs capable of being processed in parallel is appropriately reduced.

[0060] Optionally, the sub-graphs are processed in parallel by using multiple state machines, and the method further includes: in the case that the current performance indicators are abnormal, analyzing the current performance indicators to determine a processing time threshold; and in the case that the processing time of the current sub-graph is greater than the processing time threshold, discarding the current sub-graph.

[0061] In this embodiment, when the device temperature is high and the performance does not meet the expectation, a timeout mechanism is established in the hardware processing module, and when the timeout occurs, the current device temperature, the available space of the DDR memory, and other performance indicators are detected. When the available space is insufficient, the number of sub-graphs processed needs to be dynamically adjusted, and the sub-graphs in the timeout part are discarded, thereby improving the processing efficiency and reducing the processing lag.

[0062] Further, after discarding the current sub-graph, the method further includes: counting the sub-graphs that have been processed in parallel; and in the case that the number of sub-graphs that have been processed in parallel is less than the total number of sub-graphs, performing difference processing on the sub-graphs that have been processed in parallel by using a convolution kernel to obtain a preview image.

[0063] In the embodiment, in the subgraph parallel processing process, if the processing result (such as the number) of the subgraph fails to be correctly returned, in order to restore the complete preview image as much as possible, a method of using 2x2 or 3x3 convolution kernel for difference processing is used to fill the missing part. Specifically, when it is found that the processing result of a subgraph is missing, according to the information of the adjacent successfully processed subgraph, the pixel value of the missing area can be calculated by using the convolution operation on the pixel points around the missing area. The 2x2 or 3x3 convolution kernel can simulate the image texture and color change trend in the local area to a certain extent, so as to accurately fill the blank area and ensure the continuity and integrity of the preview image. This method can effectively deal with abnormal situations that may occur in the subgraph processing process without significantly affecting the preview quality, and enhance the robustness and user experience of the system.

[0064] Further, as a specific implementation of the method shown in Figures 1 to 3 The embodiment provides an image processing device, as shown in Figure 4 The device comprises a generation unit 41, a storage unit 42, an analysis unit 43 and a processing unit 44.

[0065] The generation unit 41 is configured to generate a preview RAW image according to preset preview image parameters;

[0066] The storage unit 42 is configured to split the preview RAW image to obtain a subgraph and original image data corresponding to each subgraph by a downsampling method, and store the original image data based on a general flash storage method;

[0067] The analysis unit 43 is configured to analyze a current performance index to determine a number of subgraphs that can be processed in parallel;

[0068] The processing unit 44 is configured to read the original image data, and process and fuse the subgraphs in parallel according to the number of subgraphs that can be processed in parallel, to obtain a preview image.

[0069] In a specific application scenario, the storage unit 42 is specifically configured to perform downsampling processing on the preview RAW image, and split to obtain a plurality of subgraphs and original image data corresponding to each subgraph; the original image data is sequentially marked with a serial number according to the area order of the subgraph; and the general flash storage method is used for storage according to the serial number.

[0070] In a specific application scenario, the processing unit 44 is specifically further configured to read the original image data according to the serial number, and use a multi-state machine to process the subgraphs in parallel; the multi-state machine comprises an initialization state machine, an algorithm processing state machine and an end processing state machine.

[0071] In a specific application scenario, the processing unit 44 is specifically further configured to process the first subgraph through the initialization state machine; after the initialization state machine completes processing of the first subgraph, process the first subgraph through the algorithm processing state machine and process the second subgraph through the initialization state machine; after the algorithm processing state machine completes processing of the first subgraph and the initialization state machine completes processing of the second subgraph, process the first subgraph through the end processing state machine, process the second subgraph through the algorithm processing state machine, and process the third subgraph through the initialization state machine.

[0072] In a specific application scenario, the processing unit 44 is specifically further configured to calculate the brightness proportion of each pixel point in the original RAW graph; and perform convolution fusion on the subgraph through a proportional window method based on the brightness proportion, to obtain the preview image.

[0073] In a specific application scenario, the analysis unit 43 is specifically further configured to compare the current device temperature with a preset temperature threshold to determine a temperature influence operator; and / or compare the remaining memory information with a preset memory threshold to determine a memory influence operator; and / or compare the CPU processing speed with a preset speed threshold to determine a performance influence operator; and determine the number of subgraphs capable of being processed in parallel according to the temperature influence operator, the memory influence operator, and the performance influence operator.

[0074] In a specific application scenario, the processing unit 44 is specifically further configured to analyze the current performance index in a case where the current performance index is abnormal, to determine a processing duration threshold; and discard the current subgraph in a case where the processing duration of the current subgraph is greater than the processing duration threshold.

[0075] In a specific application scenario, the processing unit 44 is specifically further configured to count the subgraphs that complete the parallel processing; and perform difference processing through a convolution kernel to obtain the preview image in a case where the number of subgraphs that complete the parallel processing is less than the total number of subgraphs.

[0076] It should be noted that other corresponding descriptions of the functions of the units involved in the image processing method provided in this embodiment can be referred to the corresponding descriptions in the Figures 1 to 3 , which will not be described here in detail.

[0077] Based on the method as shown in Figures 1 to 3 , correspondingly, the present embodiment also provides a computer readable storage medium having a computer program stored thereon, which is executed by a processor to implement the method as shown in Figures 1 to 3 .

[0078] Based on such understanding, the technical scheme of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a U disk, a mobile hard disk, etc.) and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various implementation scenarios of the present application.

[0079] Based on the method as shown in the above Figures 1 to 3 , and the virtual device embodiment as shown in the above Figure 4 , in order to achieve the above-mentioned purpose, the embodiments of the present application also provide an electronic device, such as a smart phone, a tablet computer, a drone, a smart robot, etc. The device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to implement the method as shown in the above Figures 1 to 3 .

[0080] Optionally, the above-mentioned entity device can also include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface can include a display screen (Display), an input unit such as a keyboard (Keyboard), etc. The optional user interface can also include a USB interface, a card reader interface, etc. The network interface can optionally include a standard wired interface, a wireless interface (such as a WI-FI interface), etc.

[0081] Those skilled in the art can understand that the above-mentioned entity device structure provided by the embodiments does not constitute a limitation on the entity device, and can include more or fewer components, or combine certain components, or different component arrangements.

[0082] The storage medium can also include an operating system, a network communication module. The operating system is a program that manages the hardware and software resources of the above-mentioned entity device, supports the running of information processing programs and other software and / or programs. The network communication module is used to realize the communication between the internal components of the storage medium, and the communication with other hardware and software in the information processing entity device.

[0083] Based on the method as shown in the above Figures 1 to 3 , and the virtual device embodiment as shown in the above Figure 4 , the embodiments of the present application also provide a chip, which includes one or more interface circuits and one or more processors; the interface circuit is used to receive a signal from the memory of an electronic device and send the signal to the processor, the signal includes computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device executes the method as shown in the above Figures 1 to 3 .

[0084] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software with a necessary general hardware platform, or by hardware. By applying the scheme of the present embodiment, compared with the related art, after generating a preview RAW image according to preset preview image parameters, the present embodiment reduces the memory and performance requirements by using a UFS (Universal Flash Storage) data storage strategy based on a downsampling manner, and improves the processing efficiency by using a parallel processing strategy. The present embodiment can ensure high resolution while maintaining the size of the preview image, so that the preview image is clearer.

[0085] It should be noted that, in this document, relational terms such as“first” and“second”, and the like, are used solely to distinguish one entity or action from another entity or action, without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms“comprises”,“comprising”, or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by“comprises a...” does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0086] The above description is merely one specific implementation of the present application, and those skilled in the art can understand or implement the present application based on the above description. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to these embodiments described herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image processing method, characterized by, The method comprises the following steps: generating a preview RAW image according to preset preview image parameters; splitting the preview RAW image to obtain subgraphs and corresponding original image data of each subgraph by a downsampling method, and storing the original image data based on a general flash memory storage method; analyzing a current performance index to determine a number of subgraphs that can be processed in parallel; reading the original image data, and processing and fusing the subgraphs in parallel according to the number of subgraphs that can be processed in parallel to obtain a preview image.

2. The method of claim 1, wherein, The splitting the preview RAW image to obtain subgraphs and corresponding original image data of each subgraph by a downsampling method, and storing the original image data based on a general flash memory storage method comprises the following steps: downsampling the preview RAW image to obtain a plurality of subgraphs and corresponding original image data of each subgraph; sequentially marking the original image data according to the regions of the subgraphs; storing the original image data according to the sequential marking based on a general flash memory storage method.

3. The method of claim 2, wherein, The reading the original image data, and processing and fusing the subgraphs in parallel according to the number of subgraphs that can be processed in parallel comprises the following steps: reading the original image data according to the sequential marking, and processing the subgraphs in parallel by using a multi-state machine; the multi-state machine comprises an initialization state machine, an algorithm processing state machine and an end processing state machine; in the case that the number of subgraphs that can be processed in parallel is 3, the processing the subgraphs in parallel by using a multi-state machine comprises the following steps: processing a first subgraph by using the initialization state machine; after the first subgraph is processed by using the initialization state machine, processing the first subgraph by using the algorithm processing state machine, and processing a second subgraph by using the initialization state machine; after the first subgraph is processed by using the algorithm processing state machine, and the second subgraph is processed by using the initialization state machine, processing the first subgraph by using the end processing state machine, processing the second subgraph by using the algorithm processing state machine, and processing a third subgraph by using the initialization state machine; fusing the subgraphs to obtain a preview image, comprising the following steps: calculating the brightness proportion of each pixel point in the original RAW image; convolving and fusing the subgraphs by using a proportional window method based on the brightness proportion to obtain the preview image.

4. The method of claim 1, wherein, The performance index comprises at least one of a device temperature, remaining memory information and CPU processing speed. The analyzing a current performance index to determine a number of subgraphs that can be processed in parallel comprises the following steps: comparing a current device temperature with a preset temperature threshold to determine a temperature influence operator; and / or, comparing remaining memory information with a preset memory threshold to determine a memory influence operator; and / or, comparing a CPU processing speed with a preset speed threshold to determine a performance influence operator; determining the number of subgraphs that can be processed in parallel according to the temperature influence operator, the memory influence operator and the performance influence operator.

5. The method of claim 1, wherein, The processing the subgraphs in parallel by using a multi-state machine further comprises the following steps: In the case that the current performance index is abnormal, the current performance index is analyzed to determine a processing duration threshold; In the case that the processing duration of the current subgraph is greater than the processing duration threshold, the current subgraph is discarded.

6. The method of claim 5, wherein, After the current subgraph is discarded, the method further comprises: counting the subgraphs that complete the parallel processing; In the case that the number of subgraphs that complete the parallel processing is less than the total number of subgraphs, difference processing is performed using a convolution kernel to obtain the preview image.

7. An image processing apparatus characterized by comprising: Comprise: A generating unit configured to generate a preview RAW graph according to preset preview image parameters; A storage unit configured to split the preview RAW graph by a down-sampling manner to obtain subgraphs and original image data corresponding to each subgraph, and store the original image data based on a general flash storage manner; An analysis unit configured to analyze a current performance index to determine the number of subgraphs that can be processed in parallel; A processing unit configured to read the original image data, and process and fuse the subgraphs in parallel according to the number of subgraphs that can be processed in parallel to obtain a preview image.

8. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1-6.

9. An electronic device comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, The processor executes the computer program to implement the method of any one of claims 1-6.

10. A chip, characterized by Comprise one or more interface circuits and one or more processors; the interface circuit is used to receive signals from the memory of the electronic device, and send the signals to the processor, the signals include computer instructions stored in the memory; when the processor executes the computer instructions, makes the electronic device execute the method of any one of claims 1-6.

Images