RAW image processing method and device, computer equipment and storage medium

Abstract

The invention discloses a RAW image processing method and device, computer equipment and a storage medium, and the method comprises the steps: loading a to-be-processed RAW image, and carrying out the initialization of the RAW image; responding to an operation instruction of a user, and obtaining an operation type corresponding to the operation instruction; and performing corresponding processing on the RAW image based on the operation type, and performing real-time rendering on the RAW image in combination with a pyramid hierarchical frame insertion rendering mode and an asynchronous scheduling strategy. According to the embodiment of the invention, the exchange law characteristic of the algorithm is utilized, the calculation sequence is dynamically adjusted, and'one-time noise reduction and multiple-time color modulation 'is realized, so that the calculation load in a high-frequency interaction scene can be greatly reduced. Meanwhile, through a pyramid grading and task preemption mechanism, the visual fluency during user operation is ensured, low-resolution preview is preferentially displayed, and smooth transition to a high-definition image is realized by utilizing the visual persistence characteristic of human eyes, so that the problems of jamming and resource waste during real-time color matching in the prior art are solved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a RAW image processing method, apparatus, computer device, and storage medium. Background Technology

[0002] With the continuous development of digital photography technology, RAW format images have become the preferred image format in professional photography and post-production due to their advantages of fully preserving the original information of the image sensor, having a high dynamic range, and offering ample room for post-processing adjustments. Post-processing color correction of RAW format images requires a complex rendering pipeline to process and convert the original image data into image content suitable for screen display.

[0003] The current mainstream RAW image rendering standard uses a fixed serial pipeline execution method, which requires the sequential completion of multiple linear processing steps: First, the raw RAW data is read and decompressed; then, the logarithmic or encoded image data is converted to a linear space; next, the Bayer array is restored to a full-color RGB image using demosaicing technology; then, the device-dependent color space is converted to standard color spaces such as sRGB and Rec.709; during this stage, white balance, exposure compensation, and tone mapping are also performed simultaneously; finally, spatial filtering algorithms such as noise reduction and sharpening are applied to complete image enhancement and noise reduction. Even though some image processing software has introduced GPU acceleration technologies such as OpenGL and OpenCL to optimize the processing efficiency of individual steps, the overall rendering workflow logic still does not deviate from the linear serial framework.

[0004] This traditional fixed-series rendering pipeline mode exposes numerous technical flaws in actual RAW image post-processing color grading, severely impacting user experience and processing efficiency: First, the rendering process involves a large amount of redundant computation, resulting in high processing latency. Spatial algorithms such as noise reduction and sharpening are computationally intensive, while when users only adjust color / tone parameters like color temperature and exposure, these operations do not change the image's spatial structure and do not require re-execution of spatial algorithms. However, the fixed pipeline still forces the system to recalculate subsequent steps like noise reduction, or even rerun the entire rendering process, resulting in significant wasted computing power and operational lag. Second, it lacks an effective task preemption and discarding mechanism. When users rapidly drag the color grading slider, dozens of intensive rendering requests are generated per second, exceeding the capabilities of existing technologies. The technology processes all requests sequentially, which can easily lead to a backlog of tasks. Even if the user stops operating, the screen will continue to load the previous rendering results, making it impossible to respond to the user's latest parameter adjustments in real time. Third, there is a waste of resources due to full-resolution rendering. When previewing high-resolution RAW images such as 40-megapixel images on conventional resolution screens such as 1080P and 2K, the current technology will still complete the full-size image rendering before scaling it up for display, resulting in several times the amount of invalid calculations required for actual display during the preview stage. Fourth, there is a problem with discontinuous visual feedback. When processing high-load RAW images, single-frame rendering often takes a long time. When users adjust color grading parameters, they will clearly feel frame drops, black screen flickering, and other issues, which greatly reduces the interactive experience of post-color grading. Summary of the Invention

[0005] This invention provides a RAW image processing method, apparatus, computer device, and storage medium, aiming to solve the problems of stuttering and resource waste in real-time color grading in the prior art.

[0006] In a first aspect, embodiments of the present invention provide a RAW image processing method, comprising: Load the RAW image to be processed and initialize the RAW image; In response to the user's operation command, obtain the operation type corresponding to the operation command; The RAW image is processed according to the operation type, and the RAW image is rendered in real time by combining pyramid hierarchical frame interpolation rendering and asynchronous scheduling strategy.

[0007] Secondly, embodiments of the present invention provide a RAW image processing apparatus, comprising: An image loading unit is used to load the RAW image to be processed and initialize the RAW image; The type acquisition unit is used to acquire the operation type corresponding to the user's operation instruction in response to the user's operation instruction; The image rendering unit is used to perform corresponding processing on the RAW image based on the operation type, and to perform real-time rendering of the RAW image by combining pyramid hierarchical frame interpolation rendering and asynchronous scheduling strategy.

[0008] Thirdly, embodiments of the present invention provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the RAW image processing method as described in the first aspect.

[0009] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the RAW image processing method as described in the first aspect.

[0010] This invention provides a RAW image processing method, apparatus, computer device, and storage medium. The method includes: loading a RAW image to be processed and initializing the RAW image; responding to a user's operation command and obtaining the operation type corresponding to the operation command; processing the RAW image based on the operation type; and performing real-time rendering of the RAW image using a pyramid-based hierarchical frame interpolation rendering method and an asynchronous scheduling strategy. This invention utilizes the commutative law of algorithms to dynamically adjust the calculation order, achieving "noise reduction once, color correction multiple times," thus significantly reducing the computational load in high-frequency interactive scenarios. Simultaneously, through a pyramid-based hierarchical and task preemption mechanism, it ensures visual smoothness during user operation, prioritizing the display of low-resolution previews and utilizing the persistence of vision to smoothly transition to high-resolution images, thereby solving the problems of stuttering and resource waste in real-time color correction in existing technologies. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart illustrating a RAW image processing method provided in an embodiment of the present invention; Figure 2 This is a schematic block diagram of a RAW image processing apparatus provided in an embodiment of the present invention. Detailed Implementation

[0013] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0014] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0015] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0016] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0017] Please see below. Figure 1 The present invention provides a RAW image processing method, including steps S101 to S103.

[0018] Step S101: Load the RAW image to be processed and initialize the RAW image; Step S102: In response to the user's operation command, obtain the operation type corresponding to the operation command; Step S103: Process the RAW image according to the operation type, and render the RAW image in real time by combining the pyramid hierarchical frame interpolation rendering method and the asynchronous scheduling strategy.

[0019] In this embodiment, when processing a RAW image, the RAW image to be processed is first loaded and initialized, then the user operation command is responded to and the corresponding operation type is obtained; finally, the RAW image is processed according to the operation type, and real-time rendering of the RAW image is achieved by combining the pyramid hierarchical frame interpolation rendering method and the asynchronous scheduling strategy.

[0020] This embodiment leverages the commutative law of the algorithm to dynamically adjust the calculation order, achieving "noise reduction once, color adjustment multiple times," thus significantly reducing the computational load in high-frequency interactive scenarios. Simultaneously, through a pyramid hierarchical structure and task preemption mechanism, it ensures visual smoothness during user operation, prioritizing the display of low-resolution previews and utilizing the persistence of vision to smoothly transition to high-resolution images. Furthermore, this embodiment can combine hardware acceleration and block parallelism techniques to maximize the utilization of computing resources.

[0021] In one embodiment, loading the RAW image to be processed and initializing the RAW image includes: Obtain a RAW file and perform file processing on the RAW file; wherein, the file processing includes decompression, linearization, and depixelation. Based on the results of the file processing, a RAW image with basic linearity is generated.

[0022] In this embodiment, the preprocessing of RAW images begins with decompression to decode the data in the RAW file. Next, linearization is used to convert the non-linearly encoded data to a linear space, ensuring that subsequent calculations accurately reflect the true optical information. Then, the RAW image undergoes de-mosaic processing, for example, by interpolating the Bayer array using a de-mosaic algorithm to restore the complete RGB color channels, thereby generating a RAW image with basic linearity. This series of preprocessing steps ensures high quality and consistency of the image data before it enters the rendering pipeline, laying a solid foundation for subsequent real-time color grading and rendering operations. Simultaneously, the optimized design of the preprocessing workflow reduces unnecessary computational overhead, further improving overall processing efficiency.

[0023] In one embodiment, the operation type includes adjusting color / lighting parameters, and the corresponding processing of the RAW image based on the operation type includes: When the operation type is adjusting color / lighting parameters, check if there is valid cached noise-reduced image data; If there is valid cached denoised image data, the denoised image data is read directly and color pipeline processing is performed on the denoised image data. If no valid cached denoised image data exists, then denoising and sharpening calculations are performed on the RAW image, and color pipeline processing is performed after the denoising and sharpening calculations are completed.

[0024] In this embodiment, when the user's operation command is for color / lighting parameters, such as exposure, color temperature, hue, curves, etc., the system first checks whether there is valid cached denoised image data. If cached data exists, it is directly read and processed according to the color pipeline process, for example, performing exposure processing and then converting ABC to RGB. This avoids redundant calculations of denoising and sharpening steps, thus significantly improving processing efficiency. If the cache is invalid or does not exist, denoising and sharpening calculations need to be re-performed on the RAW image, and after completion, it enters the color pipeline processing stage before being displayed on the screen. This method fully leverages the advantages of the caching mechanism, with extremely short processing time (<5ms), achieving real-time full-frame preview without reducing resolution. It significantly reduces redundant calculations while ensuring image quality, making it particularly suitable for real-time color adjustment needs in high-frequency interactive scenarios.

[0025] In one embodiment, the operation type includes adjusting image quality / structure parameters, and the corresponding processing of the RAW image based on the operation type further includes: When the operation type is adjusting color / lighting parameters, the noise-reduced image data of the RAW image is invalidated, and the basic linear RGB data is read. A pyramid strategy is used to perform hierarchical processing on the basic linear RGB data; Denoising and sharpening calculations are performed on the basic linear RGB data after hierarchical processing, and the corresponding results are written into the new denoised image data; Color pipeline processing is performed on the new denoised image data.

[0026] In this embodiment, when the user's operation command is for color / lighting parameters, such as noise reduction intensity or sharpening radius, the basic linear RGB data is first read and then processed hierarchically using a pyramid strategy. The core of hierarchical processing is to decompose the image into multiple resolution levels, processing them progressively from low to high, prioritizing calculations at lower resolution levels to quickly respond to user operations; for example, processing 270P resolution first, then 1080P resolution. Next, noise reduction and sharpening calculations are performed on the hierarchical basic linear RGB data to ensure high-quality processing results at each resolution level. After the calculations are complete, the results are written to new noise-reduced image data, forming an updated cache. Finally, color pipeline processing is performed based on the new noise-reduced image data to complete the final rendering output and display it on the screen. This approach not only optimizes the allocation of computing resources but also significantly improves the smoothness of visual feedback through hierarchical processing, allowing users to have a better interactive experience when adjusting image quality / structure parameters.

[0027] In one embodiment, the real-time rendering of the RAW image by combining pyramid-level frame interpolation rendering and an asynchronous scheduling strategy includes: Multiple rendering tasks are generated on the RAW image using a multi-layer pyramid structure, and the highest priority is assigned to one layer of the multi-layer pyramid structure to obtain the target pyramid layer. For the rendering task corresponding to the target pyramid layer, priority rendering is performed according to the preset rendering level, and the rendering task after priority rendering is processed for real-time texture mapping and screen display. After the rendering task corresponding to the target pyramid layer is displayed, other pyramid layers are rendered using a progressive refinement method, and the rendered task that has been displayed is replaced after the rendering is completed.

[0028] In this embodiment, when processing RAW images using a pyramid-style frame interpolation rendering method, the RAW image is first decomposed into a multi-layered pyramid structure, with each layer corresponding to a different resolution level. By rationally scheduling these levels, the most critical target pyramid layer for visual feedback is prioritized during user operations. Specifically, the frequency of user operations and the magnitude of parameter adjustments can be evaluated to dynamically select the pyramid layer most suitable for real-time response as the target layer. For example, when the user quickly drags the color adjustment slider, lower resolution layers are rendered first to ensure smooth visuals; while when the user stops operating or makes fine adjustments, the process gradually switches to higher resolution layers to provide clearer detail.

[0029] For the rendering task of the target pyramid layer, computing resources are allocated according to the preset rendering volume to ensure that critical tasks can be completed in the shortest possible time. After rendering is completed, real-time texture mapping is immediately applied to the screen, allowing users to see the adjustment effect immediately. This mechanism fully utilizes the human eye's tolerance for low-resolution images and its sensitivity to dynamic changes, significantly improving the interactive experience.

[0030] Meanwhile, rendering tasks for other non-target pyramid layers are completed gradually using a progressive refinement approach. Specifically, the system continuously renders the remaining layers in the background, automatically replacing the low-resolution result already displayed after each layer is rendered. This approach not only avoids the performance pressure of rendering at full resolution all at once but also achieves a natural transition from blurry to sharp through phased output. Ultimately, when all layers' rendering tasks are completed, the screen displays a fully processed, high-precision image, thus ensuring both efficiency and image quality.

[0031] For example, to address the issue of time-consuming high-resolution rendering, this embodiment constructs a 5-layer image pyramid, and uses three of these layers as examples for illustration: Level 0: Target display size (e.g., 4000P); Level 1: Half the target size (e.g., 2000P); Level 2: 1 / 4 of the target size (e.g., 1000P); The execution logic is as follows: Priority queue: When a rendering request is initiated, three rendering tasks, L2, L1, and L0, are generated synchronously, with L2 assigned the highest priority; Lightning Preview (L2): Prioritizes CPU / GPU rendering of 1000P images (data volume is only 1 / 16 of 4000P). This process will be completed first, and then the image will be applied to the screen to ensure that the screen responds in real time and without delay when the user drags the slider. Progressive refinement (L1→L0): After L2 is displayed on the screen, L1 continues to be rendered in the background; after L1 is completed, the L2 image on the screen is replaced; finally, L0 is rendered and the replacement is completed.

[0032] Visual frame interpolation: It should be noted that, in order to avoid the visual jump caused by the sudden change from blurry to clear during the switching from L2 to L0, this embodiment uses L1 as an intermediate transition frame (frame interpolation) to achieve a smooth transition in visual experience.

[0033] Furthermore, the purpose of setting up a 5-layer pyramid in this embodiment is based on current development trends: RAW images generally have side lengths of 3000-7000 pixels, while display resolutions are generally 1K, 2K, and 4K. A 5-layer pyramid can basically cover all scenarios. Only three layers are actually used because each additional layer increases rendering speed by 4 times; that is, the third layer is 4×4=16 times faster than the first layer. This efficiency is sufficient, and there is no need to add further layers. Of course, these three layers could be Level 0, 1, 2, or Level 2, 3, 4, depending on the target display size. The target display size is the size of the image to be displayed on the screen. This size may change dynamically because users will scale the image according to different proportions depending on the scene. Therefore, the selection of the three-layer pyramid also needs to be dynamically adjusted. In short, the core principle is to avoid rendering too many pixels. For example, rendering a 4K image on a 2K resolution screen is a complete waste of resources because the display cannot accommodate all the pixels of a 4K image, and the final display effect is no different from a 2K image. In such scenarios, directly rendering a 2K image is more cost-effective.

[0034] In one embodiment, the real-time rendering of the RAW image by combining pyramid-level frame interpolation rendering and an asynchronous scheduling strategy further includes: When performing hierarchical frame interpolation rendering of the RAW image through the multi-layer pyramid structure, determine whether a new operation command triggered by the user has been received. If a new operation command triggered by the user is received, a task preemption and discarding mechanism is adopted, and the RAW image is re-rendered by hierarchical frame interpolation based on the new operation command.

[0035] In addition to employing a pyramid-based hierarchical frame interpolation rendering method, this embodiment also processes RAW images using an asynchronous scheduling strategy. Specifically, when a new user-triggered operation command is detected during rendering, the currently ongoing rendering task is immediately interrupted, and the rendering priority is readjusted based on the new operation command. This task preemption and discarding mechanism ensures that the system can quickly respond to the user's latest needs without delays caused by previous computational tasks. For example, if the user quickly switches sliders while adjusting parameters, the system will prioritize processing the low-resolution rendering tasks most relevant to the current operation, while discarding high-resolution rendering tasks that are not yet completed. This approach not only avoids resource waste but also significantly improves the smoothness of interaction.

[0036] Furthermore, asynchronous scheduling strategies can further optimize resource utilization efficiency by decomposing rendering tasks into multiple independent subtasks and allocating them to different computing units for parallel processing. For example, the CPU can handle logic control and task distribution, while the GPU focuses on parallel computation of image data. With hardware acceleration support, the system can complete complex calculations in a very short time, thereby achieving efficient real-time rendering effects.

[0037] For example, if a user triggers a rendering command again (such as a change in slider position) while L2 or L1 rendering is complete and L0 (full-size) rendering is in progress, the currently running L0 rendering task will be immediately aborted, computing resources will be released, some calculated data will be discarded, and the L2→L1→L0 process will be restarted with new parameters. This mechanism prevents stale rendering tasks from blocking the latest interactive requests.

[0038] In one embodiment, the RAW image processing method further includes: The RAW image is rendered in real time using auxiliary performance optimization strategies; wherein, the auxiliary performance optimization strategies include a combination of block rendering, pipeline and multi-threaded rendering strategies, a strategy to limit the maximum rendering side length, an instruction set acceleration strategy, a local rendering strategy, and a cross-platform interoperability strategy.

[0039] This embodiment further improves the real-time rendering efficiency of RAW images by employing auxiliary performance optimization strategies, while ensuring compatibility and stability across different hardware environments. Specifically, the block rendering strategy divides the image into several small blocks, each processed independently, effectively reducing the amount of data processed per computation, minimizing memory usage, and improving cache hit rate. The pipeline and multi-threaded rendering combination strategy decomposes the rendering process into multiple stages and utilizes multi-threading technology to execute these stages in parallel, maximizing the utilization of computing resources. For example, one thread handles data preprocessing, another performs noise reduction calculations, and a third performs color adjustments; this pipeline design significantly shortens the overall processing time. In practical applications, each stage of the pipeline (decoding, conversion, rendering) can adopt a producer-consumer model to improve pipeline throughput.

[0040] The maximum rendering side length limitation strategy aims to set a maximum downsample factor during the decoding stage based on the physical resolution of the output device (e.g., a 1920×1080 screen), strictly prohibiting the rendering of pixels exceeding the screen's display capacity (e.g., not rendering 4K images on a 1080P screen). For example, when rendering a 4K image on a 2K monitor, the system will automatically limit the rendering side length to 2K, thereby saving computing resources and improving response speed. The instruction set acceleration strategy fully utilizes the advanced instruction sets of modern processors (such as AVX, SSE, etc.) to perform low-level optimizations on key algorithms, achieving more efficient numerical computation and data processing.

[0041] Instruction set acceleration refers to the ability to force the use of SIMD instruction sets (AVX2 / AVX-512 / NEON) for vectorized computation of core algorithms (such as demosaicing and color matrix multiplication).

[0042] The local rendering strategy means that when the image is in zoomed-in viewing mode, only the tiles within the viewport are rendered, and data outside the viewport is not processed.

[0043] Cross-platform interoperability strategies (such as OpenGL / Vulkan interoperability) refer to directly mapping the decoded data to GPU texture memory (PBO) and using Fragment Shader tools for final color space conversion and screen rendering, thereby reducing the time spent on data copying from the CPU to the GPU.

[0044] Through the synergistic effect of the above-mentioned multiple auxiliary performance optimization strategies, this embodiment not only achieves efficient real-time rendering of RAW images, but also significantly reduces the consumption of system resources while ensuring image quality, providing users with a smooth and high-quality image processing experience.

[0045] In summary, this embodiment fundamentally solves the problems of stuttering and latency in real-time processing of RAW images by optimizing and reconstructing the image processing pipeline and rendering architecture. Specifically, this embodiment innovatively decomposes the image processing pipeline into a spatial processing stage and a color processing stage based on the pipeline decoupling and reordering technology of operation type, and builds a dedicated caching mechanism between the two stages. Taking advantage of the characteristic that color adjustment does not change the spatial structure of the image, the noise-reduced image data can be directly reused during the user's color adjustment process, effectively avoiding repeated heavy-load calculations in high-frequency interaction scenarios and significantly reducing computing power consumption. This embodiment is based on a multi-resolution pyramid preemptive rendering architecture and designs a progressive rendering strategy of L2 (1000P) → L1 (2000P) → L0 (4000P). Combined with a preemptive mechanism that discards old high-resolution tasks as new instructions arrive, the system achieves a near-zero latency interactive feel with limited computing power. This embodiment also uses visual continuity frame interpolation technology to use the intermediate resolution level L1 as a visual buffer between the low-resolution L2 and the high-resolution L0, smoothing the switching process between images of different resolutions and completely solving the visual abruptness caused when switching directly from a low-resolution image to a high-resolution image.

[0046] Based on the aforementioned core technologies, this embodiment achieves significant performance improvements and user experience optimizations in real-time RAW image processing, with outstanding technical effects: First, the color grading response speed is improved by more than 10 times. When the user only adjusts color / lighting parameters such as exposure and white balance, the system can directly reuse cached data that has undergone heavy calculations such as decompression, de-mosaicing, and noise reduction. The rendering process only requires simple matrix multiplication, achieving ultimate color grading smoothness. Second, zero-latency interaction is achieved. By prioritizing the rendering of 270P low-resolution images that are only 1 / 16 of the original size, it ensures that no matter how large the original size of the RAW image is, the screen provides visual feedback the instant the user touches the color grading slider. First, there is no operational delay; second, it significantly reduces the power consumption of computing resources. With the help of the task discarding mechanism, the system can directly discard invalid full-size rendering of outdated frames, effectively reducing the continuous high-load running time of the CPU / GPU, saving device power and reducing hardware heat generation; third, it achieves a dynamic balance between image quality and performance. The system can intelligently adjust the rendering strategy according to the user's operation status. When the user is observing the image at rest, it automatically completes the 1080P or even full-size high-definition image to ensure the image quality needs for viewing details. When the user dynamically adjusts the color grading parameters, it prioritizes the rendering frame rate, which is in line with the psychological model of human-computer interaction and greatly improves the overall experience of post-color grading of RAW images.

[0047] Figure 2 This is a schematic block diagram of a RAW image processing apparatus 200 provided in an embodiment of the present invention. The apparatus 200 includes: Image loading unit 201 is used to load the RAW image to be processed and initialize the RAW image; The type acquisition unit 202 is used to acquire the operation type corresponding to the user's operation instruction in response to the user's operation instruction; The image rendering unit 203 is used to perform corresponding processing on the RAW image based on the operation type, and to perform real-time rendering of the RAW image by combining the pyramid hierarchical frame interpolation rendering method and the asynchronous scheduling strategy.

[0048] In one embodiment, the image loading unit 201 includes: Obtain a RAW file and perform file processing on the RAW file; wherein, the file processing includes decompression, linearization, and depixelation. Based on the results of the file processing, a RAW image with basic linearity is generated.

[0049] In one embodiment, the operation type includes adjusting color / lighting parameters, and the image rendering unit 203 includes: When the operation type is adjusting color / lighting parameters, check if there is valid cached noise-reduced image data; If there is valid cached denoised image data, the denoised image data is read directly and color pipeline processing is performed on the denoised image data. If no valid cached denoised image data exists, then denoising and sharpening calculations are performed on the RAW image, and color pipeline processing is performed after the denoising and sharpening calculations are completed.

[0050] In one embodiment, the operation type includes adjusting image quality / structure parameters, and the image rendering unit 203 further includes: When the operation type is adjusting color / lighting parameters, the noise-reduced image data of the RAW image is invalidated, and the basic linear RGB data is read. A pyramid strategy is used to perform hierarchical processing on the basic linear RGB data; Denoising and sharpening calculations are performed on the basic linear RGB data after hierarchical processing, and the corresponding results are written into the new denoised image data; Color pipeline processing is performed on the new denoised image data.

[0051] In one embodiment, the image rendering unit 203 includes: Multiple rendering tasks are generated on the RAW image using a multi-layer pyramid structure, and the highest priority is assigned to one layer of the multi-layer pyramid structure to obtain the target pyramid layer. For the rendering task corresponding to the target pyramid layer, priority rendering is performed according to the preset rendering level, and the rendering task after priority rendering is processed for real-time texture mapping and screen display. After the rendering task corresponding to the target pyramid layer is displayed, other pyramid layers are rendered using a progressive refinement method, and the rendered task that has been displayed is replaced after the rendering is completed.

[0052] In one embodiment, the image rendering unit 203 further includes: When performing hierarchical frame interpolation rendering of the RAW image through the multi-layer pyramid structure, determine whether a new operation command triggered by the user has been received. If a new operation command triggered by the user is received, a task preemption and discarding mechanism is adopted, and the RAW image is re-rendered by hierarchical frame interpolation based on the new operation command.

[0053] In one embodiment, the RAW image processing apparatus 200 further includes: The RAW image is rendered in real time using auxiliary performance optimization strategies; wherein, the auxiliary performance optimization strategies include a combination of block rendering, pipeline and multi-threaded rendering strategies, a strategy to limit the maximum rendering side length, an instruction set acceleration strategy, a local rendering strategy, and a cross-platform interoperability strategy.

[0054] Since the embodiments of the apparatus and the embodiments of the method correspond to each other, please refer to the description of the embodiments of the method for the embodiments of the apparatus, which will not be repeated here.

[0055] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed, can perform the steps provided in the above embodiments. The storage medium may include various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0056] This invention also provides a computer device, which may include a memory and a processor. The memory stores a computer program, and when the processor calls the computer program in the memory, it can implement the steps provided in the above embodiments. Of course, the computer device may also include various network interfaces, power supplies, and other components.

[0057] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to in the method section. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from the principles of this application, and these improvements and modifications also fall within the protection scope of the claims of this application.

[0058] It should also be noted that, in this specification, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

Claims

1. A RAW image processing method, characterized in that, include: Load the RAW image to be processed and initialize the RAW image; In response to the user's operation command, obtain the operation type corresponding to the operation command; The RAW image is processed according to the operation type, and the RAW image is rendered in real time by combining pyramid hierarchical frame interpolation rendering and asynchronous scheduling strategy.

2. The RAW image processing method according to claim 1, characterized in that, The loading of the RAW image to be processed and the initialization of the RAW image include: Obtain a RAW file and perform file processing on the RAW file; wherein, the file processing includes decompression, linearization, and depixelation. Based on the results of the file processing, a RAW image with basic linearity is generated.

3. The RAW image processing method according to claim 1, characterized in that, The operation type includes adjusting color / lighting parameters, and the corresponding processing of the RAW image based on the operation type includes: When the operation type is adjusting color / lighting parameters, check if there is valid cached noise-reduced image data; If there is valid cached denoised image data, the denoised image data is read directly and color pipeline processing is performed on the denoised image data. If no valid cached denoised image data exists, then denoising and sharpening calculations are performed on the RAW image, and color pipeline processing is performed after the denoising and sharpening calculations are completed.

4. The RAW image processing method according to claim 1, characterized in that, The operation type includes adjusting image quality / structure parameters, and the corresponding processing of the RAW image based on the operation type further includes: When the operation type is adjusting color / lighting parameters, the noise-reduced image data of the RAW image is invalidated, and the basic linear RGB data is read. A pyramid strategy is used to perform hierarchical processing on the basic linear RGB data; Denoising and sharpening calculations are performed on the basic linear RGB data after hierarchical processing, and the corresponding results are written into the new denoised image data; Color pipeline processing is performed on the new denoised image data.

5. The RAW image processing method according to claim 1, characterized in that, The method of combining pyramid-level frame interpolation rendering and asynchronous scheduling strategy to render the RAW image in real time includes: Multiple rendering tasks are generated on the RAW image using a multi-layer pyramid structure, and the highest priority is assigned to one layer of the multi-layer pyramid structure to obtain the target pyramid layer. For the rendering task corresponding to the target pyramid layer, priority rendering is performed according to the preset rendering level, and the rendering task after priority rendering is processed for real-time texture mapping and screen display. After the rendering task corresponding to the target pyramid layer is displayed, other pyramid layers are rendered using a progressive refinement method, and the rendered task that has been displayed is replaced after the rendering is completed.

6. The RAW image processing method according to claim 5, characterized in that, The method of combining pyramid-level frame interpolation rendering and asynchronous scheduling strategy to render the RAW image in real time also includes: When performing hierarchical frame interpolation rendering of the RAW image through the multi-layer pyramid structure, determine whether a new operation command triggered by the user has been received. If a new operation command triggered by the user is received, a task preemption and discarding mechanism is adopted, and the RAW image is re-rendered by hierarchical frame interpolation based on the new operation command.

7. The RAW image processing method according to claim 1, characterized in that, Also includes: The RAW image is rendered in real time using auxiliary performance optimization strategies; wherein, the auxiliary performance optimization strategies include a combination of block rendering, pipeline and multi-threaded rendering strategies, a strategy to limit the maximum rendering side length, an instruction set acceleration strategy, a local rendering strategy, and a cross-platform interoperability strategy.

8. A RAW image processing apparatus, characterized in that, include: An image loading unit is used to load the RAW image to be processed and initialize the RAW image; The type acquisition unit is used to acquire the operation type corresponding to the user's operation instruction in response to the user's operation instruction; The image rendering unit is used to perform corresponding processing on the RAW image based on the operation type, and to perform real-time rendering of the RAW image by combining pyramid hierarchical frame interpolation rendering and asynchronous scheduling strategy.

9. A computer device, characterized in that, The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the RAW image processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the RAW image processing method as described in any one of claims 1 to 7.

Images