RAW image processing method, chip and electronic device

Abstract

The present invention discloses an image processing method, chip, computer-readable storage medium, and electronic device. The image processing method identifies a first region and a second region of a single-frame RAW image, performs block processing on the first region and the second region based on different block partitioning strategies for the first region and the second region, performs LUT calculation based on the block partitioning results to obtain a LUT result, and performs LSC processing on the first region and the second region based on the LUT result to obtain a current frame image. The method can thereby identify a first region of interest to a user in the image, perform differentiated processing on the first region and the second region, and improve the image processing effect of the first region of interest to the user.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a RAW image processing method, chip, and electronic device. Background Art

[0002] Currently, LSC (Lens Shading Correction) processing of an image is usually performed uniformly on the entire image, and is unable to focus on the key areas of user interest, resulting in deviations in the processing effect of the key areas. Summary of the Invention

[0003] The present invention aims to at least partially solve one of the technical problems in the related art. To this end, a first object of the present invention is to provide an image processing method that can identify regions of an image and perform differentiated processing on different regions based on the region recognition results to improve the image processing effect.

[0004] To achieve the above-mentioned objectives, an embodiment of the first aspect of the present invention proposes an image processing method, comprising: identifying a first area and a second area of a single-frame raw image; performing block processing on the first area and the second area, wherein the block processing of the first area and the second area is based on different block processing strategies; and performing LUT calculation and LSC processing on the first area and the second area in sequence according to the block processing results to obtain a current frame image.

[0005] According to the image processing method of an embodiment of the present invention, by identifying the first area and the second area of a single-frame RAW image, the first area and the second area are block-processed based on different block strategies for the first area and the second area, LUT calculation is performed according to the block processing result to obtain the LUT result, and LSC processing is performed on the first area and the second area according to the LUT result to obtain the current frame image, so that the first area of the image that the user is paying attention to can be identified, and the first area and the second area are differentiated and processed, thereby improving the image processing effect of the first area that the user is paying attention to.

[0006] According to one embodiment of the present invention, identifying the first area and the second area of a single-frame RAW image includes: dividing the single-frame RAW image into regions according to focal plane data to obtain the first area and the second area, wherein the first area is the focus area and the second area is the image area outside the first area.

[0007] According to one embodiment of the present invention, identifying the first area and the second area of a single-frame RAW image includes: performing content analysis on the single-frame RAW image to obtain a main area and a background area of the single-frame RAW image, wherein the main area is the first area and the background area is the second area.

[0008] According to one embodiment of the present invention, identifying the first area and the second area of a single-frame RAW image includes: obtaining N historical frame RAW images preceding the single-frame RAW image, where N is an integer greater than 1; and obtaining areas of the single-frame RAW image that have changed relative to the historical frame RAW images, wherein the changed area is the first area, and the unchanged area is the second area.

[0009] According to one embodiment of the present invention, performing block processing on the first region and the second region includes: performing block processing on the RAW image based on different image block densities of the first region and the second region, wherein the image block density of the first region is greater than the image block density of the second region.

[0010] According to one embodiment of the present invention, a RAW image is segmented based on different image block densities in a first area and a second area, including: setting a plurality of horizontal dividing lines along a pixel row direction and a plurality of vertical dividing lines along a pixel column direction, wherein the horizontal dividing lines and the vertical dividing lines divide the RAW image into a plurality of image blocks; configuring the distances between the horizontal dividing lines and the distances between the vertical dividing lines running through the first area so that the image block density in the first area is greater than or equal to a first preset density value; configuring the distances between the horizontal dividing lines and the distances between the vertical dividing lines running through the second area so that the image block density in the second area is less than the first preset density value.

[0011] According to one embodiment of the present invention, the number of image blocks when a single-frame RAW image is segmented based on different image block densities in the first region and the second region is less than or equal to the number of image blocks when the single-frame RAW image is evenly segmented using a first preset density value.

[0012] According to one embodiment of the present invention, performing block processing on a first region and a second region includes: uniformly blocking a single-frame RAW image to obtain a plurality of original image blocks; converting the single-frame RAW image into a curved surface block image based on a distance from a pixel point in each original image block to a focal plane, wherein the distance from a pixel point in the original image block in the first region to the focal plane is smaller than the distance from a pixel point in the original image block in the second region to the focal plane; and projecting the curved surface block image in a direction perpendicular to the focal plane to obtain a projected image block, and using the projected image block as a result of the block processing.

[0013] According to one embodiment of the present invention, after performing LSC processing on the first area and the second area respectively according to the LUT result, it also includes: obtaining a historical frame image associated with the current frame image based on the current frame image; and correcting the area in the current frame image that is the same as the historical frame image based on the historical frame image to obtain a corrected current frame image.

[0014] According to an embodiment of the present invention, the method further includes: segmenting the corrected current frame image and caching the segmentation results.

[0015] To achieve the above objectives, a second embodiment of the present invention provides a RAW image processing chip, comprising:

[0016] A first chip is configured to identify a first region and a second region of a single-frame raw image; perform block processing on the first region and the second region, wherein the block processing on the first region and the second region is based on different block processing strategies, and perform LUT calculation on the first region and the second region respectively according to the block processing results to obtain LUT results;

[0017] The second chip is connected to the first chip, and is used to perform LSC processing on the first area and the second area respectively according to the LUT result to obtain the current frame image.

[0018] According to the image processing chip of the embodiment of the present invention, the first chip identifies the first area and the second area of a single-frame RAW image, and performs block processing on the first area and the second area based on different block processing strategies for the first area and the second area, and performs LUT calculation on the first area and the second area respectively according to the block processing results to obtain LUT results, and the second chip performs LSC processing on the first area and the second area respectively according to the LUT results to obtain the current frame image, so that the first area of the image that the user is paying attention to can be identified, the first area and the second area can be distinguished and processed, and the image processing effect of the first area that the user is paying attention to can be improved.

[0019] To achieve the above objectives, the third embodiment of the present invention proposes an electronic device, including a memory, a processor, and an image processing program stored in the memory and executable on the processor. When the processor executes the image processing program, the aforementioned image processing method is implemented.

[0020] According to the electronic device of the embodiment of the present invention, the image processing method can identify the first area of interest to the user in the image, perform differentiated processing on the first area and the second area, and improve the image processing effect of the first area of interest to the user.

[0021] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 This is an application scenario diagram of the image processing method according to an embodiment of the present invention;

[0023] Figure 2is a flowchart of an image processing method according to an embodiment of the present invention;

[0024] Figure 3 is a structural diagram of an image collector according to an embodiment of the present invention;

[0025] Figure 4 Image segmentation strategy for existing technologies;

[0026] Figure 5 An image segmentation strategy according to an embodiment of the present invention;

[0027] Figure 6 Schematic diagram of a curved image block converted from an original image block according to the distance between the pixel point in the original image block and the focal plane;

[0028] Figure 7 is a schematic structural diagram of an image processing system according to an embodiment of the present invention;

[0029] Figure 8 is a schematic structural diagram of an image processing chip according to an embodiment of the present invention;

[0030] Figure 9 FIG. 4 is a structural diagram of an image processing chip according to another embodiment of the present invention. DETAILED DESCRIPTION

[0031] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention.

[0032] The image processing method provided in this application can be applied to Figure 1 In the electronic device shown. The electronic device includes an image collector 11, a front-end image signal processing chip 12 and a back-end application processing chip 13, wherein the image collector 11 is used to obtain a single-frame RAW image. The front-end image signal processing chip 12 is connected to the image collector 11, and is used to receive a single-frame RAW image, identify the first area and the second area of the single-frame RAW image, and perform block processing on the first area and the second area, wherein the block processing of the first area and the second area is based on different block processing strategies, and then LUT calculation is performed on the first area and the second area respectively according to the block processing results to obtain LUT results; the back-end application processing chip 13 is used to perform LSC processing on the first area and the second area respectively according to the LUT results to obtain the current frame image. The electronic device can be a mobile phone, a tablet computer, a personal computer, a smart camera, a vehicle-mounted image acquisition device, and other devices with photo or video functions.

[0033] In this application, reference is made to Figure 2 As shown, a RAW image processing method is provided, which can be applied to Figure 1 The electronic device shown in FIG. 1 may include the following steps:

[0034] Step S101 : identifying a first region and a second region of a single-frame RAW image.

[0035] In this embodiment, Figure 3 As shown, the image acquisition device of the electronic device is composed of a lens 31 and an image sensor 32. The lens 31 is used to collect external light source signals and provide them to the image sensor. The image sensor 32 senses the light source signals from the lens 31 and converts them into digital raw image data, namely RAW images. RAW images are unprocessed and uncompressed image formats.

[0036] After acquiring a single-frame RAW image, the image acquisition device sends the single-frame RAW image to the front-end image signal processing chip. The front-end image signal processing chip performs region recognition on the single-frame RAW image to identify a first region and a second region in the single-frame RAW image. The first region is the image region of interest to the user, such as the region manually focused by the user or the region where the target object in the image is located identified using a related image algorithm, and the second region is the image region outside the first region. In the present application, the front-end image signal processing chip can use an image content analysis algorithm, an image recognition algorithm, or the like to identify the target object in the image to determine the first region and the second region of the current single-frame RAW image.

[0037] As a first example, identifying the first area and the second area of a single-frame RAW image includes: dividing the single-frame RAW image into areas according to focal plane data to obtain the first area and the second area, wherein the first area is the focus area and the second area is the image area outside the first area.

[0038] Specifically, when a user uses an electronic device to capture an image, the user usually manually focuses on the area of interest. The focus area is the area that the user is paying attention to, that is, the focus area can be used as the first area of a single-frame RAW image. When performing area recognition on the RAW image, focal plane data can be obtained. Furthermore, the image recognition model can be used to identify the target object image corresponding to the focus area. After the target object image is identified, the boundary of the target object image can be demarcated, and the area where the target object image is located can be used as the first area.

[0039] For example, when a single-frame RAW image includes a human figure, the user typically focuses on the area containing the face or body image. After the user performs the focus operation, the front-end image signal processing chip obtains the focal plane data. If the focal plane is located in the face area, the front-end image signal processing chip extracts the facial outline and determines the facial outline area as the first area, and the image area outside the first area as the second area. It is understood that a RAW image can include multiple faces, so the front-end image signal processing chip can extract multiple facial areas. In this case, there can be multiple first areas.

[0040] As a second example, identifying the first area and the second area of a single-frame RAW image includes: performing content analysis on the single-frame RAW image to obtain a subject area and a background area of the single-frame RAW image, wherein the subject area is the first area and the background area is the second area.

[0041] Specifically, the RAW image in this embodiment includes a subject area and a background area, wherein the subject area is the area in the image where the target object of the user's attention is located, and the background area is the part outside the subject area. For example, in a portrait photo, the person is the subject area, and the image area outside the person area is the background area. This embodiment can use a trained image recognition model to identify people, plants, animals, vehicles, buildings, etc. in the image, and after identifying the target object in the image, extract the outline of the target object, determine the outline area of the target object as the subject area (that is, the first area), and determine the image area outside the subject area as the background area (that is, the second area). Alternatively, a foreground extraction algorithm can be used to extract the foreground area of the image, wherein the foreground area is the subject area of the image, and the image area outside the foreground area is the background area of the image. Of course, this application can also use other methods to extract the subject area and background area of the RAW image, and this embodiment is not limited to this.

[0042] As a third example, identifying the first area and the second area of a single-frame RAW image includes: obtaining N historical frame RAW images before the single-frame RAW image, where N is an integer greater than 1; obtaining the area of the single-frame RAW image that has changed relative to the historical frame RAW image, where the changed area is the first area, and the unchanged area is the second area.

[0043] Specifically, in a video scene, the area that the user focuses on is usually a dynamically changing area. Therefore, when performing LSC processing on an image in a video scene, the changed area in each frame of the image can be processed as the first area, giving priority to ensuring the processing effect of the changed area. In this embodiment, the previous frame or the previous N frames of the historical frame RAW image of the current frame RAW image can be obtained, and the current frame RAW image can be compared with the previous frame or the previous N frames of the historical frame RAW image of the current frame RAW image to determine the area of the current frame RAW image that has changed, and the changed area is determined as the first area, and the unchanged area is determined as the second area, where the value of N can be set according to actual conditions, and this embodiment does not impose any restrictions on this.

[0044] The image processing method provided in the above embodiment extracts the first and second regions in a single-frame RAW image by performing region recognition on the single-frame RAW image, thereby enhancing the LSC processing effect of the first region that the user is concerned about during image processing. Since the user's evaluation weight of the photo quality of an image is also mainly reflected in the region that the user is concerned about, the user experience can be improved by enhancing the LSC processing effect of the first region that the user is concerned about. In addition, for preview scenes and video scenes, this solution provides a real-time differentiated block processing and LSC processing solution. The difference in region recognition results between frames will lead to differences in block density in different regions, thereby enhancing the image processing effect of the first region that the user is concerned about in each frame, which can better meet the needs of users and improve the user experience.

[0045] Step S102 : performing block processing on the first region and the second region, wherein the block processing on the first region and the second region is based on different block processing strategies.

[0046] like Figure 4As shown, currently, during LSC processing, RAW images are typically segmented uniformly across the entire image. A LUT (Look Up Table) result for the entire image is calculated based on the uniform segmentation results. Subsequent LSC processing based on the LUT result ensures consistent image processing performance. However, because the uniform segmentation of the entire image results in the same processing accuracy across all image regions, the processing results fail to reflect the primary and secondary relationships within the entire image. Furthermore, under certain computing power requirements, in order to balance the overall image processing performance, the processing standards for image regions of primary interest to the user may be lowered. This results in the overall image processing performance meeting the requirements, but the processing performance for the user's key areas of interest deviates, reducing the user experience. Therefore, after identifying a first region and a second region of a single-frame RAW image, this embodiment employs different segmentation strategies based on the identification results to segment the first and second regions. This achieves differentiated processing for the first and second regions, improves the processing accuracy of the first region, and enhances the processing performance for the user's key areas of interest without compromising the overall processing performance.

[0047] As a first example, performing block processing on the first area and the second area includes: performing block processing on a single frame RAW image based on different image block densities of the first area and the second area, wherein the image block density of the first area is greater than the image block density of the second area.

[0048] Specifically, if Figure 5 As shown, in this embodiment, when the first area and the second area are processed in blocks, different image block densities can be set for different image areas. Image block density is the number of image blocks per unit area. The larger the image block density of an area, the smaller the area of each image block, and thus the fewer pixels in the image block. Since the subsequent LUT result calculation is based on the result of the image block, each image block only calculates the LUT result of the sampling point, and the LUT result of the pixel point in the image block is calculated by the sampling point using the bilinear interpolation method. Therefore, the greater the image block density and the smaller the image block, the more accurate the calculation result of the pixel point in the image block calculated using the bilinear interpolation method. In this embodiment, the image block size of the first area is configured to be smaller than the image block size of the second area, so that the image block density of the first area is greater than the image block density of the second area, thereby improving the pixel point processing accuracy of the first area and enhancing the image processing effect of the first area.

[0049] In a specific embodiment, block processing of a RAW image includes: setting multiple horizontal dividing lines along the pixel row direction and setting multiple vertical dividing lines along the pixel column direction, wherein the horizontal dividing lines and the vertical dividing lines divide the RAW image into multiple image blocks; configuring the distance between the horizontal dividing lines and the distance between the vertical dividing lines passing through the first area so that the image block density in the first area is greater than or equal to a first preset density value; configuring the distance between the horizontal dividing lines and the vertical dividing lines passing through the second area so that the image block density in the second area is less than the first preset density value.

[0050] Specifically, when a RAW image is segmented, the size and density of the image blocks are determined by the distance between the dividing lines. The smaller the distance between the dividing lines, the smaller the image blocks, the greater the density of the image blocks per unit area, and the more accurate the LUT calculation results for the pixels within the blocks. Therefore, when setting the dividing lines, the distance between the dividing lines can be determined based on the width and height of the entire image and the desired image block density.

[0051] like Figure 5 As shown, in this embodiment, assuming that the first area is located in the middle area of the image, the distance between the horizontal dividing lines and the distance between the vertical dividing lines in the middle area are smaller, and the distance between the dividing lines in the area outside the middle area is larger, so that the image block size in the middle area of the image is small and the density is high, and the image block size in the area outside the middle area is large and the density is low.

[0052] When the image block density reaches a first preset density value, the corresponding image processing effect is the best. Therefore, in this embodiment, the image block density of the first area can be set to be greater than or equal to the first preset density value, and the image block density of the second area can be set to be less than the first preset density value, so that the image processing effect of the first area is the best and the image processing effect of the second area is second, thereby enhancing the image processing effect of the first area.

[0053] Furthermore, a second preset density value can be set, and the second preset density value is less than the first preset density value. When the image block density reaches the first preset density value, the corresponding image processing effect is the best, and when the image block density reaches the second preset density value, a basic image processing effect can be achieved. When configuring the distance between the horizontal dividing line and the vertical dividing line, the image block density of the first area is greater than or equal to the first preset density value, and the image block density of the second area is less than the first preset density value and greater than or equal to the second preset density value, so that the second area can achieve a basic image processing effect and the first area achieves a better image processing effect. On the basis of ensuring the processing effect of the entire image, the processing effect of the first area is enhanced.

[0054] Of course, the present application can also divide the image into multiple areas and set the pixel block density of different areas according to the user's attention level to different areas. For example, the second area can be divided into a secondary focus area, a non-focus area, etc. according to the distance from the first area, the secondary focus area is closer to the first area, and the non-focus area is farther from the first area. When the image is segmented, the image block density of the first area is the largest, the image block density of the secondary focus area is the second largest, and the image block density of the non-focus area is the smallest.

[0055] In one embodiment, the number of image blocks when the RAW image is divided into blocks based on the different image block densities of the first area and the second area is less than or equal to the number of image blocks when the single-frame RAW image is uniformly divided into blocks using the first preset density value.

[0056] Specifically, the number of image blocks when a single-frame RAW image is evenly divided using a first preset density value is first calculated based on the width and height of the image. When configuring the distance between the dividing lines, after the pixel block density of the first area is configured to reach the first preset density value, the image block density value of the second area is reduced so that the processing effect of the second area can achieve a basic image processing effect, and the number of image blocks in the entire RAW image is smaller than the number of image blocks when the first preset density value is used for even division. This can, on the one hand, reduce the number of blocks, reduce the amount of LUT result data required for subsequent calculations, reduce the algorithm processing pressure, reduce the system's power consumption, and improve the processing speed. On the other hand, because the image block density of the first area enables the first area to achieve a better processing effect, the processing effect of the entire image meets the user's needs, and the processing effect of the first area exceeds the user's needs, which can improve the user experience.

[0057] As a second example, performing block processing on the first region and the second region includes: uniformly blocking a single-frame RAW image to obtain multiple original image blocks; converting the single-frame RAW image into a curved surface block image based on the distance from a pixel point on each original image block to a focal plane, wherein the distance from a pixel point in the original image block in the first region to the focal plane is smaller than the distance from a pixel point in the original image block in the second region to the focal plane; and projecting the curved surface block image in a direction perpendicular to the focal plane to obtain a projected image block, and using the projected image block as the image blocking result.

[0058] When the first area and the second area are divided according to the focal plane data, the scheme of this example can be used to divide the image into blocks. Specifically, the single-frame RAW image is first evenly divided into blocks, and the image block density can be set as needed. After the even division, multiple original image blocks are obtained. The distance from the pixel point on each original image block to the focal plane is obtained. In this embodiment, the distance between the point on each original image block and the center point of the focal plane can be calculated, and then the position coordinates of the pixel point on each original image block can be converted into the distance coordinates between the pixel point and the focal plane. Since the distance from each pixel point on the image block to the focal plane is different, the single-frame RAW image can be converted into a curved block image, such as Figure 6 As shown. Each surface block on the surface block image corresponds to an original image block on the RAW image. The surface block image is projected in a direction perpendicular to the focal plane to obtain a projected image block, which is the image segmentation result. It can be seen that the closer the original image block is to the focal plane, the smaller the distance between the pixel point on it and the center point of the focal plane, and the smaller the size of the corresponding projected image block. This makes the image block size in the area where the focal plane is located smaller, and the image block size farther away from the focal plane is larger, thereby improving the data processing accuracy of the focal plane and its surrounding areas, thereby improving the image processing effect.

[0059] Step S103: performing LUT calculation according to the image segmentation result to obtain a LUT result.

[0060] Currently, lens shading correction is performed separately for each of the four Bayer channels, with each channel's correction process being relatively independent. Considering chip design, data transmission, and processing costs, the LUT result for the entire image is typically not stored. Instead, the entire image is segmented and sampled, and the gains at the sampling points are stored. For example, the gains for 128*128 sampling points are stored. Therefore, this embodiment can perform sampling based on the image segmentation results and calculate the LUT results for the sampling points. The LUT calculation method is similar to that used in traditional techniques and will not be further described here.

[0061] Step S104 , performing LSC processing on the RAW image according to the LUT result to obtain a current frame image.

[0062] After the front-end image signal processing chip calculates the LUT result, it sends it to the back-end application processing chip, which then performs lens shading correction on the RAW image based on the LUT result. During correction, the back-end application processing chip uses bilinear interpolation to calculate the LUT result for each pixel in the image block based on the LUT result at the sampling point. It then performs lens shading correction on the RAW image based on the LUT results for all pixels, obtaining the preliminarily processed current frame image.

[0063] like Figure 7 As shown, on the front-end image processing chip 12 side, after image segmentation, the LUT result can be obtained based on the information extraction module. After obtaining the LUT result, the front-end image processing chip 12 also performs LTM (Local Tone Mapping) preprocessing on the image. It should be noted that the front-end image signal processing chip 20 will use the LUT result to perform LSC image brightness preprocessing in advance before performing LTM, and then use the LTM algorithm to adjust the image brightness and contrast. After adjustment, the LSC characteristics of the image are restored to the RAW image according to the LUT result, and then the preprocessed RAW image and LUT result are sent to the back-end application processing chip 13, so that the back-end application processing chip 13 performs LSC processing and other image algorithm processing on the image based on the LUT result to achieve the expected image effect.

[0064] In one embodiment, after performing LSC processing on the first area and the second area according to the LUT result, it also includes: based on the current frame image and the historical frame image associated with the current frame image; based on the historical frame image, correcting the area that is the same as the historical frame image to obtain a corrected current frame image.

[0065] In this embodiment, a historical frame image refers to a historical frame image stored in the image database and having undergone the processing flow from steps S101 to S104. When correcting a current frame image, the historical frame image most similar to the current frame image can be found in the image database as an associated historical frame image for the current frame image. During the search, the historical frame image most similar to the current frame image can be found based on feature points on the current frame image. The historical frame image can be a full frame image or a segmented image of the full frame image.

[0066] In the previous steps, since the blocking density of the first region was greater than that of the second region when the RAW image was segmented, the processing accuracy of the first region was also greater than that of the second region. Therefore, after LSC processing, the second region can be corrected based on the previously processed historical frame image to improve the processing effect of the second region. Specifically, the pixel data of the previously processed historical frame image can be assigned to the pixels in the same region of the current frame image to complete the correction of the current frame image. Of course, in addition to correcting the second region, the first region can also be enhanced and fused based on the historical frame image to further improve the image processing effect of the first region.

[0067] When making corrections based on historical frame images, the number of historical frame images used can be dynamically controlled according to the image effect of the current frame image, without having to use a fixed number of frames. For example, dynamic adjustments can be made based on changes in the scene and the characteristics of system power consumption and temperature rise, so that the system's processing pressure and image effect reach a more balanced level. Of course, the image effect meets user requirements. Alternatively, the number of historical frame images can be adjusted according to changes in the correction effect. As the number of historical frame images used increases, the effect of image correction will gradually improve. Therefore, more historical frame data can be used at the beginning of the correction, and the number of frames can be gradually reduced when a certain correction accuracy is achieved, so as to reduce the system's processing pressure while ensuring the correction effect.

[0068] After the current frame image is corrected, the current frame image is segmented, and the image segments obtained by segmentation are cached. In this embodiment, an equal division method can be used, for example, the image can be divided into image segments in a 3*3 manner, and the image segments are cached to serve as historical frame images and provide a basis for subsequent image correction. Alternatively, the image can be segmented according to feature extraction, extracting the main area and background area in the image, and saving the main area and background area separately to serve as historical frame images and provide a basis for subsequent image correction. Of course, it is also possible to not segment the corrected image and directly cache the entire frame image.

[0069] The image processing method provided in the above embodiment identifies the first area and the second area of a single-frame RAW image, performs block processing on the first area and the second area based on different block strategies for the first area and the second area, performs LUT calculation based on the block processing results to obtain the LUT result, and performs LSC processing on the first area and the second area based on the LUT result to obtain the current frame image, thereby identifying the first area of the image that the user is paying attention to, and performing differentiated processing on the first area and the second area, thereby improving the image processing effect of the first area that the user is paying attention to.

[0070] In addition, if Figure 8 As shown, one embodiment of the present application provides an image processing chip, including a first chip 110 and a second chip 120. The first chip 110 is used to identify a first region and a second region of a single-frame RAW image, and perform block processing on the first region and the second region, wherein the block processing of the first region and the second region is based on different block processing strategies, and a LUT calculation is performed on the first region and the second region based on the block processing results to obtain a LUT result. The second chip 120 is connected to the first chip 110 and is used to perform LSC processing on the first region and the second region based on the LUT result to obtain a current frame image.

[0071] like Figure 9As shown, in one embodiment, the first chip 110 includes a region identification module 111, which is used to perform region identification on a single-frame RAW image to obtain a first region and a second region.

[0072] In one embodiment, the region identification module 111 is specifically configured to divide the RAW image into regions according to the focal plane data to obtain a first region and a second region, wherein the first region is a focus region and the second region is an image region outside the first region.

[0073] In another embodiment, the region identification module 111 is specifically configured to perform content analysis on a single-frame RAW image to obtain a subject region and a background region of the single-frame RAW image, wherein the subject region is a first region and the background region is a second region.

[0074] In another embodiment, the region identification module 111 is specifically used to obtain the previous N historical RAW frames of a single-frame RAW image, where N is an integer greater than 1; and obtain the region of the single-frame RAW image that has changed relative to the historical RAW images, where the changed region is the first region, and the unchanged region is the second region.

[0075] like Figure 9 As shown, in one embodiment, the first chip 110 further includes an image blocking module 112, and the image blocking module 112 is configured to perform blocking processing on the first region and the second region based on different blocking strategies for the first region and the second region.

[0076] In one embodiment, the image blocking module 112 is specifically configured to perform blocking processing on the RAW image based on different image block densities in the first region and the second region, wherein the image block density in the first region is greater than that in the second region.

[0077] Specifically, the image segmentation module 112 is configured to set a plurality of horizontal dividing lines along the pixel rows and a plurality of vertical dividing lines along the pixel columns, wherein the horizontal dividing lines and the vertical dividing lines divide a single-frame RAW image into a plurality of image blocks. The distances between the horizontal dividing lines and the vertical dividing lines passing through the first region are configured so that the image block density within the first region is greater than or equal to a first preset density value. The distances between the horizontal dividing lines and the vertical dividing lines passing through the second region are configured so that the image block density within the second region is less than the first preset density value.

[0078] In one embodiment, the number of image blocks when a single-frame RAW image is divided into blocks based on different image block densities of the first region and the second region is less than or equal to the number of image blocks when the single-frame RAW image is uniformly divided into blocks using a first preset density value.

[0079] In another embodiment, the image blocking module 112 is specifically configured to uniformly block a single-frame RAW image to obtain multiple raw image blocks. The single-frame RAW image is converted into a curved surface block image based on the distance from a pixel in each raw image block to a focal plane, wherein the distance from a pixel in the raw image block within a first region to the focal plane is smaller than the distance from a pixel in the raw image block within a second region to the focal plane. The curved surface block image is projected in a direction perpendicular to the focal plane to obtain a projected image block, which is used as the image blocking result.

[0080] like Figure 9 As shown, in one embodiment, the first chip 110 further includes a processing module 113, and the processing module 113 is configured to perform LUT calculations on the first region and the second region respectively according to the block processing results to obtain LUT results.

[0081] like Figure 9 As shown, in one embodiment, the second chip 120 further includes a correction module 121, which is specifically used to obtain a historical frame image associated with the current frame image based on the current frame image, and to correct the area in the current frame image that is the same as the historical frame image based on the historical frame image to obtain a corrected current frame image.

[0082] like Figure 9 As shown, in one embodiment, the second chip 120 further includes a segmentation module 122 and a buffer module 123, wherein the segmentation module 122 is used to segment the corrected current frame image, and the buffer module 123 is used to buffer the segmentation results.

[0083] It should be noted that for the description of the image processing chip in this application, please refer to the description of the image processing method in this application, and the details will not be repeated here.

[0084] The image processing chip provided in the above embodiment identifies the first area and the second area of a single-frame RAW image through the first chip, and performs block processing on the first area and the second area based on different block processing strategies for the first area and the second area, and performs LUT calculation on the first area and the second area respectively according to the block processing results to obtain LUT results, and performs LSC processing on the first area and the second area respectively according to the LUT results through the second chip to obtain the current frame image, thereby identifying the first area of the image that the user is concerned about, distinguishing the first area and the second area, and improving the image processing effect of the first area that the user is concerned about.

[0085] Another embodiment of the present application provides an electronic device, comprising a memory, a processor, and an image processing program stored in the memory and executable on the processor. When the processor executes the image processing program, the aforementioned image processing method is implemented.

[0086] The electronic device described above can identify the first area of interest to the user in the image through the aforementioned image processing method, perform differentiated processing on the first area and the second area, and improve the image processing effect of the first area of interest to the user.

[0087] It should be noted that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic device), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing it in another suitable manner if necessary, and then storing it in a computer memory.

[0088] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0089] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0090] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of the technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0091] In the present invention, unless otherwise specified or limited, the terms "installed," "connected," "connect," "fixed," etc. should be understood in a broad sense. For example, they can refer to fixed connection, detachable connection, or integration; mechanical connection, electrical connection; direct connection, or indirect connection through an intermediate medium; internal communication between two components, or interaction between two components, unless otherwise specified. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0092] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A RAW image processing method, characterized in that: include: Identifying a first region and a second region of a single-frame RAW image; The first area is an image area that the user is interested in, and the second area is an image area outside the first area; performing block processing on the first area and the second area, wherein the block processing on the first area and the second area is based on different block processing strategies; Perform LUT calculation on the first area and the second area respectively according to the block processing result to obtain LUT results; Perform LSC processing on the first region and the second region respectively according to the LUT result to obtain a current frame image; The first area and the second area are divided into blocks, comprising: performing block processing on the RAW image based on different image block densities of the first region and the second region, wherein the image block density of the first region is greater than the image block density of the second region; or Evenly dividing the single-frame RAW image into blocks to obtain a plurality of original image blocks; Converting the single-frame RAW image into a curved block image according to a distance from a pixel point on each of the original image blocks to a focal plane, wherein the distance from a pixel point in the original image blocks within the first area to the focal plane is smaller than the distance from a pixel point in the original image blocks within the second area to the focal plane; The curved surface block image is projected along a direction perpendicular to the focal plane to obtain a projected image block, and the projected image block is used as the block processing result.

2. The RAW image processing method according to claim 1, wherein: The identifying of the first area and the second area of the single-frame RAW image includes: The single-frame RAW image is divided into regions according to the focal plane data to obtain the first region and the second region, wherein the first region is a focus region and the second region is an image region outside the first region.

3. The RAW image processing method according to claim 1, wherein: The identifying the first area and the second area of the single-frame RAW image includes: Content analysis is performed on the single-frame RAW image to obtain a subject area and a background area of the single-frame RAW image, wherein the subject area is the first area and the background area is the second area.

4. The RAW image processing method according to claim 1, wherein: The identifying of the first area and the second area of the single-frame RAW image includes: Acquire N historical RAW image frames preceding the single RAW image, where N is an integer greater than 1; A region of the single-frame RAW image that has changed relative to the historical frame RAW image is acquired, wherein the region that has changed is the first region, and the region that has not changed is the second region.

5. The RAW image processing method according to claim 1, wherein: The RAW image is subjected to block processing based on different image block densities of the first area and the second area, comprising: Setting a plurality of horizontal dividing lines along a pixel row direction and a plurality of vertical dividing lines along a pixel column direction, wherein the horizontal dividing lines and the vertical dividing lines divide the RAW image into a plurality of image blocks; configuring a distance between the horizontal dividing lines and a distance between the vertical dividing lines passing through the first area so that a density of image blocks in the first area is greater than or equal to a first preset density value; The distance between the horizontal dividing lines and the distance between the vertical dividing lines passing through the second area are configured so that the density of the image blocks in the second area is less than a first preset density value.

6. The RAW image processing method according to claim 5, characterized in that: The number of image blocks when the single-frame RAW image is divided into blocks based on the different image block densities of the first area and the second area is less than or equal to the number of image blocks when the single-frame RAW image is evenly divided into blocks using the first preset density value.

7. The RAW image processing method according to claim 1, wherein: After performing LSC processing on the first area and the second area respectively according to the LUT result, the method further includes: Acquire, according to the current frame image, a historical frame image associated with the current frame image; According to the historical frame image, the region in the current frame image that is the same as the historical frame image is corrected to obtain a corrected current frame image.

8. The RAW image processing method according to claim 7, characterized in that: Also includes: The corrected current frame image is segmented, and the segmentation results are cached.

9. A RAW image processing chip, characterized in that: include: a first chip, wherein the first chip is used to identify a first area and a second area of a single-frame RAW image; The first area is an image area of interest to a user, and the second area is an image area outside the first area; the first area and the second area are subjected to block processing, wherein the block processing of the first area and the second area is based on different block processing strategies, and LUT calculation is performed on the first area and the second area respectively according to the block processing results to obtain LUT results, wherein the block processing of the first area and the second area includes: performing block processing on the RAW image based on different image block densities of the first area and the second area, wherein the image block density of the first area is greater than the image block density of the second area; or uniformly blocking the single-frame RAW image to obtain multiple original image blocks; converting the single-frame RAW image into a curved surface block image according to the distance from the pixel point on each of the original image blocks to the focal plane, wherein the distance from the pixel point in the original image block in the first area to the focal plane is less than the distance from the pixel point in the original image block in the second area to the focal plane; projecting the curved surface block image along a direction perpendicular to the focal plane to obtain a projected image block, and using the projected image block as the block processing result; The second chip is connected to the first chip, and the second chip is used to perform LSC processing on the first area and the second area according to the LUT result to obtain a current frame image.

10. An electronic device, characterized in that: The image processing method comprises a memory, a processor, and an image processing program stored in the memory and executable on the processor. When the processor executes the image processing program, the image processing method according to any one of claims 1 to 8 is implemented.

Images