Multi-channel sparse MSFA layout demosaicing method and system and storage medium

Abstract

The invention relates to a multi-channel sparse MSFA layout demosaicing method and system and a storage medium. According to the method, demosaicing processing is mainly carried out on an image of a multi-channel sparse MSFA layout in the field of multi-spectral imaging, and interpolation processing is carried out on pixel value offsets of different pixel channel areas according to specific effective pixel and interpolation gap design of the multi-channel sparse MSFA layout. Moreover, the process can be executed in parallel according to the arrangement mode of the multi-pixel channels, and the image demosaicing processing of the multi-channel sparse MSFA layout can be quickly and accurately realized.

Description

Technical Field

[0001] The present invention is applicable to the field of multispectral imaging technology, and particularly relates to a multi-channel sparse MSFA layout demosaicking method, system and storage medium. Background Art

[0002] Digital color cameras generally can only sense three broad and overlapping bands in the visible electromagnetic spectrum, and there are limitations in characterizing the reflectivity of objects. To break through this limitation, multispectral cameras have emerged, which can provide multiple channels related to narrow bands and have wide applications in fields such as medical imaging, art research, and food safety.

[0003] In a multispectral camera, an image can be formed by sequentially selecting filters, but this structure is not suitable for dynamic scenes. For this reason, single-sensor multispectral cameras often use a multispectral filter array (abbreviated as MSFA) to image dynamic scenes. The MSFA layout design is determined based on the probability of appearance (abbreviated as PoA) of each channel pixel, as Figure 1 shown. Figure 1 shows a typical binary tree-based MSFA layout from 5 channels to 16 channels. Due to the nature of filter imaging, demosaicking processing is required for MSFA imaging to calculate and restore the missing information of each channel in order to obtain a complete multi-channel spectral image. Conventional demosaicking processing methods include bilinear interpolation, BTES (binary tree-based edge-sensing), PCBSD (PoA based convolution filter based bilinear spectral difference), LMMSE (linear minimum mean square error), interpolation algorithms based on RI (regularized interpolation), etc.

[0004] Sparse MSFA is a special layout design, which aims to expand the spectral sampling range or improve the spatial resolution without increasing the number of filters. Each pixel point corresponding to each filter in this layout can obtain more representative spectral information, which can reduce redundant sampling and improve the overall sampling efficiency. A comparison between a conventional 8-channel MSFA layout and a sparse MSFA layout is as Figure 2As shown by a and b in the figure, it can be seen that although the number of channels is the same, the known pixels in each channel of the sparse MSFA layout are not single pixels, but an overall rectangular area of 3×3. The interval between the known pixels also becomes a rectangular area. This means that the number of effective pixels for interpolation calculation becomes less, and the gap to be interpolated becomes larger, making it difficult for conventional interpolation algorithms to be used for demosaicing of multi-channel sparse MSFA layouts. Summary of the Invention

[0005] The present invention provides a method, a system and a storage medium for demosaicing a multi-channel sparse MSFA layout, aiming to solve the technical problem that existing methods are difficult to be used for demosaicing of multi-channel sparse MSFA layouts.

[0006] To solve the above technical problem, in a first aspect, the present invention provides a method for demosaicing a multi-channel sparse MSFA layout, including the following steps: S101. Obtain a captured image that conforms to a multi-channel sparse MSFA layout and is captured by a multi-spectral camera; S102. Divide the captured image into a plurality of channel regions according to pixel channels, and each of the channel regions is a rectangular region of pixel points including pixel values of the same pixel channel; S103. Take one of the divided channel regions as an interpolation region, and take another channel region adjacent to the interpolation region as a first reference region, and start to reconstruct the pixel points in the interpolation region; S104. Compare the edge pixel points of the interpolation region and the first reference region respectively to obtain a first brightness difference factor between the two regions; S105. Reconstruct all the pixel points in the interpolation region according to the first brightness difference factor; S106. Select another channel region adjacent to the interpolation region as the new first reference region, and return to step S104 until the traversal of the channel regions adjacent to the interpolation region is completed; S107. Define the channel regions in the captured image that are not adjacent to the interpolation region as second reference regions, and calculate a second brightness difference factor according to the gradient change direction of the surrounding pixels of the second reference regions; S108. Interpolate and fill all the pixel points in the interpolation region according to the second brightness difference factor to complete the reconstruction of the pixel points in the interpolation region; S109. Return to step S103 until the traversal of all the channel regions as the interpolation region is completed; S1010. Output the captured image with all pixel points reconstructed as its corresponding demosaiced image.

[0007] Furthermore, step S104 is specifically as follows: Calculate the average pixel value a of n pixel points on the adjacent side of the interpolation area and the first reference area, and the average pixel value b of n pixel points on the adjacent side of the first reference area and the interpolation area, where n is a positive integer. Define the first brightness difference factor as f1, and it satisfies the following conditions: f1 = b / a

[0008] Furthermore, step S105 is specifically as follows: Make the pixel value c of all pixel points in the interpolation area become the product of the pixel value d of the corresponding pixel point in the first reference area and the first brightness difference factor f1, that is, it satisfies the following conditions: c = d × f1

[0009] Furthermore, step S107 is specifically as follows: Calculate the average pixel value l of all pixel points in the second reference area, and the average pixel value mh of 2n pixel points adjacent to the left and right of the second reference area, where n is a positive integer. Define the second brightness difference factor as f2, and it satisfies the following conditions: f2 = l / mh; Or, calculate the average pixel value l of all pixel points in the second reference area, and the average pixel value mv of 2n pixel points adjacent to the top and bottom of the second reference area, where n is a positive integer. Define the second brightness difference factor as f2, and it satisfies the following conditions: f2 = l / mv

[0010] Furthermore, step S108 is specifically as follows: Make the pixel value c of all pixel points in the interpolation area become the product of the pixel value h of the corresponding pixel point in the second reference area and the second brightness difference factor f2, that is, it satisfies the following conditions: c = h × f2

[0011] Furthermore, in step S107, it also includes: Judge the magnitude relationship between the horizontal gradient dh between 2n pixel points adjacent to the left and right of the second reference area and the vertical gradient dv between 2n pixel points adjacent to the top and bottom of the second reference area, where: If dh < dv, then make the second brightness difference factor satisfy the following conditions: f2 = l / mh; If dh > dv, then make the second brightness difference factor satisfy the following conditions: f2 = l / mv

[0012] In a second aspect, the present invention further provides a multi-channel sparse MSFA layout demosaicing system, comprising: An acquisition module, configured to acquire an acquisition image that conforms to the multi-channel sparse MSFA layout and is acquired by a multi-spectral camera; A channel identification module, configured to divide the acquisition image into a plurality of channel regions according to pixel channels, and each of the channel regions is a rectangular region of pixel points including pixel values of the same pixel channel; A processing module, configured to start reconstructing pixel points in the interpolation region, with one of the divided channel regions as the interpolation region and another adjacent channel region to the interpolation region as the first reference region; A first difference calculation module, configured to compare the edge pixel points of the interpolation region and the first reference region respectively to obtain a first brightness difference factor between the two regions; A first reconstruction module, configured to reconstruct all pixel points in the interpolation region according to the first brightness difference factor; A first iteration module, configured to select another channel region adjacent to the interpolation region as the new first reference region and return to the first difference calculation module until traversal of the channel regions adjacent to the interpolation region is completed; A second difference calculation module, configured to determine a channel region in the acquisition image that is not adjacent to the interpolation region as the second reference region, and calculate a second brightness difference factor according to the gradient change direction of the pixels around the second reference region; A second reconstruction module, configured to perform interpolation filling on all pixel points in the interpolation region according to the second brightness difference factor to complete the reconstruction of pixel points in the interpolation region; A second iteration module, configured to return to the processing module until traversal of all channel regions as the interpolation region is completed; An output module, configured to output the acquisition image with all pixel points reconstructed as its corresponding demosaiced image.

[0013] In a third aspect, the present invention further provides a computer device, comprising: a memory, a processor, and a multi-channel sparse MSFA layout demosaicing program stored on the memory and executable on the processor, and when the processor executes the multi-channel sparse MSFA layout demosaicing program, it implements the steps in the multi-channel sparse MSFA layout demosaicing method as described in any one of the above embodiments.

[0014] Fourthly, the present invention also provides a storage medium, on which a multi-channel sparse MSFA layout demosaicing program is stored. When the multi-channel sparse MSFA layout demosaicing program is executed by a processor, the steps in the multi-channel sparse MSFA layout demosaicing method described in any one of the above embodiments are implemented.

[0015] The beneficial effects achieved by the present invention are as follows: A demosaicing method for an image with a multi-channel sparse MSFA layout in the multi-spectral imaging field is proposed. This method is designed for the effective pixels and interpolation gaps unique to the multi-channel sparse MSFA layout, performs interpolation processing on the pixel value offsets in different pixel channel regions, and this process can be executed in parallel according to the arrangement of multiple pixel channels, and can quickly and accurately implement the demosaicing process of an image with a multi-channel sparse MSFA layout. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a schematic diagram of a typical MSFA layout based on a binary tree from 5 channels to 16 channels in the prior art; Figure 2 is a comparison schematic diagram of a conventional MSFA layout and a sparse MSFA layout with 8 channels in the prior art; Figure 3 is a flowchart of the steps of the multi-channel sparse MSFA layout demosaicing method provided by an embodiment of the present invention; Figure 4 is a schematic diagram of an 8-channel sparse MSFA layout provided by an embodiment of the present invention; Figure 5 is a schematic diagram of the reconstruction of pixel points in the vertex direction in an 8-channel sparse MSFA layout provided by an embodiment of the present invention; Figure 6 is a schematic diagram of the reconstruction result of pixel points regarding pixel channel 1 in an 8-channel sparse MSFA layout provided by an embodiment of the present invention; Figure 7 is a schematic diagram of the structure of a multi-channel sparse MSFA layout demosaicing system provided by an embodiment of the present invention; Figure 8 is a schematic diagram of the structure of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0018] Please refer to Figure 3 , Figure 3It is a flowchart of the steps of the multi-channel sparse MSFA layout demosaicing method provided by an embodiment of the present invention. The flowchart of the steps of the multi-channel sparse MSFA layout demosaicing method includes the following steps: S101. Obtain a captured image that conforms to the multi-channel sparse MSFA layout and is captured by a multi-spectral camera.

[0019] S102. Divide the captured image into multiple channel regions according to pixel channels. Each of the channel regions is a rectangular region of pixel points that contain pixel values of the same pixel channel.

[0020] S103. Use one of the divided channel regions as an interpolation region, and another channel region adjacent to the interpolation region as a first reference region, and start reconstructing the pixel points in the interpolation region.

[0021] Exemplarily, an embodiment of the present invention uses an 8-channel sparse MSFA layout as an example to illustrate the above method steps. An 8-channel sparse MSFA layout is as Figure 4 shown. Among them, the occurrence probabilities of the channel pixels of each pixel channel are equal, so that the effective pixel layout is uniform. For the captured image, only the pixel coordinate offsets of each pixel channel are different. Figure 4 In, each grid represents a pixel point, and the number on it represents the known value of the pixel channel corresponding to the number at this pixel point. Therefore, for channel region 1, it is necessary to calculate the interpolation of the other 7 pixel channels. In an embodiment of the present invention, Figure 4 the pixel channel 1 in the central region boxed in is used as the interpolation region to start the process description of reconstructing pixel points.

[0022] S104. Compare the edge pixel points of the interpolation region and the first reference region respectively to obtain a first brightness difference factor between the two regions.

[0023] S105. Reconstruct all the pixel points in the interpolation region according to the first brightness difference factor.

[0024] Furthermore, step S104 is specifically: Calculate the average pixel value a of n pixel points on the adjacent side of the interpolation region and the first reference region, and the average pixel value b of n pixel points on the adjacent side of the first reference region and the interpolation region, where n is a positive integer. Define the first brightness difference factor as f1, and satisfy the following conditions: f1 = b / a.

[0025] Step S105 is specifically: The pixel values ​​c of all pixels in the interpolation area are changed to the product of the pixel values ​​d of the corresponding pixels in the first reference area and the first brightness difference factor f1, that is, the following conditions are satisfied: c=d×f1.

[0026] like Figure 4 As shown, the first reference area above the interpolation area where the pixel channel 1 is located is the channel area where the pixel channel 8 is located. In the embodiment of the present invention, as shown in FIG. Figure 4 The size of each channel area shown is 3×3, so in step S104, the average pixel value a is calculated for the three pixel points on the upper side of the pixel channel 1, and the average pixel value b is calculated for the three pixel points on the upper side of the pixel channel 8, thereby obtaining the first brightness difference factor f1 between the pixel channel 1 and the pixel channel 8; Furthermore, in step S105, interpolation calculation is performed on the pixel points at corresponding positions in pixel channel 1 (the corresponding positions refer to the positions of each pixel point in the 3×3 matrix) according to the pixel values ​​d of the pixel points at corresponding positions in pixel channel 8, thereby completing the pixel point reconstruction of pixel channel 1 based on pixel channel 8.

[0027] S106, selecting another channel region adjacent to the interpolation region as the new first reference region, and returning to step S104 until the traversal of the channel regions adjacent to the interpolation region is completed.

[0028] like Figure 4 As shown, other channel regions directly adjacent to pixel channel 1 also include pixel channels 3, 4, 5, 6, and 7. In step S106, pixel channel 1 is sequentially reconstructed with other pixel channels through traversal control. It can be understood that, since the data required for pixel reconstruction between different pixel channels do not affect each other, steps S104-S105 can also be performed in parallel on the pixel reconstruction process between pixel channel 1 and all other directly adjacent channel regions, thereby improving the execution efficiency of the method proposed in the embodiment of the present invention.

[0029] In the embodiment of the present invention, the first reference area in the vertex direction adjacent to the interpolation area is particularly described. Figure 5 As shown, pixel channel 3 is located at Figure 4The upper left corner of pixel channel 1 indicated as the interpolation region. At the same time, pixel channel 3 is also at the lower right of other pixel channels 1. In this case, in step S104, the calculation of the first luminance difference factor can be based on the average pixel values of 3 pixel points at the upper left corner and the lower right corner in pixel channel 3 to calculate the average pixel values at the corresponding positions in pixel channel 1. At this time, two first luminance difference factors (corresponding to the upper left corner and the lower right corner) can be obtained; in step S105, according to the pixel point positions corresponding to the calculation of the first luminance difference factor, the pixel points in pixel channel 1 are reconstructed. For the 3 pixel points on the diagonal of pixel channel 1, since they do not correspond to the calculation positions of any first luminance difference factors, the average value of the two first luminance difference factors calculated corresponding to the upper left corner and the lower right corner can be used for pixel point reconstruction processing. The processing method of pixel channel 4 can also be carried out in a similar way.

[0030] After step S106 is completed, the pixel point reconstruction result of pixel channel 1 in the 8-channel sparse MSFA layout is as Figure 6 shown. At this time, pixel channel 1 has completed the interpolation calculation for pixel channels 3, 4, 5, 6, 7, and 8. Pixel channel 2 is not adjacent to pixel channel 1, so further processing of non-adjacent pixel channels is still required.

[0031] S107. Define the channel region in the acquired image that is not adjacent to the interpolation region as the second reference region, and calculate the second luminance difference factor according to the gradient change direction of the surrounding pixels of the second reference region.

[0032] S108. Perform interpolation filling on all pixel points in the interpolation region according to the second luminance difference factor to complete the pixel point reconstruction of the interpolation region.

[0033] Furthermore, step S107 is specifically as follows: Step S107 is specifically as follows: Calculate the average pixel value l of all pixel points in the second reference region, and the average pixel value mh of 2n pixel points adjacent to the left and right of the second reference region, where n is a positive integer. Define the second luminance difference factor as f2, and it satisfies the following conditions: f2 = l / mh; Or, calculate the average pixel value l of all pixel points in the second reference region, and the average pixel value mv of 2n pixel points adjacent to the upper and lower of the second reference region, where n is a positive integer. Define the second luminance difference factor as f2, and it satisfies the following conditions: f2 = l / mv.

[0034] Step S108 is specifically as follows: Change the pixel value c of all pixel points in the interpolation region to the product of the pixel value h of the corresponding pixel point in the second reference region and the second luminance difference factor f2, that is, the following condition is satisfied: c = h × f2.

[0035] As Figure 6 shown, due to the design of the multi-channel sparse MSFA layout, in the 8-channel sparse MSFA layout, pixel channel 2 is not adjacent to pixel channel 1, and there is at least one channel region between them. Therefore, it is impossible to perform difference calculation according to adjacent pixel points in the manner of steps S104 - S105. In the embodiments of the present invention, for non-adjacent second reference regions, horizontal interpolation or vertical interpolation is used for processing.

[0036] Specifically, in step S107, the following steps are further included: Judge the magnitude relationship between the horizontal gradient dh between 2n pixel points adjacent to the left and right of the second reference region, and the vertical gradient dv between 2n pixel points adjacent to the top and bottom of the second reference region, where: If dh < dv, then make the second luminance difference factor satisfy the following condition: f2 = l / mh; If dh > dv, then make the second luminance difference factor satisfy the following condition: f2 = l / mv.

[0037] The numerical magnitudes of the horizontal gradient and the vertical gradient directly reflect the texture direction and the degree of change intensity of the local region. Selecting the gradient with the smaller numerical value to determine the interpolation direction can better preserve edge details while reducing blurring or artifacts. Of course, the method for determining the interpolation direction is optional. According to the usage scenario of the multispectral camera, during implementation, the interpolation direction can be selected according to actual needs.

[0038] S109. Return to step S103 until all the channel regions are traversed as the interpolation region.

[0039] S1010. Output the acquired image with all pixel points reconstructed as its corresponding demosaiced image.

[0040] After step S108 is completed, as Figure 4 shown, all pixel points in pixel channel 1 have the pixel information of pixel channels 2, 3, 4, 5, 6, 7, and 8, that is, the reconstruction of the pixel points in pixel channel 1 is completed. According to the method of steps S103 - S108, other pixel channels are processed, and then the reconstruction process of all pixel points in the acquired image can be completed, and the corresponding demosaiced image can be obtained.

[0041] The beneficial effects achieved by the present invention are as follows: a method for demosaicking images with a multi-channel sparse MSFA layout in the field of multi-spectral imaging is proposed. This method is designed for the effective pixels and interpolation gaps unique to the multi-channel sparse MSFA layout, interpolates the pixel value offsets in different pixel channel regions, and this process can be executed in parallel according to the arrangement of multiple pixel channels, enabling fast and accurate demosaicking processing of images with a multi-channel sparse MSFA layout.

[0042] An embodiment of the present invention also provides a multi-channel sparse MSFA layout demosaicking system 200. Please refer to Figure 7 , Figure 7 which is a schematic structural diagram of the multi-channel sparse MSFA layout demosaicking system provided by an embodiment of the present invention. It includes: An acquisition module 201, configured to acquire an acquisition image that conforms to a multi-channel sparse MSFA layout through a multi-spectral camera; A channel recognition module 202, configured to divide the acquisition image into multiple channel regions according to pixel channels, and each of the channel regions is a rectangular region of pixel points containing pixel values of the same pixel channel; A processing module 203, configured to start reconstructing pixel points in the interpolation region with one of the divided channel regions as the interpolation region and another adjacent channel region to the interpolation region as the first reference region; A first difference calculation module 204, configured to compare the edge pixel points of the interpolation region and the first reference region respectively to obtain a first brightness difference factor between the two regions; A first reconstruction module 205, configured to reconstruct all pixel points in the interpolation region according to the first brightness difference factor; A first iteration module 206, configured to select another channel region adjacent to the interpolation region as the new first reference region and return to the first difference calculation module until traversal of the channel regions adjacent to the interpolation region is completed; A second difference calculation module 207, configured to determine the channel regions in the acquisition image that are not adjacent to the interpolation region as the second reference region, and calculate a second brightness difference factor according to the gradient change direction of the pixels around the second reference region; A second reconstruction module 208, configured to perform interpolation filling on all pixel points in the interpolation region according to the second brightness difference factor to complete the reconstruction of pixel points in the interpolation region; A second iteration module 209, configured to return to the processing module until traversal of all channel regions as the interpolation region is completed; The output module 210 is configured to output the acquired image that has completed the reconstruction of all pixel points as its corresponding demosaicked image.

[0043] The multi-channel sparse MSFA layout demosaicking system 200 can implement the steps in the multi-channel sparse MSFA layout demosaicking method in the above embodiments, and can achieve the same technical effects. Refer to the description in the above embodiments, and details are not repeated here.

[0044] An embodiment of the present invention further provides a computer device. Please refer to Figure 8 , Figure 8 which is a schematic structural diagram of the computer device provided by the embodiment of the present invention. The computer device 300 includes: a memory 302, a processor 301, and a multi-channel sparse MSFA layout demosaicking program stored on the memory 302 and executable on the processor 301.

[0045] The processor 301 calls the multi-channel sparse MSFA layout demosaicking program stored in the memory 302 to execute the steps in the multi-channel sparse MSFA layout demosaicking method provided by the embodiment of the present invention. Please refer to Figure 3 , specifically including the following steps: S101. Obtain an acquired image that conforms to the multi-channel sparse MSFA layout and is acquired by a multi-spectral camera.

[0046] S102. Divide the acquired image into multiple channel regions according to pixel channels. Each channel region is a rectangular region of pixel points containing pixel values of the same pixel channel.

[0047] S103. Use one of the divided channel regions as an interpolation region, and another channel region adjacent to the interpolation region as a first reference region, and start reconstructing the pixel points in the interpolation region.

[0048] S104. Compare the edge pixel points of the interpolation region and the first reference region respectively to obtain a first brightness difference factor between the two regions.

[0049] S105. Reconstruct all pixel points in the interpolation region according to the first brightness difference factor.

[0050] Furthermore, step S104 is specifically: Calculate the average pixel value a of n pixel points on the adjacent side of the interpolation region and the first reference region, and the average pixel value b of n pixel points on the adjacent side of the first reference region and the interpolation region, where n is a positive integer. Define the first brightness difference factor as f1, and it satisfies the following condition: f1 = b / a.

[0051] Further, step S105 is specifically as follows: Make the pixel value c of all pixel points in the interpolation area become the product of the pixel value d of the pixel point at the corresponding position in the first reference area and the first brightness difference factor f1, that is, satisfy the following condition: c = d × f1.

[0052] S106. Select another channel area adjacent to the interpolation area as the new first reference area, and return to step S104 until the traversal of the channel areas adjacent to the interpolation area is completed.

[0053] S107. Determine the channel areas in the acquired image that are not adjacent to the interpolation area as the second reference area, and calculate the second brightness difference factor according to the gradient change direction of the pixels around the second reference area.

[0054] S108. Perform interpolation filling on all pixel points in the interpolation area according to the second brightness difference factor to complete the reconstruction of the pixel points in the interpolation area.

[0055] Further, step S107 is specifically as follows: Calculate the average pixel value l of all pixel points in the second reference area, and the average pixel value mh of 2n pixel points adjacent to the left and right of the second reference area, where n is a positive integer. Define the second brightness difference factor as f2, and satisfy the following condition: f2 = l / mh; Or, calculate the average pixel value l of all pixel points in the second reference area, and the average pixel value mv of 2n pixel points adjacent to the top and bottom of the second reference area, where n is a positive integer. Define the second brightness difference factor as f2, and satisfy the following condition: f2 = l / mv.

[0056] Further, step S108 is specifically as follows: Make the pixel value c of all pixel points in the interpolation area become the product of the pixel value h of the pixel point at the corresponding position in the second reference area and the second brightness difference factor f2, that is, satisfy the following condition: c = h × f2.

[0057] Further, in step S107, it also includes: Judge the magnitude relationship between the horizontal gradient dh between 2n pixel points adjacent to the left and right of the second reference area and the vertical gradient dv between 2n pixel points adjacent to the top and bottom of the second reference area, where: If dh < dv, then make the second luminance difference factor satisfy the following condition: f2 = l / mh; If dh > dv, then make the second luminance difference factor satisfy the following condition: f2 = l / mv.

[0058] S109. Return to step S103 until all the channel regions are traversed as the interpolation regions.

[0059] S1010. Output the acquired image with all pixel points reconstructed as its corresponding demosaicked image.

[0060] The computer device 300 provided by the embodiments of the present invention can implement the steps in the multi-channel sparse MSFA layout demosaicking method in the above embodiments, and can achieve the same technical effects. Refer to the description in the above embodiments, and details are not described herein again.

[0061] The embodiments of the present invention further provide a storage medium, on which a multi-channel sparse MSFA layout demosaicking program is stored. When the multi-channel sparse MSFA layout demosaicking program is executed by a processor, it implements each process and step in the multi-channel sparse MSFA layout demosaicking method provided by the embodiments of the present invention, and can achieve the same technical effects. To avoid repetition, details are not described herein again.

[0062] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by a multi-channel sparse MSFA layout demosaicking program to instruct relevant hardware (which can be a mobile phone, a computer, a server, or a network device, etc.). The program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc.

[0063] It should be noted that in this article, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including that element.

[0064] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. What is disclosed is only the preferred embodiments of the present invention. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many equivalent changes in form without departing from the spirit of the present invention and the scope protected by the claims, and all of them fall within the protection scope of the present invention.

Claims

1. A multi-channel sparse MSFA layout demosaicing method, characterized in that, It includes the following steps: S101. Obtain a captured image that conforms to the multi-channel sparse MSFA layout collected by a multi-spectral camera; S102. Divide the captured image into multiple channel regions according to pixel channels, and each of the channel regions is a rectangular region of pixel points containing pixel values of the same pixel channel; S103. Use one of the divided channel regions as an interpolation region, and another channel region adjacent to the interpolation region as a first reference region, and start reconstructing the pixel points in the interpolation region; S104. Compare the edge pixel points of the interpolation region and the first reference region respectively to obtain a first brightness difference factor between the two regions; S105. Reconstruct all the pixel points in the interpolation region according to the first brightness difference factor; S106. Select another channel region adjacent to the interpolation region as the new first reference region, and return to step S104 until the traversal of the channel regions adjacent to the interpolation region is completed; S107. Define the channel regions in the captured image that are not adjacent to the interpolation region as second reference regions, and calculate a second brightness difference factor according to the gradient change direction of the surrounding pixels of the second reference region; S108. Perform interpolation filling on all the pixel points in the interpolation region according to the second brightness difference factor to complete the reconstruction of the pixel points in the interpolation region; S109. Return to step S103 until the traversal of all the channel regions as the interpolation region is completed; S1010. Output the captured image with all pixel points reconstructed as its corresponding demosaiced image.

2. The multi-channel sparse MSFA layout demosaicing method according to claim 1, characterized in that Step S104 is specifically as follows: Calculate the average pixel value a of n pixel points on the adjacent side of the interpolation region and the first reference region, and the average pixel value b of n pixel points on the adjacent side of the first reference region and the interpolation region, where n is a positive integer. Define the first brightness difference factor as f1, and it satisfies the following conditions: f1 = b / a.

3. The multi-channel sparse MSFA layout demosaicing method according to claim 2, characterized in that, Step S105 is specifically as follows: Make the pixel value c of all pixel points in the interpolation region become the product of the pixel value d of the corresponding position pixel point in the first reference region and the first brightness difference factor f1, that is, it satisfies the following conditions: c = d×f1.

4. The multi-channel sparse MSFA layout demosaicing method according to claim 1, characterized in that Step S107 is specifically as follows: Calculate the average pixel value l of all pixel points in the second reference region, and the average pixel value mh of 2n pixel points adjacent to the left and right of the second reference region, where n is a positive integer. Define the second brightness difference factor as f2, and it satisfies the following conditions: f2 = l / mh; Or, calculate the average pixel value l of all pixel points in the second reference region, and the average pixel value mv of 2n pixel points adjacent to the top and bottom of the second reference region, where n is a positive integer. Define the second brightness difference factor as f2, and it satisfies the following conditions: f2 = l / mv.

5. The multi-channel sparse MSFA layout demosaicing method according to claim 4, wherein Step S108 is specifically as follows: Make the pixel value c of all pixel points in the interpolation region become the product of the pixel value h of the pixel point at the corresponding position in the second reference region and the second brightness difference factor f2, that is, satisfy the following condition: c = h × f2.

6. The multi-channel sparse MSFA layout demosaicking method according to claim 4, characterized in that In step S107, it further includes: Judge the magnitude relationship between the horizontal gradient dh between 2n pixel points adjacent to the left and right of the second reference region and the vertical gradient dv between 2n pixel points adjacent to the top and bottom of the second reference region, where: If dh < dv, make the second brightness difference factor satisfy the following condition: f2 = l / mh; If dh > dv, make the second brightness difference factor satisfy the following condition: f2 = l / mv.

7. A multi-channel sparse MSFA layout demosaicing system, characterized in that, It includes: An acquisition module, configured to acquire an acquisition image that conforms to a multi-channel sparse MSFA layout acquired by a multi-spectral camera; A channel identification module, configured to divide the acquisition image into multiple channel regions according to pixel channels, and each of the channel regions is a rectangular region of pixel points including pixel values of the same pixel channel; A processing module, configured to start reconstructing pixel points in the interpolation region with one of the divided channel regions as the interpolation region and another channel region adjacent to the interpolation region as the first reference region; A first difference calculation module, configured to compare the edge pixel points of the interpolation region and the first reference region respectively to obtain a first brightness difference factor between the two regions; A first reconstruction module, configured to reconstruct all pixel points in the interpolation region according to the first brightness difference factor; A first iteration module, configured to select another channel region adjacent to the interpolation region as the new first reference region and return to the first difference calculation module until the traversal of the channel regions adjacent to the interpolation region is completed; A second difference calculation module, configured to define the channel region in the acquisition image that is not adjacent to the interpolation region as the second reference region, and calculate a second brightness difference factor according to the gradient change direction of the surrounding pixels of the second reference region; A second reconstruction module, configured to perform interpolation filling on all pixel points in the interpolation region according to the second brightness difference factor to complete the reconstruction of pixel points in the interpolation region; A second iteration module, configured to return to the processing module until the traversal of all channel regions as the interpolation region is completed; An output module, configured to output the acquisition image with all pixel points reconstructed as its corresponding demosaiced image.

8. A computer device, characterized in that, It includes: A memory, a processor, and a multi-channel sparse MSFA layout demosaicing program stored on the memory and executable on the processor. When the processor executes the multi-channel sparse MSFA layout demosaicing program, it implements the steps in the multi-channel sparse MSFA layout demosaicing method described in any one of claims 1-6.

9. A storage medium, characterized in that, The storage medium stores a multi-channel sparse MSFA layout demosaicing program, and when the multi-channel sparse MSFA layout demosaicing program is executed by a processor, it implements the steps in the multi-channel sparse MSFA layout demosaicing method described in any one of claims 1-6.

Images