Anti-illumination interference hyperspectral image segmentation extraction method, device and system

Abstract

The application discloses a kind of anti-light interference hyperspectral image segmentation extraction method, equipment and system, it is related to image data processing and computer vision technical field, the method includes obtaining the bottom layer original image of camera output;According to microlens array arrangement, fixed step staggered sampling is carried out pixel-level demultiplexing, and single-band image is extracted and stacked into three-dimensional data cube;Along spectral channel, pixel mean is calculated, and two-dimensional fusion reference map is obtained by linear stretching;Acquire target analysis area framing frame, the highest gray outside frame is generated whitening shielding map;In frame, preliminary target mask is obtained by executing inverse Otsu threshold algorithm, and final target mask is generated by morphological operation and connected domain analysis;Mask is mapped back to cube, and the gray mean of each band in effective area is calculated, and target spectral curve is obtained by summarizing.The application can realize anti-light interference, accurate segmentation target area and stable extraction spectral characteristics.

Description

Technical Field

[0001] This application relates to the fields of image data processing and computer vision technology, and in particular to a method, device and system for segmenting and extracting hyperspectral images that is resistant to illumination interference. Background Technology

[0002] Hyperspectral imaging technology divides light into dozens of continuous narrow bands to acquire the response characteristics of objects in different spectral channels, generating "spectral curves" for material composition identification. It is widely used in agricultural monitoring, material sorting, medical testing, and low-light target identification. Snapshot hyperspectral cameras (such as those with a 4×4 microlens array) can acquire information from all spectral channels in a single exposure, overcoming the limitation of traditional pushbroom cameras in capturing moving objects. The core challenge in this field is how to accurately reconstruct multiple spatially aligned single-band images from a single high-resolution snapshot hyperspectral camera, and then accurately segment the target for spectral extraction.

[0003] Existing technologies for snapshot hyperspectral image processing still have the following drawbacks: ① Existing methods employ macroscopic region segmentation, dividing the RAW image into equal blocks as images for each band. For example, if the resolution of the underlying raw image data is 4096×4096, and the microlens array of the snapshot hyperspectral camera is 4×4, then existing methods divide the entire image proportionally into 16 macroscopic regions (each region being 1024×1024 in size), and directly treat each region as a single-band channel image. All regions are then stacked along the spectral channel direction to form a three-dimensional data cube. This coarse-grained segmentation completely ignores the actual staggered arrangement of the microlens array, causing the spatial coordinates of each single-band channel image to shift, making pixel-level alignment impossible. This results in subsequent target recognition and spectral extraction losing their spatial consistency basis. ② Single narrowband light throughput is low, and the signal-to-noise ratio of a single-channel image is poor. Existing technologies directly rely on a single channel to identify targets, which easily produces holes and spikes, causing complete targets to be misjudged as multiple fragmented connected domains. ③ Optical imaging has vignetting. When using the global adaptive threshold segmentation method to extract dark targets, vignetting is easily misidentified as a target, lowering the threshold and causing the true target to be truncated and lost, resulting in a large amount of false noise. ④ Existing methods use the ratio of target gray level to background gray level, i.e., the relative reflectance normalization method, to evaluate the spectrum. When the target reflectance is extremely low and the background is bright, the spectral absorption characteristics of the weak target are compressed to a near-flat line, losing their distinctiveness. Summary of the Invention

[0004] The purpose of this application is to provide a method, device and system for segmenting and extracting hyperspectral images that is resistant to illumination interference. This method can effectively resist the interference of uneven illumination and ambient light intensity fluctuations, accurately segment the target area and restore its intrinsic spectral response characteristic curve with high fidelity.

[0005] To achieve the above objectives, this application provides the following solution: In a first aspect, this application provides a method for segmenting and extracting hyperspectral images that resists illumination interference, including: Acquire the underlying raw image data output by a snapshot hyperspectral camera.

[0006] Based on the physical arrangement of the microlens array in the snapshot hyperspectral camera, the underlying raw image data is demultiplexed at the pixel level using a fixed-step staggered sampling method to extract several single-band channel images. All the single-band channel images are then stacked along the spectral channel direction to generate a three-dimensional data cube.

[0007] The mean pixel intensity at each spatial coordinate is calculated for the three-dimensional data cube along the spectral channel direction, and all the mean values ​​are linearly stretched to a preset grayscale range to obtain a two-dimensional fusion reference image.

[0008] A viewfinder containing the target to be tested is obtained, and the gray values ​​of the pixels located outside the viewfinder in the two-dimensional fusion reference image are assigned the highest gray values ​​to generate a whitening masking image.

[0009] Using the viewfinder of the analysis area as a constraint, the reverse Otsu thresholding algorithm is executed in the whitening masking image to generate a preliminary target mask.

[0010] Morphological operations are performed on the preliminary target mask to obtain the mask after morphological operations.

[0011] Two-dimensional connected component analysis is performed on the mask after morphological operations are completed. Connected components with an area smaller than a preset filtering threshold are identified as pseudo-feature blocks and removed to generate the final target mask.

[0012] Using the final target mask as an index, after mapping it back to the three-dimensional data cube in situ, the absolute original grayscale mean of each pixel in each band channel is calculated within the effective area covered by the final target mask.

[0013] The absolute original grayscale mean values ​​of each pixel in each band channel are summarized, and all the absolute original grayscale mean values ​​are sent to the computer to generate the intrinsic spectral response characteristic curve of the target under test.

[0014] In a second aspect, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the anti-light interference hyperspectral image segmentation and extraction method described above.

[0015] Thirdly, this application provides a hyperspectral image segmentation and extraction system resistant to illumination interference, comprising: A snapshot hyperspectral camera is used to acquire raw, underlying image data.

[0016] A processor for executing the illumination-resistant hyperspectral image segmentation and extraction method described in any of the above-mentioned embodiments.

[0017] A computer is used to generate the intrinsic spectral response characteristic curve of the target under test based on all absolute raw grayscale averages sent by the processor.

[0018] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides a method, device, and system for segmenting and extracting hyperspectral images with resistance to illumination interference. It abandons the equal-division block method and instead strictly adheres to the physical arrangement of the microlens array in a snapshot hyperspectral camera, employing a pixel-level demultiplexing strategy with staggered sampling at a fixed step size. This design ensures that the extracted single-band channel images precisely correspond to the same object-side field of view in space, giving the three-dimensional data cube generated by stacking along the spectral channels a real physical spatial meaning, thus guaranteeing the spatial consistency of the spectral data from the source. Secondly, a multi-band fusion noise reduction mechanism is constructed: by calculating the mean pixel intensity at each spatial coordinate along the spectral channel direction and linearly stretching it to a preset grayscale range, a high signal-to-noise ratio two-dimensional fusion reference map is generated. This fusion reference map integrates the light intensity information of the entire spectrum, effectively suppressing single-channel random noise, providing a robust data foundation for subsequent segmentation, and avoiding holes, spikes, and broken connected components caused by poor single-channel quality. Secondly, by acquiring a viewfinder containing the target area, the grayscale values ​​of the dark corner regions outside the frame are forcibly assigned the highest grayscale values. Therefore, when executing the reverse Otsu thresholding algorithm, since the algorithm itself uses the lowest grayscale pixel group as the target class and the highest grayscale pixel group as the background class, the whitened dark corner regions are automatically classified as the background class and no longer participate in the target class threshold calculation. This effectively prevents dark corner regions from lowering the global segmentation threshold, avoiding the truncation and loss of truly weak targets due to excessively low thresholds, and significantly suppressing false noise generated by dark corners. Finally, after generating the final target mask, this method uses the mask as an index to map back to the original 3D data cube in situ, directly calculating the absolute original grayscale mean of each pixel in each band channel within the effective area covered by the mask, rather than calculating the ratio to the background. This absolute quantity extraction method without normalization fully preserves the spectral absorption depth of the weak target itself, avoiding the stretching and compression effect of the bright background on the target spectral curve, thereby significantly improving the spectral recognition of low-reflectivity targets. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is an application environment diagram of a hyperspectral image segmentation and extraction method resistant to illumination interference according to an embodiment of this application; Figure 2 A flowchart illustrating a hyperspectral image segmentation and extraction method resistant to illumination interference provided in an embodiment of this application; Figure 3 A schematic diagram of a hyperspectral image segmentation and extraction system resistant to illumination interference provided in an embodiment of this application; Figure 4 A schematic diagram of an electronic device provided in an embodiment of this application; Figure 5 This is a schematic diagram of an electronic device processor structure provided in an embodiment of this application; Figure 6 This is a schematic diagram of the staggered sampling principle provided in an embodiment of this application; Figure 7 This is a schematic diagram illustrating the principle of whitening shielding and reverse Otsu threshold segmentation in one embodiment of this application. Figure 7 (a) in the diagram is a schematic of the ROI before whitening constraint treatment; Figure 7 (b) in the diagram is a schematic diagram of the ROI after whitening constraint treatment; Figure 8 A comparison diagram of the absolute original grayscale mean and the normalized spectral curve of relative reflectance provided for an embodiment of this application; Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0021] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0022] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0023] The anti-light interference hyperspectral image segmentation and extraction method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be set up separately, integrated into server 104, or placed in the cloud or on other servers. Terminal 102 can send the underlying raw image data output by the snapshot hyperspectral camera to server 104. After receiving the underlying raw image data output by the snapshot hyperspectral camera, server 104, based on the physical arrangement of the microlens array in the snapshot hyperspectral camera, performs pixel-level demultiplexing of the underlying raw image data using a fixed-step staggered sampling method, extracting several single-band channel images. All the single-band channel images are stacked along the spectral channel direction to generate a three-dimensional data cube. The mean pixel intensity at each spatial coordinate of the three-dimensional data cube is calculated along the spectral channel direction, and all the mean values ​​are linearly stretched to a preset grayscale range to obtain a two-dimensional fusion reference image. A viewfinder containing the target to be measured is obtained, and the image outside the viewfinder of the two-dimensional fusion reference image is... The highest gray value is assigned to each pixel to generate a whitening mask. Using the analysis area frame as a constraint, the reverse Otsu thresholding algorithm is executed on the whitening mask to generate a preliminary target mask. Morphological operations are performed on the preliminary target mask to obtain a morphologically processed mask. Two-dimensional connected component analysis is performed on the morphologically processed mask, and connected components with areas smaller than a preset filtering threshold are identified as pseudo-feature blocks and removed to generate the final target mask. Using the final target mask as an index, it is mapped back to the three-dimensional data cube in situ. Within the effective area covered by the final target mask, the absolute original gray value of each pixel in each band channel is calculated. The absolute original gray values ​​of each pixel in each band channel are summarized, and all the absolute original gray values ​​are sent to the computer to generate the intrinsic spectral response characteristic curve of the target under test. The server 104 can feed back the obtained intrinsic spectral response characteristic curve of the target under test to the terminal 102. In addition, in some embodiments, the anti-light-interference hyperspectral image segmentation and extraction method can also be implemented by the server 104 or the terminal 102 alone. For example, the terminal 102 can directly process the underlying raw image data output by the snapshot hyperspectral camera, or the server 104 can obtain the underlying raw image data output by the snapshot hyperspectral camera from the data storage system and process it.

[0024] The terminal 102 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted devices. The server 104 can be implemented using a standalone server or a server cluster composed of multiple servers, or it can be a cloud server.

[0025] In one exemplary embodiment, such as Figure 2 As shown, a method for segmenting and extracting hyperspectral images resistant to illumination interference is provided. This method is executed by a computer device, specifically by a terminal or server alone, or by both a terminal and a server. In this embodiment, the method is applied to... Figure 1 Taking server 104 as an example, the explanation includes the following steps 201 to 209. Wherein: Step 201: Obtain the underlying raw image data output by the snapshot hyperspectral camera, such as RAW image data.

[0026] Step 202: Based on the physical arrangement of the microlens array in the snapshot hyperspectral camera, the underlying raw image data is demultiplexed at the pixel level using a fixed-step staggered sampling method to extract several single-band channel images. All the single-band channel images are then stacked along the spectral channel direction to generate a three-dimensional data cube. The microlens array in the snapshot hyperspectral camera can be a 4×4 physical optical structure.

[0027] This step uses step sampling that matches the hardware to replace traditional block cutting, solving the problems of "misalignment" and "ghosting" in the spatial dimension of multispectral channel images.

[0028] Step 203: Calculate the mean value of pixel intensity at each spatial coordinate along the spectral channel direction for the three-dimensional data cube, and linearly stretch all the mean values ​​to a preset grayscale range to obtain a two-dimensional fusion reference image.

[0029] Step 204: Obtain the viewfinder of the analysis region containing the target to be tested, and assign the grayscale value of the pixel outside the viewfinder of the analysis region in the two-dimensional fusion reference image to the highest grayscale value to generate a whitening masking image.

[0030] Step 205: Using the viewfinder of the analysis area as a constraint, perform the reverse Otsu thresholding algorithm in the whitening masking image to generate a preliminary target mask.

[0031] Step 206: Perform morphological operations on the preliminary target mask to obtain the mask after morphological operations.

[0032] Step 207: Perform two-dimensional connected component analysis on the mask after morphological operation, identify connected components with an area smaller than a preset filtering threshold as pseudo-feature blocks and remove them, and generate the final target mask.

[0033] Step 208: Using the final target mask as an index, after mapping it back to the spatially aligned 3D data cube in situ, the absolute original grayscale mean of each pixel in each band channel (a total of N band channels) is calculated within the effective area covered by the final target mask. The purpose of this step is to abandon the normalization calculation method based on relative reflectance and adopt a strategy of mapping the final target mask back to the original 3D data cube in situ and directly extracting the absolute original grayscale mean of each pixel in each band within the effective area. This strategy can maximally preserve and amplify the subtle spectral fluctuations of dark targets (such as pure black or high-absorbency materials) in their low-brightness range, thereby effectively avoiding the compression and stretching effect of the bright background on the intrinsic spectral response characteristic curve, and significantly improving the spectral identification of high-absorbency materials.

[0034] Step 209: Summarize the absolute original grayscale average values ​​of each pixel in each band channel, and send all the absolute original grayscale average values ​​to the computer to generate the intrinsic spectral response characteristic curve of the target under test, and output or store the intrinsic spectral response characteristic curve of the target under test.

[0035] The x-axis of the intrinsic spectral response characteristic curve of the target under test is the spectral channel index, and the y-axis is the absolute original gray mean.

[0036] By implementing steps 201 to 209 above, it is possible to effectively resist interference from uneven illumination and ambient light intensity fluctuations, accurately segment the target area, and reproduce its intrinsic spectral response characteristic curve with high fidelity. In another exemplary embodiment of this application, step 202 is replaced by the following steps: Step 21: Set the row step size and column step size, which are respectively equal to the number of rows and columns of the microlens array in the snapshot hyperspectral camera.

[0037] Step 22: Traverse the sub-pixel positions of the microlens array, and extract the single-band channel image corresponding to each sub-pixel position from the underlying raw image data according to the row sampling step size and the column sampling step size. The sampling start row coordinates and start column coordinates corresponding to different sub-pixel positions are all different.

[0038] Step 23: All the extracted single-band channel images are stacked along the spectral channel direction (i.e. across all bands) according to the arrangement order of each sub-pixel position in the microlens array to generate a three-dimensional data cube.

[0039] Wherein, the resolution of the underlying raw image data is the width value W multiplied by the height value H, and the number of rows and columns of the microlens array is m, then the extracted k-th single-band channel image Represented as Where i and j are coordinate variables, and satisfy... , The total number of channels N equals m × m.

[0040] In another exemplary embodiment of this application, in traditional hyperspectral image segmentation methods, to extract the target (especially black or dim targets), a single spectral channel (such as the first band channel or the center wavelength channel) is often directly selected as the reference image. Then, a brightness threshold is calculated using a global adaptive thresholding method. Pixels with gray values ​​below the threshold are classified as targets, and those above the threshold are classified as background. Finally, the target contour is output by filtering through an area threshold. However, single-channel images have low signal-to-noise ratios and are easily affected by noise. Furthermore, the global threshold is easily affected by vignetting or bright backgrounds, leading to truncation of dim targets or the generation of false connected components. Therefore, this application adopts a multi-band fusion scheme to improve the signal-to-noise ratio and robustness of the reference image. Step 203 is replaced by the following steps: Step 31: Obtain the pixel intensity value of each spatial coordinate in the three-dimensional data cube across all spectral channels.

[0041] Step 32: For each spatial coordinate, based on the acquired pixel intensity value, calculate the arithmetic mean of the pixel intensity values ​​of the corresponding spatial coordinate across all spectral channels, and use this arithmetic mean as the pixel value of the corresponding spatial coordinate in the two-dimensional fusion reference map.

[0042] Step 33: Summarize the pixel values ​​of all the spatial coordinates in the two-dimensional fusion reference map to obtain the initial fusion reference map.

[0043] Step 34: Determine the minimum and maximum grayscale values ​​in the initial fusion reference image.

[0044] Step 35: Using a linear stretching algorithm, the pixel values ​​at each spatial coordinate in the initial fusion reference image are linearly mapped according to the minimum gray value and the maximum gray value, and projected to a preset gray range of 0 to 255 to obtain a two-dimensional fusion reference image.

[0045] The pixel value of a spatial coordinate (x, y) in the two-dimensional fusion reference map satisfies the following formula: ; Where N is the total number of channels. In the two-dimensional fusion baseline image, the pixel value corresponding to the spatial coordinates (x, y) is represented. Let be the pixel value corresponding to the spatial coordinates (x, y) in the k-th single-band channel image.

[0046] This step utilizes the multisampling principle inherent in fixed-step staggered sampling demultiplexing based on the physical arrangement of microlens arrays, which can significantly eliminate high-frequency dark noise introduced by single-band channel images during beam splitting. The resulting two-dimensional fusion reference map has the characteristics of high signal-to-noise ratio and clear object edges, providing a high-quality data foundation for subsequent target segmentation and effectively preventing feature fragmentation problems during the generation of the initial target mask and subsequent morphological operations.

[0047] In another exemplary embodiment of this application, step 204 is replaced by the following steps: Step 41: Obtain the region of interest (ROI) containing the target to be tested, as input by the user.

[0048] Step 42: Perform the first operation on each pixel in the two-dimensional fusion reference image to obtain the grayscale value of each pixel.

[0049] Step 43: Summarize the grayscale values ​​of all the pixels to generate the whitening masking image.

[0050] The first operation is as follows: A first determination result is obtained by determining whether the spatial coordinates of the target pixel are within the analysis space defined by the viewfinder of the analysis area. The target pixel is any pixel in the two-dimensional fusion reference image. If the first determination result is yes, the grayscale value of the target pixel is kept unchanged. If the first determination result is no, the grayscale value of the target pixel is set to the highest grayscale value, which is 255 (i.e., pure white).

[0051] In another exemplary embodiment of this application, step 205 is replaced by the following steps: Step 51: Using the viewfinder of the analysis region as a constraint, perform the Inverse Otsu thresholding algorithm within the pixel region located inside the viewfinder of the analysis region in the whitening masking image to generate target pixels and background pixels. Specifically, in the Inverse Otsu thresholding algorithm, the pixel group with the lowest grayscale value is designated as the target class, and the pixel group with the highest grayscale value is designated as the background class.

[0052] Step 52: Calculate the optimal segmentation threshold for separating the target class from the background class.

[0053] Step 53: Based on the optimal segmentation threshold, perform binarization processing on the pixels inside the viewfinder of the analysis area, mark the pixels with gray values ​​lower than or equal to the optimal segmentation threshold as target pixels, and mark the pixels with gray values ​​higher than the optimal segmentation threshold as background pixels.

[0054] Step 54: Generate a preliminary target mask based on the target pixels and the background pixels.

[0055] By generating the whitening masking image, a "whitening masking" mechanism is constructed, which forcibly "whitens" the vignetting of the optical lens and the complex external background. This forces the computation domain of the reverse Otsu thresholding algorithm to be in a simplified bimodal histogram model of "pure white background + pure dark target". This mechanism enables the reverse Otsu thresholding algorithm to accurately distinguish between the target class and the background class. The calculated optimal segmentation threshold is entirely determined by the grayscale characteristics of the target itself, thereby completely eliminating the adverse effects of uneven illumination and edge vignetting on the target segmentation accuracy and significantly improving the robustness against illumination interference.

[0056] In another exemplary embodiment of this application, step 206 is replaced by the following steps: Step 61: Perform a morphological opening operation on the preliminary target mask to obtain the opening result mask. The morphological opening operation is used to eliminate isolated speckle noise on the outer side of the target under test.

[0057] Step 62: Perform a morphological closing operation on the opening operation result mask to obtain a mask after morphological operation. The morphological closing operation is used to repair holes inside the target object.

[0058] In another exemplary embodiment of this application, step 207 is replaced by the following steps: Step 71: Perform two-dimensional connected component labeling on the mask after morphological operation to identify all interconnected connected components composed of target pixels in the mask after morphological operation, and obtain multiple candidate connected components.

[0059] Step 72: Calculate the number of pixels contained in each candidate connected region, which is used as the area of ​​the corresponding candidate connected region.

[0060] Step 73: Compare the area of ​​each candidate connected component with the preset filtering threshold, remove candidate connected components with an area smaller than the preset filtering threshold, and retain candidate connected components with an area greater than or equal to the preset filtering threshold.

[0061] Step 74: Generate the final target mask based on the retained candidate connected components.

[0062] By performing morphological opening and closing operations on the initial target mask and removing pseudo-feature blocks with areas smaller than a preset filtering threshold based on two-dimensional connected component analysis, a secondary purification of the binarized mask is achieved. This purification process effectively eliminates isolated noise points and fractures caused by uneven illumination or sensor noise, ensuring that the spatial contour of the final target mask closely matches the edges of the real physical entity. This provides an accurate spatial indexing basis for subsequently mapping the final target mask back to a three-dimensional data cube in situ and performing high-precision spectral readings.

[0063] This application also provides an application scenario in which the above-described anti-illumination interference hyperspectral image segmentation and extraction method is applied. Specifically, the anti-illumination interference hyperspectral image segmentation and extraction method provided in this embodiment can be applied to the automatic detection and recognition of low-contrast, dim targets in complex backgrounds. This scenario includes an image acquisition stage, an image preprocessing and target segmentation stage, and a target recognition and determination stage. The anti-illumination interference hyperspectral image segmentation and extraction method provided in this embodiment belongs to the image preprocessing and target segmentation stage.

[0064] It should also be noted that this application has the following effects: Through steps 201 to 202 above, high-precision spatial alignment and spectral reconstruction of the underlying raw image data of the snapshot hyperspectral camera are achieved. Specifically, the "fixed-step staggered sampling" demultiplexing strategy adopted in step 202 is based on strictly following the physical distribution law of the microlens array within the camera (such as a 4×4 array). By setting row and column sampling step sizes equal to the number of rows and columns of the array, single-band channel images corresponding to each sub-pixel position are extracted from the raw image in a staggered sampling manner. These images are then stacked along the spectral channel direction according to the arrangement order of the microlens array to generate a three-dimensional data cube. This process abandons the traditional coarse block division method of "macroscopic block cutting," fundamentally avoiding the spatial misalignment and ghosting problems of different spectral channel images caused by offset cutting positions or failure to follow array rules. The "interleaved sampling demultiplexing" mechanism based on the physical arrangement of microlens arrays extracts homogeneous pixels from a large substrate, ensuring absolute alignment of the final generated single-band channel images in spatial coordinates. This lays a solid geometric foundation for accurate mapping of target masks and accurate extraction of spectral features. This is the key to the present application's ability to completely solve the problems of "spatial misalignment" and "ghosting" in multispectral channels. A schematic diagram of the principle can be found in [link to relevant documentation]. Figure 6 .

[0065] Steps 204 and 205 constitute one of the core mechanisms of this application for resisting illumination interference, namely the "whitening masking constraint mechanism". Real snapshot-type hyperspectral camera lenses commonly exhibit a "vignetting effect" due to low edge illumination. If a traditional global adaptive thresholding algorithm (such as the standard Otsu thresholding algorithm) is used to directly segment the two-dimensional fusion reference image, it is easy to misjudge the viridescent region as a dark target, leading to distorted segmentation thresholds and the generation of large-area noise blocks. To solve this problem, in step 204, this application uses the analysis region viewfinder to force the external region to be assigned the highest grayscale value of 255, generating a whitening masking image; subsequently, in step 205, using the analysis region viewfinder as a constraint, the inverse Otsu thresholding algorithm is performed only on the internal pixels. This mechanism artificially purifies the image grayscale histogram into a bimodal distribution of "pure white background + pure dark target", forcing the optimization range of the inverse Otsu thresholding algorithm to focus on the optimal separation of the target and background, thereby effectively eliminating the interference of optical vignetting and uneven background illumination, and outputting a high-precision preliminary target mask with clean contours and no burrs. Meanwhile, the high signal-to-noise ratio (SNR) two-dimensional fusion benchmark image generated in step 203 also provides a robust data foundation for this segmentation. The synergistic effect of these two elements significantly improves the anti-interference capability and accuracy of weak target segmentation, completely overcoming the algorithm collapse trap caused by hardware "optical vignetting." A schematic diagram of this process is provided below. Figure 7 . Figure 7 (a) in the figure is a schematic diagram of the ROI before whitening constraint processing, which has dark corners and noise interference. Figure 7 (b) in the diagram is a schematic diagram of the ROI after whitening constraint processing, which can achieve precise isolation of the target.

[0066] In related technologies, the relative reflectance normalization method is commonly used when extracting the spectral characteristics of a target. This method calculates the average intensity of each pixel within the target area and divides it by the average brightness of the entire image or the background whiteboard. This ratio yields a relative reflectance ratio curve ranging from 0 to 1, which characterizes the spectral response of the object. However, when the target reflectance is extremely low and the background is bright, this normalization method compresses the absorption depth of the weak target, making its spectral curve nearly flat and losing its distinctiveness. Therefore, this application abandons the above ratio calculation method and adopts an absolute original grayscale mean extraction strategy. Specifically, the final target mask generated in the previous step is used as an index and mapped back to the three-dimensional data cube generated in step 201. Within the effective area covered by the final target mask, the pixel intensity value of each spatial coordinate in each band channel is calculated, and the arithmetic mean of all pixels in that area in each band is obtained to obtain the absolute original grayscale mean of each band. The absolute original grayscale mean values ​​of all bands are collected and sent to the computer. The intrinsic spectral response characteristic curve of the target under test is plotted with the spectral channel index on the horizontal axis and the absolute original grayscale mean value on the vertical axis.

[0067] The intrinsic spectral feature extraction method implemented in steps 208 and 209 above overcomes the information loss caused by forced normalization in conventional algorithms, significantly improving the spectral recognition and fidelity of high-absorption (black / dark) materials. This advantage mainly stems from the mechanism of in-situ mapping of the target mask and extraction of intrinsic spectral response in steps 208 and 209. Figure 8 As shown, a line graph comparing relative reflectance and absolute raw grayscale mean is presented. When detecting various pure black or similarly dark materials, existing technologies generally employ a relative reflectance algorithm based on "target brightness / bright white background brightness." Due to the extremely large denominator, the numerator representing minute differences in black targets is severely compressed, resulting in the spectral curves of multiple high-absorbency objects appearing as "smooth dead lines" at the bottom of the graph, losing their discernible value. This application takes the opposite approach, using the high-precision final target mask generated in step 207 to perform "in-situ penetration mapping," directly and forcibly extracting the "absolute raw grayscale mean (Raw Intensity)" of the target across 16 bands. This mechanism preserves and amplifies the subtle spectral fluctuations of dark materials within their own weak brightness range to the greatest extent, solving the industry pain point of "difficulty in distinguishing similar colors" for high-absorbency materials.

[0068] Based on the same inventive concept, such as Figure 3 As shown in the illustration, this application also provides a hyperspectral image segmentation and extraction system resistant to illumination interference, the system comprising: A snapshot hyperspectral camera is used to acquire raw, underlying image data.

[0069] A processor for executing the illumination-resistant hyperspectral image segmentation and extraction method described in any of the above-mentioned embodiments.

[0070] A computer is used to generate the intrinsic spectral response characteristic curve of the target under test based on all absolute raw grayscale averages sent by the processor.

[0071] In another exemplary embodiment of this application, such as Figure 5 As shown, the processor in the above embodiments specifically includes: The spatial alignment and demultiplexing module is used to acquire the underlying raw image data output by the snapshot hyperspectral camera. Based on the physical arrangement structure of the microlens array in the snapshot hyperspectral camera, the underlying raw image data is demultiplexed at the pixel level using a fixed step-size staggered sampling method to extract several single-band channel images. All the single-band channel images are then stacked along the spectral channel direction to generate a three-dimensional data cube.

[0072] The multi-band fusion noise reduction module is used to calculate the mean pixel intensity at each spatial coordinate along the spectral channel direction of the three-dimensional data cube, and to linearly stretch all the mean values ​​to a preset grayscale range to obtain a two-dimensional fusion reference map.

[0073] The whitening constraint and adaptive segmentation module is used to obtain the analysis region viewfinder containing the target to be tested, and assign the gray values ​​of the pixels located outside the analysis region viewfinder in the two-dimensional fusion reference image to the highest gray value to generate a whitening mask image. Using the analysis region viewfinder as the constraint range, the reverse Otsu thresholding algorithm is executed in the whitening mask image to generate a preliminary target mask.

[0074] The morphological refinement and mask extraction module is used to perform morphological operations on the preliminary target mask to obtain a mask after morphological operations. It then performs two-dimensional connected component analysis on the mask after morphological operations, identifies connected components with an area smaller than a preset filtering threshold as pseudo-feature blocks and removes them, and generates the final target mask.

[0075] The intrinsic spectral feature extraction module is used to use the final target mask as an index to map back to the three-dimensional data cube in situ. Within the effective area covered by the final target mask, it iterates and calculates the absolute original grayscale mean of each pixel in each band channel, summarizes the absolute original grayscale mean of each pixel in each band channel, and sends all the absolute original grayscale mean values ​​to the computer to generate the intrinsic spectral response feature curve of the target under test.

[0076] The connection relationships (data flow) between the various modules are as follows: The modules described above communicate with each other sequentially via image data links. The specific connection and interaction relationships are as follows: The input of the spatial alignment and demultiplexing module receives the underlying raw image data (RAW), and its output has two main branches: the first branch is connected to the input of the multi-band fusion and noise reduction module, and the second branch skips the intermediate process and is directly connected to the first data input of the intrinsic spectral feature extraction module.

[0077] The output of the multi-band fusion noise reduction module is connected to the input of the whitening constraint and adaptive segmentation module.

[0078] The output of the whitening constraint and adaptive segmentation module is connected to the input of the morphological refinement and mask extraction module.

[0079] The output of the morphological refinement and mask extraction module is connected to the second data input of the intrinsic spectral feature extraction module.

[0080] After the intrinsic spectral feature extraction module aggregates the data from the two branches, its output terminal outputs the final hyperspectral feature curve.

[0081] In one exemplary embodiment, such as Figure 4 As shown, the electronic device 400 in this embodiment includes a processor 401 (which may be a CPU or GPU), a memory 402 (which stores computer programs and hyperspectral data), a system bus, and a communication interface 403 (which supports network / input / output). The communication interface 403 is connected to the hyperspectral camera terminal (data source) and is used to acquire the underlying raw image data.

[0082] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 9 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs in the non-volatile storage media to run. The database stores the intrinsic spectral response characteristic curves of the target object. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a hyperspectral image segmentation and extraction method resistant to illumination interference.

[0083] Figure 9 The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0084] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0085] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0086] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with relevant regulations and be authorized by the owner of the corresponding device.

[0087] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0088] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0089] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0090] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for segmenting and extracting hyperspectral images resistant to illumination interference, characterized in that, The anti-light interference hyperspectral image segmentation and extraction method includes: Acquire the underlying raw image data output by a snapshot hyperspectral camera; Based on the physical arrangement of the microlens array in the snapshot hyperspectral camera, the original image data at the bottom layer is demultiplexed at the pixel level by using a fixed step-size staggered sampling method to extract several single-band channel images. All the single-band channel images are then stacked along the spectral channel direction to generate a three-dimensional data cube. The mean pixel intensity at each spatial coordinate is calculated for the three-dimensional data cube along the spectral channel direction, and all the mean values ​​are linearly stretched to a preset grayscale range to obtain a two-dimensional fusion reference image. A viewfinder containing the target to be tested is obtained, and the gray values ​​of the pixels located outside the viewfinder in the two-dimensional fusion reference image are assigned the highest gray values ​​to generate a whitening masking image. Using the viewfinder of the analysis area as a constraint, the reverse Otsu thresholding algorithm is executed in the whitening masking image to generate a preliminary target mask; Perform morphological operations on the preliminary target mask to obtain the mask after morphological operations are completed; Two-dimensional connected component analysis is performed on the mask after morphological operations are completed. Connected components with an area smaller than a preset filtering threshold are identified as pseudo-feature blocks and removed to generate the final target mask. Using the final target mask as an index, after mapping it back to the three-dimensional data cube in situ, the absolute original grayscale mean of each pixel in each band channel is calculated within the effective area covered by the final target mask. The absolute original grayscale mean values ​​of each pixel in each band channel are summarized, and all the absolute original grayscale mean values ​​are sent to the computer to generate the intrinsic spectral response characteristic curve of the target under test.

2. The method for segmenting and extracting hyperspectral images against illumination interference according to claim 1, characterized in that, Based on the physical arrangement of the microlens array in the snapshot hyperspectral camera, the underlying raw image data is demultiplexed at the pixel level using a fixed-step staggered sampling method to extract several single-band channel images. All of these single-band channel images are then stacked along the spectral channel direction to generate a three-dimensional data cube, specifically including: Set the row step size and the column step size; the row step size and the column step size are respectively equal to the number of rows and the number of columns of the microlens array in the snapshot hyperspectral camera; The sampling process iterates through the sub-pixel positions of the microlens array and extracts the single-band channel image corresponding to each sub-pixel position from the underlying raw image data according to the row sampling step size and the column sampling step size; wherein the sampling start row coordinates and start column coordinates corresponding to different sub-pixel positions are different. All extracted single-band channel images are stacked along the spectral channel direction according to the arrangement order of each sub-pixel position in the microlens array to generate a three-dimensional data cube.

3. The method for segmenting and extracting hyperspectral images against illumination interference according to claim 1, characterized in that, The mean pixel intensity at each spatial coordinate is calculated for the three-dimensional data cube along the spectral channel direction, and all the mean values ​​are linearly stretched to a preset grayscale range to obtain a two-dimensional fusion reference image, specifically including: Obtain the pixel intensity value of each spatial coordinate in all spectral channels in the three-dimensional data cube; For each spatial coordinate, based on the acquired pixel intensity value, the arithmetic mean of the pixel intensity values ​​of the corresponding spatial coordinate across all spectral channels is calculated and used as the pixel value of the corresponding spatial coordinate in the two-dimensional fusion reference map; The pixel values ​​of all the aforementioned spatial coordinates are summarized in the two-dimensional fusion reference map to obtain the initial fusion reference map; Determine the minimum and maximum gray values ​​in the initial fusion reference image; Using a linear stretching algorithm, the pixel values ​​at each spatial coordinate in the initial fusion reference image are linearly mapped according to the minimum gray value and the maximum gray value, and projected to a preset gray range of 0 to 255 to obtain a two-dimensional fusion reference image.

4. The method for segmenting and extracting hyperspectral images against illumination interference according to claim 3, characterized in that, The pixel value of a spatial coordinate (x, y) in the two-dimensional fusion reference map satisfies the following formula: ； Where N is the total number of channels. In the two-dimensional fusion reference image, the pixel value corresponding to the spatial coordinates (x, y) is represented. Let be the pixel value corresponding to the spatial coordinates (x, y) in the k-th single-band channel image.

5. The method for segmenting and extracting hyperspectral images resistant to illumination interference according to claim 1, characterized in that, Obtain a viewfinder containing the target to be tested for the analysis region, and assign the highest grayscale value to the pixels located outside the viewfinder in the two-dimensional fusion reference image to generate a whitening masking image, specifically including: Obtain the viewfinder frame of the analysis area containing the target to be tested, as input by the user; Perform the first operation on each pixel in the two-dimensional fusion reference image to obtain the grayscale value of each pixel; The grayscale values ​​of all the pixels are aggregated to generate the whitening masking image.

6. The method for segmenting and extracting hyperspectral images resistant to illumination interference according to claim 5, characterized in that, The first operation is: Determine whether the spatial coordinates of the target pixel are within the analysis space defined by the viewfinder of the analysis area to obtain a first determination result; the target pixel is any pixel in the two-dimensional fusion reference image; If the first determination result is yes, then the grayscale value of the target pixel is retained unchanged; If the first determination result is negative, then the grayscale value of the target pixel is set to the highest grayscale value; the highest grayscale value is 255.

7. The method for segmenting and extracting hyperspectral images resistant to illumination interference according to claim 1, characterized in that, Using the viewfinder of the analysis region as a constraint, the reverse Otsu thresholding algorithm is executed in the whitening masking image to generate a preliminary target mask, specifically including: Using the viewfinder of the analysis region as a constraint, the reverse Otsu thresholding algorithm is performed within the pixel region located inside the viewfinder of the analysis region in the whitening masking image to generate target pixels and background pixels; A preliminary target mask is generated based on the target pixels and the background pixels; In the reverse Otsu thresholding algorithm, the pixel group with the lowest gray value is designated as the target class, and the pixel group with the highest gray value is designated as the background class. An optimal segmentation threshold is calculated to separate the target class from the background class. Based on the optimal segmentation threshold, the pixels inside the viewfinder of the analysis area are binarized. Pixels with gray values ​​lower than or equal to the optimal segmentation threshold are marked as target pixels, and pixels with gray values ​​higher than the optimal segmentation threshold are marked as background pixels.

8. The method for segmenting and extracting hyperspectral images resistant to illumination interference according to claim 1, characterized in that, The x-axis of the intrinsic spectral response characteristic curve of the target under test is the spectral channel index, and the y-axis is the absolute original gray mean.

9. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that the processor executes the computer program to implement the illumination-resistant hyperspectral image segmentation and extraction method according to any one of claims 1-8.

10. A hyperspectral image segmentation and extraction system resistant to illumination interference, characterized in that, The anti-light interference hyperspectral image segmentation and extraction system includes: Snapshot hyperspectral camera, used to acquire raw, underlying image data; A processor for executing the illumination-resistant hyperspectral image segmentation and extraction method according to any one of claims 1-8; A computer is used to generate the intrinsic spectral response characteristic curve of the target under test based on all absolute raw grayscale averages sent by the processor.

Images