Method and device for extracting region of interest of hyperspectral image, and electronic equipment

Abstract

The embodiment of the invention discloses a method and a device for extracting a region of interest of a hyperspectral image, and electronic equipment. The method comprises the following steps: acquiring a hyperspectral image collected for a target object and a pseudo-color image corresponding to the hyperspectral image; performing image segmentation on the pseudo-color image to obtain an image segmentation result, and extracting a first region of interest where the target object is located in the hyperspectral image based on the image segmentation result; performing clustering analysis on each pixel point of the hyperspectral image based on the spectral features of each pixel point of the hyperspectral image to obtain a second region of interest where the target object is located in the hyperspectral image; and comprehensively determining a target region of interest of the hyperspectral image based on the first region of interest and the second region of interest. According to the scheme, the spectral information of the hyperspectral image and the spatial information of the pseudo-color image corresponding to the hyperspectral image can be combined, and the region of interest of the target object can be accurately obtained by integrating the two kinds of information.

Description

Technical Field

[0001] This invention relates to the field of spectral analysis technology, and more specifically, to a method, apparatus, electronic device, storage medium, and computer program product for extracting regions of interest from hyperspectral images. Background Technology

[0002] Hyperspectral imaging technology can acquire hyperspectral images containing rich spectral information, which is crucial for the accurate identification and classification of material components and is currently widely used in various fields. Hyperspectral image acquisition typically involves collecting samples from isolated specimens in a laboratory environment. This method allows for relatively accurate capture of the target object's spectral information, with a simple background and controllable interference factors. However, in actual greenhouse cultivation scenarios, the acquired hyperspectral images not only include the target object but also a large number of background objects, such as greenhouse facilities and the coexistence of leaves, stems, flowers, and fruits from living plants. Due to the interference caused by this complex background, it is difficult to accurately extract the region of interest based on a single spectral feature or traditional threshold segmentation algorithms. Summary of the Invention

[0003] The present invention addresses the aforementioned problems. It provides a method, apparatus, electronic device, storage medium, and computer program for extracting regions of interest (ROIs) from hyperspectral images. This approach combines the spectral information of the hyperspectral image with the spatial information of the corresponding pseudo-color image to accurately obtain the ROI of the target object.

[0004] According to one aspect of the present invention, a method for extracting a region of interest (ROI) from a hyperspectral image is provided. The method includes: acquiring a hyperspectral image of a target object and a corresponding pseudo-color image, wherein each pixel of the hyperspectral image has spectral features, and each pixel of the pseudo-color image has a pixel value in a target color channel; performing image segmentation on the pseudo-color image to obtain an image segmentation result, the image segmentation result indicating the position of the target object in the pseudo-color image; extracting a first ROI from the hyperspectral image based on the image segmentation result; performing cluster analysis on each pixel of the hyperspectral image based on the spectral features of each pixel to obtain a second ROI from the hyperspectral image; and comprehensively determining a target ROI from the hyperspectral image based on the first and second ROIs, the target ROI being the region in the hyperspectral image where the target object is located.

[0005] Optionally, based on the spectral features of each pixel in the hyperspectral image, cluster analysis is performed on each pixel in the hyperspectral image to obtain a second region of interest where the target object is located in the hyperspectral image. This includes: performing cluster analysis on each pixel in the hyperspectral image based on the spectral features of each pixel in the hyperspectral image to obtain cluster analysis results of the hyperspectral image. The cluster analysis results are used to indicate the cluster to which each pixel in the hyperspectral image belongs. At least some clusters in the cluster analysis results have a corresponding category, and the category includes the target object; and determining the second region of interest based on the cluster analysis results.

[0006] Optionally, the target region of interest is determined based on a combination of the first region of interest and the second region of interest, including: taking the intersection of the first region of interest and the second region of interest to obtain the target region of interest.

[0007] Optionally, the clustering analysis results include cluster labels for each pixel in the hyperspectral image, where the cluster labels indicate the cluster to which the corresponding pixel belongs. Determining a second region of interest (ROI) for the target object in the hyperspectral image based on the clustering analysis results includes: performing a first operation and / or a second operation on the clustering analysis results to obtain candidate regions for the target object in the hyperspectral image; determining the second ROI from the candidate regions; the first operation includes: performing statistical analysis on multiple clusters in the clustering analysis results to obtain the number of pixels in each cluster and / or the distance between the spectral features of each pixel and the cluster center of its respective cluster; and calculating the number of pixels included. Clusters whose quantity does not meet the preset requirement are removed, and / or, for each of multiple clusters, pixels whose spectral features within the cluster are not within the preset distance requirement from the cluster center are removed to obtain a new clustering analysis result, which includes the remaining clusters after removal; the second operation includes: extracting connected components from the clustering analysis result using a preset connected component analysis algorithm; performing morphological operations on the connected components to obtain new connected components, which are candidate regions; wherein, when only the first operation is performed, the region where the remaining clusters are located is the candidate region; when performing the first and second operations, the second operation is performed after the first operation.

[0008] Optionally, the morphological operations include opening and / or closing operations.

[0009] Optionally, statistical analysis is performed on multiple clusters from the clustering analysis results, including: analyzing the normal distribution characteristics of the number of pixels in each cluster using the standard score standardization method; removing clusters whose number of pixels does not meet the preset requirement, including identifying abnormal clusters that do not meet the preset requirement based on the normal distribution characteristics, wherein the preset requirement is that the deviation of the number of pixels contained from the mean of the normal distribution does not exceed a preset deviation threshold; and removing abnormal clusters.

[0010] Optionally, performing image segmentation on the pseudo-color image to obtain an image segmentation result includes: using a connectivity detection algorithm to segment the pseudo-color image to determine the target connected components in the pseudo-color image, the image segmentation result including the target connected components, the target connected components representing the region where the target object is located; or, performing image segmentation on the pseudo-color image to obtain an image segmentation result includes: inputting the pseudo-color image into a trained image segmentation model to obtain the image segmentation result output by the image segmentation model, the image segmentation result including the category label of each pixel in the pseudo-color image, the category label indicating the category to which the corresponding pixel belongs, the category including the target object.

[0011] Optionally, the target object is the target plant, or the target part of the target plant.

[0012] According to another aspect of the present invention, a device for extracting a region of interest (ROI) from a hyperspectral image is also provided, comprising: an acquisition module for acquiring a hyperspectral image of a target object and a pseudo-color image corresponding to the hyperspectral image, wherein each pixel of the hyperspectral image has spectral features and each pixel of the pseudo-color image has a pixel value in a target color channel; a segmentation module for performing image segmentation on the pseudo-color image to obtain an image segmentation result, wherein the image segmentation result is used to indicate the position of the target object in the pseudo-color image, and extracting a first ROI in the hyperspectral image based on the image segmentation result; a clustering module for performing cluster analysis on each pixel of the hyperspectral image based on the spectral features of each pixel of the hyperspectral image to obtain a second ROI in the hyperspectral image; and a determination module for comprehensively determining a target ROI in the hyperspectral image based on the first ROI and the second ROI, wherein the target ROI is the region in the hyperspectral image where the target object is located.

[0013] According to another aspect of the present invention, an electronic device is also provided, comprising: a processor and a memory, wherein the memory stores computer program instructions, which are executed by the processor to perform the above-described method for extracting the region of interest of a hyperspectral image.

[0014] According to another aspect of the present invention, a storage medium is also provided, on which program instructions are stored, which, when executed, are used to perform the above-described method for extracting the region of interest of a hyperspectral image.

[0015] According to another aspect of the present invention, a computer program product is also provided, including computer program instructions, which, when executed, are used to perform the region of interest extraction method for hyperspectral images as described above.

[0016] The above technical solution employs an image segmentation algorithm to quickly and efficiently segment pseudo-color images based on their image features. Even when spectral features overlap, by introducing spatial information about the target object in the pseudo-color image, the first region of interest (ROI) can be extracted relatively accurately from the hyperspectral image. By using the spectral features of the hyperspectral image to perform cluster analysis on each pixel, the second ROI can be obtained. Both the first and second ROIs can be considered regions containing the target object and potentially some background area. By comprehensively determining the target ROI using both the first and second ROIs, and combining the spectral features of the hyperspectral image and the image features of the pseudo-color image, the target ROI can be more accurately determined based on these two types of information.

[0017] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0018] The above and other objects, features, and advantages of the present invention will become more apparent from the more detailed description of the embodiments of the invention in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same parts or steps.

[0019] Figure 1 A schematic flowchart of a method for extracting the region of interest from a hyperspectral image according to an embodiment of the present invention is shown;

[0020] Figure 2 A schematic block diagram of a region of interest extraction apparatus for a hyperspectral image according to an embodiment of the present invention is shown;

[0021] Figure 3 A schematic block diagram of an electronic device according to an embodiment of the present invention is shown. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of the present invention more apparent, exemplary embodiments according to the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely a part of the embodiments of the present invention, and not all of the embodiments of the present invention. It should be understood that the present invention is not limited to the exemplary embodiments described herein. Based on the embodiments of the present invention described herein, all other embodiments obtained by those skilled in the art without inventive effort should fall within the protection scope of the present invention.

[0023] As mentioned above, existing techniques for extracting regions of interest (ROIs) from hyperspectral images have significant drawbacks when faced with complex backgrounds and spectral overlap. In actual planting environments, the spectral features of target objects (such as plants) and background objects (such as soil, facilities, weeds, etc.) in greenhouses often overlap, and the spectral characteristics of background objects may be similar to those of target objects. This makes it difficult for traditional methods based on spectral features to effectively distinguish between different objects.

[0024] To at least partially solve the aforementioned technical problems, embodiments of the present invention provide a method, apparatus, electronic device, storage medium, and computer program product for extracting regions of interest from hyperspectral images. This solution combines the spectral information of the hyperspectral image with the spatial information of the corresponding pseudo-color image to accurately obtain the region of interest of the target object.

[0025] Please see Figure 1 The diagram shown is a schematic flowchart of a method for extracting the region of interest (ROI) of a hyperspectral image according to an embodiment of the present invention. According to one aspect of the present invention, a method for extracting the ROI of a hyperspectral image is provided, the method comprising: steps S110 to S140.

[0026] In step S110, a hyperspectral image and a pseudo-color image corresponding to the hyperspectral image are acquired for the target object. Each pixel of the hyperspectral image has spectral features, and each pixel of the pseudo-color image has a pixel value in the target color channel.

[0027] For example, the target object can be any ground feature or any part of a ground feature. A hyperspectral camera (also known as a hyperspectral imager) can be used to acquire hyperspectral images of the target object. For example, each pixel in the hyperspectral image has corresponding spectral characteristics. These characteristics can include, for example, the radiance of the pixel in different spectral bands. Radiance is a physical quantity obtained by converting the intensity value (DN value) directly measured by the hyperspectral camera into a radiometric calibration formula, which reflects the energy intensity of the ground feature in a specific spectral band. Alternatively, the spectral characteristics of the pixel can include the intensity value (DN value) directly measured by the hyperspectral camera in different spectral bands.

[0028] For example, a pseudo-color image can be obtained by converting a hyperspectral image. Specifically, one or more bands can be selected from the spectral features of each pixel in the hyperspectral image. Each band can correspond to a target color channel, such as a red channel, a green channel, and a blue channel. For example, the target color channel corresponding to the 700nm wavelength band is the red channel, the target color channel corresponding to the 550nm wavelength band is the green channel, and the target color channel corresponding to the 450nm wavelength band is the blue channel. The spectral features of each pixel in the selected band are normalized according to a preset pixel value range (e.g., 0–255) to convert the spectral features of that pixel into pixel values ​​under the corresponding target color channel, thereby converting the hyperspectral image into a pseudo-color image. It can be understood that each pixel in the pseudo-color image has a pixel value under the target color channel. Taking the target color channels as including red, green, and blue channels as an example, each pixel in the hyperspectral image can have pixel values ​​under the red, green, and blue channels respectively, resulting in an RGB image.

[0029] In step S120, the pseudo-color image is segmented to obtain the image segmentation result. The image segmentation result is used to indicate the position of the target object in the pseudo-color image. Based on the image segmentation result, the first region of interest where the target object is located in the hyperspectral image is extracted.

[0030] For example, pseudo-color images can be segmented using either traditional segmentation algorithms or deep learning-based image segmentation algorithms. Traditional segmentation algorithms can segment the connected components containing the target object, while deep learning-based image segmentation algorithms can determine the pixels whose category labels indicate the target object. Regardless of whether the algorithm is traditional or deep learning-based, the resulting image segmentation indicates the location of the target object in the pseudo-color image. The image segmentation result can be a mask image.

[0031] For example, since the image segmentation result can indicate the position of the target object in the pseudo-color image, and the pseudo-color image is converted from the hyperspectral image, each pixel in the pseudo-color image corresponds one-to-one with each pixel in the hyperspectral image. Therefore, the position of the target object in the pseudo-color image is consistent with the position of the target object in the hyperspectral image. Using the image segmentation result, the region where the target object is located in the hyperspectral image, i.e., the first region of interest, can be extracted.

[0032] In step S130, based on the spectral features of each pixel in the hyperspectral image, cluster analysis is performed on each pixel in the hyperspectral image to obtain the second region of interest where the target object is located in the hyperspectral image.

[0033] For example, by extracting the spectral features of each pixel in a hyperspectral image, a multidimensional spectral value can be obtained for each pixel. This multidimensional spectral value represents the intensity value (DN value) of the corresponding pixel in multiple different spectral bands. The multidimensional spectral value can be used for pixel clustering analysis. By using the multidimensional spectral value as input features for a preset clustering algorithm, unsupervised classification of each pixel in the hyperspectral image can be performed. Clustering algorithms such as K-Means and K-Medoids can be used. Through clustering analysis, the category of each pixel can be determined, thereby obtaining the second region of interest (ROI) where the target object in the hyperspectral image is located.

[0034] In step S140, the target region of interest in the hyperspectral image is determined based on the first region of interest and the second region of interest. The target region of interest is the region in the hyperspectral image where the target object is located.

[0035] For example, the target region of interest (ROI) can be determined by combining the first and second ROIs. For instance, the mask images corresponding to the first and second ROIs, along with a pseudo-color image, can be concatenated and input into a trained segmentation model to obtain the final target ROI. Alternatively, the intersection or union of the first and second ROIs can be taken as the target ROI. Another example is that the union of the first and second ROIs can be followed by morphological opening and / or closing operations to obtain the target ROI.

[0036] The above technical solution employs an image segmentation algorithm to quickly and efficiently segment pseudo-color images based on their image features. Even when spectral features overlap, by introducing spatial information about the target object in the pseudo-color image, the first region of interest (ROI) can be extracted relatively accurately from the hyperspectral image. By using the spectral features of the hyperspectral image to perform cluster analysis on each pixel, the second ROI can be obtained. Both the first and second ROIs can be considered regions containing the target object and potentially some background area. By comprehensively determining the target ROI using both the first and second ROIs, and combining the spectral features of the hyperspectral image and the image features of the pseudo-color image, the target ROI can be more accurately determined based on these two types of information.

[0037] Optionally, based on the spectral features of each pixel in the hyperspectral image, cluster analysis is performed on each pixel in the hyperspectral image to obtain a second region of interest where the target object is located in the hyperspectral image. This includes: performing cluster analysis on each pixel in the hyperspectral image based on the spectral features of each pixel in the hyperspectral image to obtain cluster analysis results of the hyperspectral image. The cluster analysis results are used to indicate the cluster to which each pixel in the hyperspectral image belongs. At least some clusters in the cluster analysis results have a corresponding category, and the category includes the target object; and determining the second region of interest based on the cluster analysis results.

[0038] For example, users can determine the number of cluster centers, or clusters, when performing cluster analysis on hyperspectral images based on the acquisition environment. Clustering algorithms can then be used to obtain the cluster analysis results of the hyperspectral images. These results can be represented by cluster label images, where the pixel value of each pixel indicates the cluster it belongs to. In all clusters of the cluster analysis results, at least some clusters can have a corresponding category, and the target object belongs to one of these categories. For example, based on the cluster analysis results, a second region of interest (ROI) can be determined where the target object is located in the hyperspectral image. It can be understood that the category corresponding to the cluster to which the pixels within the second ROI belong is the target object.

[0039] The above technical solution can automatically group pixels with similar spectral features in hyperspectral images into one category by performing cluster analysis on hyperspectral images. This helps to ensure that the pixels of the target object are grouped into one category so that the second region of interest can contain the complete target object.

[0040] Optionally, the target region of interest is determined based on a combination of the first region of interest and the second region of interest, including: taking the intersection of the first region of interest and the second region of interest to obtain the target region of interest.

[0041] For example, when performing cluster analysis on pixels in a hyperspectral image, the inventors discovered that pixels corresponding to different parts of a plant are easily grouped into the same cluster. For instance, in the cluster analysis results, the cluster corresponding to the leaves of a target grape and the cluster corresponding to the flower buds of a target grape belong to the same group; that is, two different parts are grouped into one category. This embodiment of the invention effectively avoids misidentification by taking the intersection of the first region of interest and the second region of interest. Specifically, a first mask image can be generated based on the first region of interest, and a second mask image can be generated based on the second region of interest. It can be understood that the first mask image can serve as the basis for determining the location of the target region at the spatial level, and the second mask image can serve as the basis for determining the location of the target region at the spectral level. A pixel-by-pixel logical AND operation can be performed on the first mask image and the second mask image. The logic of the AND operation is as follows: if a certain pixel in the first mask image belongs to the mask region, and a pixel in the second mask image whose position is the same as that pixel in the first mask image also belongs to the mask region, then the pixel in the hyperspectral image whose position is the same as the positions of these two pixels in their respective images can be used as the pixel of the target region of interest.

[0042] The above technical solution takes the intersection of the first region of interest and the second region of interest, which can take into account both spectral and spatial information. This results in a target region of interest that is a set of pixels with highly consistent spectral and spatial information. This solution can accurately obtain the true outline and distribution of the target object under complex background conditions, providing a reliable foundation for subsequent analysis of hyperspectral images.

[0043] Optionally, the clustering analysis results include cluster labels for each pixel in the hyperspectral image, where the cluster labels indicate the cluster to which the corresponding pixel belongs. Determining a second region of interest (ROI) for the target object in the hyperspectral image based on the clustering analysis results includes: performing a first operation and / or a second operation on the clustering analysis results to obtain candidate regions for the target object in the hyperspectral image; determining the second ROI from the candidate regions; the first operation includes: performing statistical analysis on multiple clusters in the clustering analysis results to obtain the number of pixels in each cluster and / or the distance between the spectral features of each pixel and the cluster center of its respective cluster; and calculating the number of pixels included. Clusters whose quantity does not meet the preset requirement are removed, and / or, for each of multiple clusters, pixels whose spectral features within the cluster are not within the preset distance requirement from the cluster center are removed to obtain a new clustering analysis result, which includes the remaining clusters after removal; the second operation includes: extracting connected components from the clustering analysis result using a preset connected component analysis algorithm; performing morphological operations on the connected components to obtain new connected components, which are candidate regions; wherein, when only the first operation is performed, the region where the remaining clusters are located is the candidate region; when performing the first and second operations, the second operation is performed after the first operation.

[0044] For example, the clustering analysis result may include a cluster label for each pixel of the hyperspectral image. Taking a cluster label image as an example, the pixel value of each pixel in the cluster label image can be mapped to a cluster label, which indicates the cluster to which the corresponding pixel belongs. In a specific embodiment, if the cluster labels mapped to the pixel values ​​of pixel A and pixel B in the cluster label image are both "0", it means that the two pixels in the hyperspectral image corresponding to pixel A and pixel B respectively belong to cluster 0.

[0045] For example, when determining a second region of interest in a hyperspectral image, a first operation can be performed on the clustering analysis results. Specifically, for each of the multiple clusters in the clustering analysis results, the number of pixels within that cluster can be counted, and clusters whose number of pixels does not meet a preset requirement can be removed. The preset requirement may, for example, include being greater than a first preset threshold, less than a second preset threshold, or greater than or equal to the first preset threshold and less than or equal to the second preset threshold. In some embodiments, the pixel number distribution of each cluster can be analyzed using a Z-score normalization method, and clusters whose Z-values ​​do not meet a preset value (e.g., less than -2 or greater than +2) can be removed. In this case, the preset requirement is that the Z-value corresponding to the number of pixels meets the preset value.

[0046] For example, for each of the multiple clusters resulting from clustering analysis, the distance between the spectral features of each pixel within that cluster and the cluster center can be statistically analyzed. Clustering algorithms can determine cluster centers corresponding one-to-one with multiple clusters, and the vector dimension of the cluster center is consistent with the vector dimension of the spectral features. The "distance" can be Euclidean distance, cosine distance, Mahalanobis distance, etc. Preset distance requirements may include, for example, that the distance between the spectral features of a pixel and the cluster center of its respective cluster is less than or equal to a preset distance threshold. In some embodiments, the distribution of the distances between the spectral features of pixels within each cluster and their corresponding cluster centers can be statistically analyzed. The distance threshold for that cluster is set using the mean distance, standard deviation distance, and quantile distance of each pixel within the cluster; for example, the distance threshold can be equal to the sum of the mean distance and twice the standard deviation distance. When the distance between the spectral features of any pixel within a cluster and its corresponding cluster center is greater than the distance threshold, the spectral features of that pixel are considered to be too far from the corresponding cluster center, and the pixel is discarded. In other embodiments, for each cluster, pixels whose corresponding distance exceeds the distances of 95% of the pixels within that cluster can be discarded. In this case, the preset quantity requirement is that the corresponding distance does not exceed the distance of 95% of the pixels in the cluster.

[0047] For example, the operation can be performed by counting the number of pixels in a cluster and removing clusters that do not meet the preset number requirement; or it can be performed by counting the distance between the spectral features of a pixel and the center of its cluster and removing pixels that do not meet the preset distance requirement; or both operations can be performed. The final new clustering analysis result includes the remaining clusters after removal. It is understood that for the initial clustering analysis result, some clusters in multiple clusters may be completely removed, and some pixels in each of at least some clusters in multiple clusters may be removed. When only the first operation is performed, the region where each pixel of the remaining cluster is located can be used as a candidate region, and the second region of interest is within the candidate region.

[0048] For example, when determining the second region of interest in a hyperspectral image, a second operation can be performed on the clustering analysis results. Specifically, a preset connected component analysis algorithm can be used to extract spatially continuous regions, i.e., connected components, from the clustering analysis results. This embodiment of the invention does not specifically limit the connected component analysis algorithm; for example, it can be a four-neighborhood-based algorithm, or an eight-neighborhood-based algorithm. After obtaining the connected components, morphological operations (such as dilation, erosion, etc.) can be performed on them to obtain new connected components, which are then considered candidate regions. It is understood that candidate regions can be obtained solely through the first operation, solely through the second operation, or through both operations. When both operations are required, the second operation is performed after the first operation, and the clustering analysis result targeted by the second operation is the new clustering analysis result obtained in the first operation.

[0049] The above technical solution can remove clusters with abnormal number of pixels and / or pixels within a cluster whose corresponding spectral features are abnormally far from the cluster center through the first operation. This can retain clusters with a reasonable number of pixels, and / or retain pixels within a cluster whose spectral features are closer to the cluster center. This can enhance the stability and representativeness of the clustering analysis results, thereby obtaining candidate regions with higher quality and stronger reliability.

[0050] Optionally, the morphological operations include opening and / or closing operations.

[0051] For example, morphological operations may include opening operations, which are composite operations performed in the order of erosion followed by dilation. Opening operations can eliminate small white noise points in connected components, break narrow connections, and smooth the boundaries of larger connected components without significantly altering their area. Morphological operations may also include closing operations, which are composite operations performed in the order of dilation followed by erosion. Closing operations can fill small holes and cracks within connected components, connect disconnected adjacent parts, and smooth boundaries without significantly altering their area. Through morphological opening and closing operations, connected components can be smoothed and denoised, eliminating discrete regions with excessively small areas or irregular shapes, resulting in new connected components with spatial integrity and reasonable morphology.

[0052] Optionally, statistical analysis is performed on multiple clusters from the clustering analysis results, including: analyzing the normal distribution characteristics of the number of pixels in each cluster using the standard score standardization method; removing clusters whose number of pixels does not meet the preset requirement, including identifying abnormal clusters that do not meet the preset requirement based on the normal distribution characteristics, wherein the preset requirement is that the deviation of the number of pixels contained from the mean of the normal distribution does not exceed a preset deviation threshold; and removing abnormal clusters.

[0053] For example, the normal distribution characteristics of the number of pixels in each cluster can be analyzed using the standard score normalization method (i.e., the Z-score normalization method in the aforementioned embodiment). For any cluster, the normalized score Z of the cluster is equal to the ratio between the difference between the number of pixels in the cluster and the average number of pixels in all clusters and the standard deviation of the number of pixels in all clusters. After obtaining the normal distribution characteristics of the number of pixels in each cluster, a preset deviation threshold T can be set. The preset deviation threshold T can define the maximum allowable deviation of the size of a cluster (i.e., the number of pixels it contains) under the standard normal distribution. Each cluster can be judged according to the preset deviation threshold T. The judgment criterion is: if the absolute value of the normalized score of the cluster, |Z| > T, then the cluster is judged as an abnormal cluster. In this case, the preset quantity requirement includes: the deviation of the number of pixels contained from the mean of the normal distribution does not exceed the preset deviation threshold T. In practical applications, the preset deviation threshold T can be set according to the specific scenario. For example, in a standard normal distribution, T=2 or T=3 is often chosen, corresponding to approximately 95% and 99.7% of the data falling within the interval [μ-2σ, μ+2σ] or [μ-3σ, μ+3σ] around the mean, respectively. Clusters falling outside these intervals are considered outliers. Finally, outliers can be removed from the clusters.

[0054] The above technical solution can effectively filter out clusters with too few pixels (which may represent noise, background interference, or segmentation fragments) or too many pixels (which may represent incorrect merging of multiple different regions), which helps to ensure that each cluster used for analysis has a statistically reasonable number of pixels and helps to improve the quality of cluster analysis results.

[0055] Optionally, performing image segmentation on the pseudo-color image to obtain an image segmentation result includes: using a connectivity detection algorithm to segment the pseudo-color image to determine the target connected components in the pseudo-color image, the image segmentation result including the target connected components, the target connected components representing the region where the target object is located; or, performing image segmentation on the pseudo-color image to obtain an image segmentation result includes: inputting the pseudo-color image into a trained image segmentation model to obtain the image segmentation result output by the image segmentation model, the image segmentation result including the category label of each pixel in the pseudo-color image, the category label indicating the category to which the corresponding pixel belongs, the category including the target object.

[0056] In some embodiments, connectivity detection algorithms can be used for image segmentation of pseudo-color images. Connectivity detection algorithms can be color consistency-based segmentation algorithms, such as thresholding algorithms. They can also be edge gradient-based segmentation algorithms, such as graph cut algorithms. Furthermore, they can be region connectivity-based segmentation algorithms, such as region growing algorithms. Connectivity detection algorithms do not rely on deep learning; instead, they segment based on low-level features of the image itself (such as color, texture, and edges) and spatial relationships. Connectivity detection algorithms can segment target connected regions, which can represent the region where the target object is located. For example, in thresholding algorithms, the target connected region can be a connected region that meets preset pixel value requirements. In region growing algorithms, the connected region formed by merging pixels according to preset similarity criteria (such as pixel value or grayscale difference being less than a corresponding preset threshold) is the target connected region. In graph cut algorithms, by minimizing an energy function that combines region and boundary attributes, the connected region formed by the set of pixels assigned to the foreground source point is the target connected region.

[0057] In other embodiments, a pseudo-color image can be input into a trained image segmentation model. The image segmentation model can be a semantic segmentation model, specifically a lightweight semantic segmentation model such as a Fully Convolutional Network (FCN) or a U-Net, which allows for fine-tuning of the model's parameters using a small training set. The image segmentation model can distinguish between pixels belonging to the target object and pixels belonging to the background region in the input image. For each pixel, the image segmentation model can determine its category label, which indicates the category to which the corresponding pixel belongs. All categories that the image segmentation model can determine can include the target object. The output image segmentation result of the image segmentation model can include the category label of each pixel in the input image, and can also include a pixel-level mask image, in which regions with pixel values ​​of a specific mask value (e.g., 255) are the regions where the target object is located.

[0058] The above technical solution can segment connected pixel regions with similar features in pseudo-color images through connectivity detection algorithms, thereby quickly obtaining the region where the target object is located. It has low computational resource requirements and does not rely on training data. The above technical solution can also perform pixel-level segmentation of pseudo-color images through trained image segmentation models. This deep learning-based segmentation method has high segmentation accuracy and strong robustness, and is especially suitable for scenes with complex backgrounds.

[0059] For example, before step S120, a preprocessing operation can be performed on the pseudo-color image. The preprocessing operation may include a superpixel segmentation operation. Performing a superpixel segmentation operation on the pseudo-color image can transform it from an image with pixels as the smallest unit into an image with blocks as the smallest unit. Through superpixel segmentation, adjacent pixels with similar features such as color, texture, and brightness can be grouped into blocks, and these blocks can be considered as superpixels. In this case, the new pseudo-color image can be a superpixel label map, where each superpixel has a corresponding pixel value, such as the average pixel value of the pixels that make up the superpixel.

[0060] For example, the preprocessing operation may also include an edge detection operation, which can detect locations in the pseudo-color image where there are drastic changes in grayscale, color, or texture. In this case, the new pseudo-color image can be an edge intensity image, where the pixel value of each pixel in the edge intensity image can represent the confidence that the pixel belongs to an edge. Alternatively, the new pseudo-color image can be an edge enhancement image, where the pixel values ​​of pixels belonging to edges can be preset pixel values, such as (255, 255, 255), or their pixel values ​​in each target color channel can be the difference between 255 and the pixel value of the corresponding pixel in the pseudo-color image in that target color channel, or their pixel values ​​can be the sum of the pixel value of the corresponding pixel in the pseudo-color image and a preset offset. The pixel values ​​of pixels belonging to non-edges in the edge enhancement image can remain unchanged or can optionally be adjusted according to the actual situation.

[0061] For example, for a pseudo-color image, only one of the preprocessing operations—edge detection and superpixel segmentation—may be performed, or both. In the latter case, superpixel segmentation can be performed first, followed by edge detection on the new pseudo-color image obtained from the superpixel segmentation to obtain the final pseudo-color image for image segmentation. Alternatively, edge detection can be performed first, followed by superpixel segmentation on the new pseudo-color image obtained from the edge detection to obtain the final pseudo-color image for image segmentation.

[0062] For example, before step S120, the pseudo-color image can be input into a trained object detection model. The initial object detection model can be a deep learning model pre-trained using a publicly available image recognition dataset. Pre-trained object detection models have high detection accuracy and real-time performance. To improve the adaptability of the object detection model in the application scenarios of this embodiment, a limited number of scene samples can be used to perform task transfer and parameter fine-tuning on the object detection model, thereby enhancing the model's ability to recognize the target object category. This embodiment does not specifically limit the type of object detection model. For example, the object detection model can employ region-based convolutional neural networks (R-CNN), You Only LookOnce (YOLO), Single Shot MultiBox Detector (SSD), etc. The object detection model can output object detection results based on the input pseudo-color image. The object detection results can indicate the target image region where the target object is located in the pseudo-color image. The object detection results can include the target image region and the confidence that the target image region contains the target object. More specifically, the number of target image regions can be one or more, and each target image region can be considered as a bounding box, which is a desired region that may contain the target object and part of the background region. For example, in step S130, image segmentation can be an operation targeting only the target image regions in the pseudo-color image. Specifically, the target image regions in the pseudo-color image can be cropped to obtain target image patches, and image segmentation can be performed on the target image patches. For example, the target image patches can be processed using a connectivity detection algorithm, or the target image patches can be input into a trained image segmentation model. If the preprocessing operations of the aforementioned embodiments need to be performed, preprocessing operations can be performed on the target image patches.

[0063] Optionally, the target object is the target plant, or the target part of the target plant.

[0064] For example, in practical applications, greenhouses contain a diverse range of plant species, and their morphological and spectral characteristics change significantly at different growth stages. Traditional methods often rely on pre-defined spectral libraries or specific training sample sets, which are poorly adaptable to new species or plants at different growth stages. This invention, by combining spectral and spatial information, can adaptively perform clustering and image segmentation, thereby handling the identification of various plant types. Furthermore, it maintains high extraction accuracy under different light and climatic conditions, demonstrating strong adaptability and the ability to accommodate different species, growth stages, or target parts of target plants.

[0065] Please see Figure 2 The diagram shown is a schematic block diagram of a region of interest extraction apparatus 200 for a hyperspectral image according to one embodiment of the present invention. According to another aspect of the present invention, a region of interest extraction apparatus for a hyperspectral image is also provided, comprising:

[0066] The acquisition module 210 is used to acquire a hyperspectral image of the target object and a pseudo-color image corresponding to the hyperspectral image. Each pixel of the hyperspectral image has spectral features, and each pixel of the pseudo-color image has a pixel value in the target color channel.

[0067] The segmentation module 220 is used to perform image segmentation on the pseudo-color image to obtain image segmentation results. The image segmentation results are used to indicate the position of the target object in the pseudo-color image. Based on the image segmentation results, the first region of interest where the target object is located in the hyperspectral image is extracted.

[0068] Clustering module 230 is used to perform clustering analysis on each pixel of the hyperspectral image based on the spectral features of each pixel to obtain the second region of interest where the target object is located in the hyperspectral image.

[0069] The determination module 240 is used to determine the target region of interest in the hyperspectral image based on the first region of interest and the second region of interest. The target region of interest is the region in the hyperspectral image where the target object is located.

[0070] Please see Figure 3 As shown, it is a schematic block diagram of an electronic device 300 according to an embodiment of the present invention. According to another aspect of the present invention, an electronic device is also provided, including: a processor 310 and a memory 320, wherein the memory 320 stores computer program instructions, which are executed by the processor 310 to perform the above-described method for extracting the region of interest of a hyperspectral image.

[0071] According to another aspect of the present invention, a storage medium is also provided, on which program instructions are stored. When the program instructions are executed by a computer or processor, the computer or processor performs the corresponding steps of the hyperspectral image region of interest extraction method described in the embodiments of the present invention, and is used to implement the corresponding modules in the hyperspectral image region of interest extraction apparatus or the corresponding modules in the hyperspectral image region of interest extraction apparatus described in the embodiments of the present invention. The storage medium may, for example, include a memory card of a smartphone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. A computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0072] According to another aspect of the present invention, a computer program product is also provided, including computer program instructions, which, when executed, are used to perform the region of interest extraction method for hyperspectral images as described above.

[0073] Those skilled in the art can understand the specific implementation and beneficial effects of the above-described method for extracting regions of interest (ROI) from hyperspectral images by reading the detailed description above. For the sake of brevity, further details will not be elaborated here.

[0074] Although exemplary embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above exemplary embodiments are merely illustrative and are not intended to limit the scope of the invention thereto. Various changes and modifications can be made therein by those skilled in the art without departing from the scope and spirit of the invention. All such changes and modifications are intended to be included within the scope of the invention as claimed in the appended claims.

[0075] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0076] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed.

[0077] Numerous specific details are set forth in the specification provided herein. However, it will be understood that embodiments of the invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0078] Similarly, it should be understood that, in order to streamline the invention and aid in understanding one or more of the various aspects of the invention, features of the invention are sometimes grouped together in a single embodiment, figure, or description thereof in the description of exemplary embodiments of the invention. However, this approach should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as reflected in the corresponding claims, its inventive point lies in solving the corresponding technical problem with fewer features than all of those in a single disclosed embodiment. Therefore, the claims following the detailed description are hereby expressly incorporated into that detailed description, wherein each claim itself is a separate embodiment of the invention.

[0079] Those skilled in the art will understand that, apart from the mutual exclusion of features, all features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all processes or units of any method or apparatus so disclosed can be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) may be replaced by an alternative feature that serves the same, equivalent, or similar purpose.

[0080] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, in the claims, any of the claimed embodiments can be used in any combination.

[0081] The various component embodiments of the present invention can be implemented in hardware, or as software modules running on one or more processors, or a combination thereof. Those skilled in the art will understand that microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some modules in the apparatus for region of interest extraction of hyperspectral images according to embodiments of the present invention. The present invention can also be implemented as an apparatus program (e.g., a computer program and computer program product) for performing part or all of the methods described herein. Such programs implementing the present invention can be stored on a computer-readable medium or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, provided on a carrier signal, or provided in any other form.

[0082] It should be noted that the above embodiments are illustrative of the invention and not restrictive, and that those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names.

[0083] The above are merely specific embodiments or descriptions of the present invention, and the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for extracting regions of interest from hyperspectral images, characterized in that, include: Acquire a hyperspectral image of the target object and a pseudo-color image corresponding to the hyperspectral image, wherein each pixel of the hyperspectral image has spectral features and each pixel of the pseudo-color image has a pixel value in the target color channel; The pseudo-color image is segmented to obtain an image segmentation result, which is used to indicate the position of the target object in the pseudo-color image. Based on the image segmentation result, a first region of interest where the target object is located in the hyperspectral image is extracted. Based on the spectral features of each pixel in the hyperspectral image, cluster analysis is performed on each pixel in the hyperspectral image to obtain the second region of interest where the target object is located in the hyperspectral image; The target region of interest in the hyperspectral image is determined by combining the first region of interest and the second region of interest. The target region of interest is the region in the hyperspectral image where the target object is located.

2. The method according to claim 1, characterized in that, The step of performing cluster analysis on each pixel of the hyperspectral image based on the spectral features of each pixel to obtain the second region of interest where the target object is located in the hyperspectral image includes: Based on the spectral features of each pixel in the hyperspectral image, cluster analysis is performed on each pixel in the hyperspectral image to obtain the cluster analysis results of the hyperspectral image. The cluster analysis results are used to indicate the cluster to which each pixel in the hyperspectral image belongs. At least some clusters in the cluster analysis results have a corresponding category, and the category includes the target object. The second region of interest is determined based on the clustering analysis results.

3. The method according to claim 1, characterized in that, The step of determining the target region of interest based on the first region of interest and the second region of interest includes: The intersection of the first region of interest and the second region of interest is taken to obtain the target region of interest.

4. The method according to claim 2, characterized in that, The clustering analysis results include cluster labels for each pixel in the hyperspectral image, whereby the cluster labels indicate the cluster to which the corresponding pixel belongs. Determining the second region of interest (ROI) in the hyperspectral image based on the clustering analysis results includes: Perform a first operation and / or a second operation on the clustering analysis results to obtain candidate regions of the target object in the hyperspectral image, and determine the second region of interest from the candidate regions. The first operation includes: Statistical analysis is performed on multiple clusters from the clustering analysis results to obtain the number of pixels in each cluster and / or the distance between the spectral characteristics of each pixel and the cluster center of its respective cluster; Clusters containing a number of pixels that do not meet the preset number requirement are removed, and / or, for each of the multiple clusters, pixels whose spectral features and the distance between the cluster center do not meet the preset distance requirement are removed, so as to obtain a new clustering analysis result, which includes the remaining clusters after removal. The second operation includes: Using a preset connected component analysis algorithm, connected components are extracted from the clustering analysis results; Perform morphological operations on the connected components to obtain new connected components, which are the candidate regions. Specifically, when only the first operation is performed, the region where the remaining clusters are located is the candidate region; when both the first and second operations are performed, the second operation is performed after the first operation.

5. The method according to claim 4, characterized in that, The morphological operations include opening and / or closing operations.

6. The method according to claim 4, characterized in that, The statistical analysis of multiple clusters in the clustering analysis results includes: The normal distribution characteristics of the number of pixels in each of the multiple clusters were analyzed using the standard score normalization method. The process of removing clusters whose number of pixels does not meet the preset requirement includes... Based on the normal distribution characteristics, abnormal clusters that do not meet the preset number requirement are identified. The preset number requirement is that the deviation of the number of pixels contained therein from the mean of the normal distribution does not exceed a preset deviation threshold. Remove the abnormal clusters.

7. The method according to any one of claims 1-6, characterized in that, The step of performing image segmentation on the pseudo-color image to obtain image segmentation results includes: A connectivity detection algorithm is used to segment the pseudo-color image to determine the target connected components in the pseudo-color image. The image segmentation result includes the target connected components, which represent the region where the target object is located. or, The step of performing image segmentation on the pseudo-color image to obtain image segmentation results includes: The pseudo-color image is input into a trained image segmentation model to obtain the image segmentation result output by the image segmentation model. The image segmentation result includes a category label for each pixel of the pseudo-color image. The category label is used to indicate the category to which the corresponding pixel belongs, and the category includes the target object.

8. The method according to any one of claims 1-6, characterized in that, The target object is the target plant, or the target part of the target plant.

9. A device for extracting regions of interest from hyperspectral images, characterized in that, include: The acquisition module is used to acquire a hyperspectral image of a target object and a pseudo-color image corresponding to the hyperspectral image. Each pixel of the hyperspectral image has spectral features, and each pixel of the pseudo-color image has a pixel value in the target color channel. A segmentation module is used to perform image segmentation on the pseudo-color image to obtain an image segmentation result, the image segmentation result being used to indicate the position of the target object in the pseudo-color image, and to extract a first region of interest in the hyperspectral image where the target object is located based on the image segmentation result; The clustering module is used to perform clustering analysis on each pixel of the hyperspectral image based on the spectral features of each pixel to obtain the second region of interest where the target object is located in the hyperspectral image. The determination module is used to comprehensively determine the target region of interest of the hyperspectral image based on the first region of interest and the second region of interest, wherein the target region of interest is the region in the hyperspectral image where the target object is located.

10. An electronic device, characterized in that, The device includes a processor and a memory, characterized in that the memory stores computer program instructions, which, when executed by the processor, are used to perform the region of interest extraction method for hyperspectral images as described in any one of claims 1-8.

11. A storage medium storing a computer program / instructions, characterized in that, The computer program / instructions, when running, are used to perform the region of interest extraction method for hyperspectral images as described in any one of claims 1-8.

12. A computer program product comprising computer program instructions, characterized in that, The computer program instructions, when executed, are used to perform the region of interest extraction method for hyperspectral images as described in any one of claims 1-8.

Images