Hyperspectral image instance segmentation method and system based on deep learning prediction probability

Abstract

The present invention provides a method and system for hyperspectral image instance segmentation based on deep learning prediction probability, comprising obtaining a spectral dataset, inputting the spectral dataset into a spectral classification model for training and verification to obtain pre-trained weights, obtaining a hyperspectral raw data result graph and processing it to obtain endmember reflectance and wavelength data curves, inputting the pre-trained weights to obtain endmember type values ​​and endmember type prediction probabilities, generating an endmember type result graph based on the endmember type values, processing the endmember type prediction probabilities to obtain probability grayscale values ​​and generate a type prediction probability graph, combining the endmember type result graph with the type prediction probability graph to obtain hyperspectral instance segmentation data, and then post-processing to obtain instance segmentation results. The present invention uses relatively simple and efficient one-dimensional spectrum annotation instead of two-dimensional image manual annotation, fully exploring and utilizing the effective information of hyperspectral data, reducing the influence of human subjectivity, and improving the efficiency and performance of hyperspectral data processing and analysis.

Description

Technical Field

[0001] The present invention belongs to the technical field of hyperspectral data processing, and in particular relates to a hyperspectral image instance segmentation method and system based on deep learning prediction probability. Background Art

[0002] Hyperspectral imaging technology is capable of performing static or dynamic scanning imaging across hundreds of bands, and has advantages such as high spectral resolution, a large number of bands, and unified spectral images. The three-dimensional data cube acquired by hyperspectral imaging technology contains both one-dimensional spectral information and two-dimensional spatial information of the target object, with the amount of information far exceeding that of single-channel grayscale images and three-channel color images. Therefore, based on a variety of platforms such as satellites, aircraft, ground vehicles, and submarines, hyperspectral imaging technology has been widely used in industrial inspection, medical health, and agricultural management. At present, hyperspectral imaging technology has the ability to obtain spectral information with nanometer-level resolution, and the introduction of artificial intelligence methods such as deep learning has given computers the ability to quickly process and analyze large quantities of hyperspectral data, greatly improving the accuracy of target classification, target detection, and semantic segmentation. However, its instance segmentation performance and efficiency are limited by manual data labeling.

[0003] In the field of hyperspectral imaging, instance segmentation is mainly achieved through deep learning methods based on 2D images. However, the feature extraction process of 2D images is highly dependent on manually annotated image datasets. Not only is the annotation process complex and inefficient, but it is also affected by human subjectivity, which inevitably leads to instability in segmentation results and accuracy. The method for segmenting farmland blocks in satellite remote sensing images based on the improved Mask R-CNN model, published with publication number CN113870278A, implements instance segmentation of farmland blocks through a deep learning method based on manual annotation, and obtains the boundaries and positions of farmland block instances. The method for instance segmentation of hyperspectral images and the method for instance segmentation of buildings, published with publication number CN116188774A, use hyperspectral image datasets that have been annotated with instances for deep learning model training and instance segmentation. The instance segmentation effect of this method is better than that of existing methods, but no efficient or complete alternative method for data annotation has been proposed. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the purpose of the present invention is to provide a hyperspectral image instance segmentation method and system based on deep learning prediction probability, using relatively simple and efficient one-dimensional spectrum annotation instead of two-dimensional image manual annotation, fully exploring and utilizing the effective information of hyperspectral data, reducing the influence of human subjectivity, and improving the efficiency and performance of hyperspectral data processing and analysis.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A hyperspectral image instance segmentation method based on deep learning prediction probability, comprising:

[0007] Acquire a spectral data set, the spectral data set including multiple groups of input variables and multiple groups of corresponding output variables, the input variables including at least a standard reflectance and wavelength data curve, and the output variables including at least a type value;

[0008] The spectral dataset is input into the spectral classification model for training and verification to obtain pre-training weights;

[0009] Obtain the hyperspectral raw data result graph and process it to obtain the reflectance and wavelength data curve of the end member. Then input the reflectance and wavelength data curve of the end member into the pre-trained weight to obtain the type value of the end member and the type prediction probability of the end member;

[0010] Convert the type value of the end member into the end member color and generate the end member type result map;

[0011] Process the type prediction probability of the end member to obtain a probability gray value and generate a type prediction probability map;

[0012] The endmember type result map is combined with the type prediction probability map to obtain hyperspectral instance segmentation data, and the hyperspectral instance segmentation data is processed to obtain the instance segmentation result.

[0013] Furthermore, the method for obtaining the standard reflectivity and wavelength data curve is:

[0014] Obtain standard reflectance and wavelength data curves through the spectral database, or

[0015] Obtain a standard light intensity and wavelength data curve through a spectral database, perform baseline correction and reflectance calibration on the standard light intensity and wavelength data curve to obtain a standard reflectance and wavelength data curve;

[0016] The method for processing the hyperspectral raw data result graph to obtain the reflectivity and wavelength data curve of the end member is:

[0017] The hyperspectral raw data result graph is standardized to obtain the light intensity and wavelength data curve, and the light intensity and wavelength data curve is baseline corrected and reflectance calibrated to obtain the reflectance and wavelength data curve.

[0018] Furthermore, the spectral dataset was randomly divided into a spectral training set and a spectral validation set according to the validation ratio, and input into the spectral classification model for training and validation. The validation ratio ranged from 0.05 to 0.30, and hyperparameters were set, including learning rate, batch size, and number of iterations. The spectral classification model was a one-dimensional deep learning network model.

[0019] Furthermore, the method of processing the type prediction probability of the end member to obtain the probability gray value and generate the type prediction probability map is specifically as follows:

[0020] The type prediction probabilities are linearly stretched and rounded in sequence to obtain probability grayscale values, and a type prediction probability map is generated based on the probability grayscale values. The size of the type prediction probability map is the same as the size of the end member type result map.

[0021] Furthermore, the overlapping particle instance segmentation method processes the hyperspectral instance segmentation data to obtain an instance segmentation result, specifically comprising the following steps:

[0022] A region growing algorithm is used to process the hyperspectral instance segmentation data to obtain a class instance segmentation result, wherein the class instance segmentation result includes a plurality of class instance masks, and the class instance masks include class instance boundaries and class instance colors;

[0023] Determine the probability threshold, map all class instance boundaries into the probability layer and take the union to obtain the class instance area;

[0024] Run the watershed algorithm on the class instance area to obtain the instance segmentation result.

[0025] Furthermore, the method of using the region growing algorithm to process the hyperspectral instance segmentation data to obtain the class instance segmentation result is as follows:

[0026] Determine the seed point selection method, growth criteria and stopping conditions for hyperspectral instance segmentation data;

[0027] According to the seed point selection method, growth criterion and stopping condition, the region growing algorithm is run in the end member type result map, the seed point is selected as the initial end member, and the end member that meets the growth criterion is searched until the stopping condition is met to obtain the class instance segmentation result.

[0028] Furthermore, the method of running the watershed algorithm on the class instance area to obtain the instance segmentation result is as follows:

[0029] End members that meet the overlapping grain boundary condition in the class instance region are classified as overlapping grain boundaries. The overlapping grain boundary condition is that the end member type value is equal to the standard type value and the probability gray value is less than or equal to the probability threshold.

[0030] A hyperspectral image instance segmentation system based on deep learning prediction probability, comprising:

[0031] A first acquisition module is configured to acquire a spectral dataset, wherein the spectral dataset includes multiple sets of input variables and multiple sets of corresponding output variables, wherein the input variables include at least a standard reflectance and wavelength data curve, and the output variables include at least a type value;

[0032] The training module is used to input the spectral dataset into the spectral classification model for training and verification to obtain pre-trained weights;

[0033] The second acquisition module is used to obtain the hyperspectral raw data result graph and process it to obtain the reflectance and wavelength data curve of the end member, and then input the reflectance and wavelength data curve of the end member into the pre-trained weight to obtain the type value of the end member and the type prediction probability of the end member;

[0034] A third acquisition module is used to convert the type value of the endmember into an endmember color and generate an endmember type result map;

[0035] A fourth acquisition module is used to process the type prediction probability of the end member to obtain a probability grayscale value and generate a type prediction probability map;

[0036] The fifth acquisition module is used to combine the endmember type result map with the type prediction probability map to obtain hyperspectral instance segmentation data, and process the hyperspectral instance segmentation data to obtain instance segmentation results.

[0037] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method for instance segmentation of hyperspectral images based on deep learning prediction probability is implemented.

[0038] A non-transitory computer-readable storage medium stores a computer program, which, when executed by a processor, implements the above-mentioned hyperspectral image instance segmentation method based on deep learning prediction probability.

[0039] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0040] (1) The present invention uses relatively simple and efficient one-dimensional spectrum annotation instead of two-dimensional image annotation, and uses deep learning technology to accelerate and expand the recognition and supervised classification of one-dimensional spectral information, thereby improving data processing efficiency, quality and stability.

[0041] (2) The end member type result map generated by the computer algorithm is used for instance segmentation, removing unfavorable factors such as blur and noise that may exist in the original image data.

[0042] (3) By combining the region growing algorithm and the watershed algorithm, we can effectively distinguish multiple spatially overlapping individuals of the same type under one-dimensional annotation conditions and obtain complete instance segmentation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] The accompanying drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the accompanying drawings:

[0044] Figure 1 Schematic diagram of the process of hyperspectral image instance segmentation method based on deep learning prediction probability of the present invention;

[0045] Figure 2 Schematic diagram of the process of the overlapping particle instance segmentation method of the present invention;

[0046] Figure 3 This is a graph showing the hyperspectral raw data results of a specific embodiment of the present invention;

[0047] Figure 4 This is a diagram of the end member type results of a specific embodiment of the present invention;

[0048] Figure 5 This is an example segmentation result diagram of a specific embodiment of the present invention;

[0049] Figure 6 Schematic diagram of the hyperspectral image instance segmentation system based on deep learning prediction probability of the present invention. DETAILED DESCRIPTION

[0050] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention.

[0051] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, features defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0052] Example 1

[0053] Example 1 provides a hyperspectral image instance segmentation method based on deep learning prediction probability, such as Figure 1 As shown, including:

[0054] Step S1: Acquire a spectral data set, wherein the spectral data set includes multiple groups of input variables and multiple groups of corresponding output variables, wherein the input variables at least include spectral curves obtained by processing the hyperspectral raw data result graph, and the output variables at least include type values;

[0055] Step S2: Input the spectral dataset into the spectral classification model for training and verification to obtain pre-training weights;

[0056] Step S3: Obtain a new hyperspectral raw data result graph and process it to obtain the spectral curve of the endmember, then input the spectral curve of the endmember into the pre-trained weight to obtain the type value of the endmember and the type prediction probability of the endmember;

[0057] Step S4: converting the type value of the endmember into the endmember color and generating an endmember type result graph;

[0058] Step S5: Processing the type prediction probability of the end member to obtain a probability grayscale value and generating a type prediction probability map;

[0059] Step S6: combining the endmember type result map with the type prediction probability map to obtain hyperspectral instance segmentation data, and processing the hyperspectral instance segmentation data to obtain instance segmentation results.

[0060] In step S1 and step S3 of this embodiment, the method for generating the reflectivity and wavelength data curve is:

[0061] The hyperspectral raw data result graph is normalized to obtain the light intensity and wavelength data curve, where the data size (i.e., the number of bands) of the light intensity and wavelength data curve is N, where N is a positive integer, the wavelength range of the light intensity and wavelength data curve is [SW, EW], the unit is nm, and the light intensity and wavelength data curve is one-dimensional data;

[0062] Baseline correction and reflectance calibration are performed on the light intensity and wavelength data curve to obtain a reflectance and wavelength data curve, and the reflectance and wavelength data curve is used as a spectral curve, where the data size (i.e., the number of bands) of the reflectance and wavelength data curve is N, the wavelength range is [SW, EW], the unit is nm, the reflectance range is [0, 1], and the reflectance and wavelength data curve is one-dimensional data.

[0063] Reference for existing standardized methods: GB / T6040-2019 General rules for infrared spectroscopy analysis methods.

[0064] In step S1 of this embodiment, the type value is a non-negative number, and the type value generally includes the type value corresponding to the sample and the type value corresponding to the background in the hyperspectral raw data result graph. For example, when the sample is dolomite or quartz stone, the type value corresponding to the background can be set to 0, the type value corresponding to the dolomite can be set to 1, and the type value corresponding to the quartz stone can be set to 2.

[0065] In step S2 of this embodiment, the spectral dataset is randomly divided into a spectral training set and a spectral validation set according to the validation ratio, and input into the spectral classification model for training and validation. The validation ratio ranges from 0.05 to 0.30, and hyperparameters are set. The hyperparameters include learning rate, batch size, and number of iterations. The spectral classification model is a one-dimensional deep learning network model.

[0066] In step S3 of this embodiment, the type value of the end member is a non-negative integer, and the range of the type prediction probability of the end member is [0, 1].

[0067] In step S4 of this embodiment, the color space is an RGB color space, the end member type result image is an RGB color image, and the size of the end member type result image is W*H, in units of px, where W represents the width size value in pixels; H represents the height size value in pixels.

[0068] In step S5 of this embodiment, the method of processing the type prediction probability of the end member to obtain the probability grayscale value and generating the type prediction probability map is specifically as follows:

[0069] The type prediction probabilities are linearly stretched and rounded in sequence to obtain probability grayscale values, and a type prediction probability map is generated based on the probability grayscale values. The size of the type prediction probability map is the same as that of the endmember type result map.

[0070] Specifically, the probability grayscale value = [type prediction probability * 255].

[0071] The probability grayscale value is a non-negative integer in the range [0, 255].

[0072] The type prediction probability map is a grayscale image.

[0073] The size of the type prediction probability map is W*H, in px, which is the same as the size of the endmember type result map.

[0074] In step S6 of this embodiment, Figure 2 As shown in FIG, the overlapping particle instance segmentation method is used to process the hyperspectral instance segmentation data to obtain the instance segmentation result, which specifically includes the following steps:

[0075] Step S601: using a region growing algorithm to process the hyperspectral instance segmentation data to obtain a class instance segmentation result, where the class instance segmentation result includes a number of class instance masks, and the class instance masks include class instance boundaries and class instance colors;

[0076] Step S602: Determine a probability threshold, map all class instance boundaries into the probability layer and take the union to obtain the class instance region;

[0077] Step S604: running a watershed algorithm on the class instance region to obtain an instance segmentation result.

[0078] Specifically, the size of the hyperspectral instance segmentation data is W*H*2.

[0079] Hyperspectral instance segmentation data includes a type layer (i.e., endmember type result map) and a probability layer (i.e., type prediction probability map).

[0080] The size of the type layer and the probability layer are both W*H.

[0081] In step S601, the method for using the region growing algorithm to process the hyperspectral instance segmentation data to obtain the class instance segmentation result is specifically as follows:

[0082] Step S6011: determining a seed point selection method, a growth criterion, and a stop condition for the hyperspectral instance segmentation data;

[0083] Step S6012: According to the seed point selection method, growth criteria and stopping conditions, run the region growing algorithm in the end member type result map, select the seed point as the initial end member, search for the end member that meets the growth criteria until the stopping conditions are met, and obtain the class instance segmentation result.

[0084] The reason why it is called "class instance segmentation result" is that, compared with semantic segmentation, it can distinguish different individuals of the same type, which is something that only instance segmentation can do. Semantic segmentation can only distinguish different types. Compared with instance segmentation, the only thing it cannot do is distinguish individuals of the same type that overlap in space. Therefore, this embodiment continues to process the class instance segmentation result to obtain individuals of the same type that overlap in space.

[0085] Specifically, the class instance segmentation result includes several class instance masks.

[0086] The class instance mask includes the class instance boundary and the class instance color.

[0087] The color of the class instance is the same as the color of the endmember located inside the class instance boundary in the endmember type result map. Although the colors are the same, the region growing criterion obtains the boundary point and type information compared with the endmember type result map.

[0088] In step S604, the method for running the watershed algorithm on the class instance region to obtain the instance segmentation result is specifically as follows:

[0089] The end members that meet the overlapping particle boundary conditions in the class instance region are classified as overlapping particle boundaries. The overlapping particle boundary conditions are that the end member type value is equal to the standard type value and the probability gray value is less than or equal to the probability threshold.

[0090] Specifically, the overlapping particle boundary condition is that the end member type value = 1 and the probability gray value ≤ the probability threshold.

[0091] The instance segmentation result includes several instance masks.

[0092] The instance mask includes instance boundaries and instance colors.

[0093] The hyperspectral image instance segmentation method based on deep learning prediction probability proposed in this embodiment is described in detail below using a specific embodiment.

[0094] This specific example uses a Zhongdari VIX-N220 line scan hyperspectral camera and a 12mm fixed-focus FA lens to perform hyperspectral imaging on all six dolomite particles, obtaining hyperspectral raw data in the range of 400-1000nm, with a background of blue nitrile rubber.

[0095] In step S1, the hyperspectral raw data result graph is standardized to obtain a light intensity and wavelength data curve, and the light intensity and wavelength data curve is baseline corrected and reflectance calibrated to obtain a reflectance and wavelength data curve. The size (i.e., the number of bands) is 601, the wavelength range is [400, 1000], the unit is nm, and the reflectance range is [0, 1].

[0096] In this specific embodiment, the sample type is dolomite, and the corresponding type value is 1; the background is blue nitrile rubber, and the corresponding type value is 0.

[0097] In step S2, the spectral dataset was randomly divided into a spectral training set and a spectral validation set based on the validation ratio. Hyperparameters were then set and the spectral classification model was input for training and validation to obtain pretrained weights. The validation ratio was 0.30. Hyperparameters included the learning rate, batch size, and number of iterations. The learning rate was 0.0001, the batch size was 16, and the number of iterations was 100. The spectral classification model was a one-dimensional deep residual shrinkage network.

[0098] In step S3, after slicing the hyperspectral raw data result graph, the reflectivity and wavelength data curve of the end member is obtained, and then the end member spectrum curve is input into the pre-trained weight to obtain the type value of the end member and the type prediction probability of the end member. Figure 3 The figure shows the hyperspectral raw data result of a specific embodiment of the present invention, with a data size of 400px*170px*601. The data size (ie, the number of bands) of the end member reflectivity and wavelength data curve is 601.

[0099] In step S4, the size of the end member type result graph is 400px*170px, as shown in Figure 4 As shown, this is a diagram of the end member type results of a specific embodiment of the present invention.

[0100] In step S5, the size of the type prediction probability map is 400px*170px.

[0101] In step S6, the endmember type result map is combined with the type prediction probability map to obtain hyperspectral instance segmentation data. The overlapping particle instance segmentation method is used to process the hyperspectral instance segmentation data to obtain the instance segmentation result. The size of the hyperspectral instance segmentation data is 400px*170px*2. The hyperspectral instance segmentation data includes a type layer and a probability layer, and the size of the type layer and the probability layer are both 400px*170px.

[0102] The overlapping particle instance segmentation method includes the following steps:

[0103] Determine the seed point selection method, growth criterion, and stopping condition. The seed point selection method is to randomly select endmembers with endmember type values ​​of 0 and 1 respectively. The growth criterion is that the endmembers are spatially adjacent to the seed point and have the same endmember type value. The stopping condition is to find all W*H endmembers in the image.

[0104] According to the seed point selection method, growth criteria, and stopping conditions, a region growing algorithm is run in the type layer. The seed point is selected as the initial end member, and the end member that meets the growth criteria is searched until the stopping condition is met. The class instance segmentation result is obtained. The class instance segmentation result includes the background and four dolomite particles.

[0105] Determine the probability threshold, map all class instance boundaries into the probability layer and take the union to obtain the class instance area. The probability threshold is 255*16.7%=42.6.

[0106] The watershed algorithm is run on the class instance region, and the end members that meet the overlapping particle boundary condition in the class instance region are classified as overlapping particle boundaries to obtain the instance segmentation result. The overlapping particle boundary condition is that the end member type value = 1 and the probability gray value ≤ the probability threshold. The instance segmentation result includes the background and all 6 dolomite particles, such as Figure 5 , which is a diagram of an example segmentation result according to a specific embodiment of the present invention.

[0107] Example 2

[0108] Example 2 provides a hyperspectral image instance segmentation system based on deep learning prediction probability, such as Figure 5 Shown, including:

[0109] A first acquisition module is configured to acquire a spectral dataset, wherein the spectral dataset includes multiple sets of input variables and multiple sets of corresponding output variables, wherein the input variables include at least a standard reflectance and wavelength data curve, and the output variables include at least a type value;

[0110] The training module is used to input the spectral dataset into the spectral classification model for training and verification to obtain pre-trained weights;

[0111] The second acquisition module is used to obtain the hyperspectral raw data result graph and process it to obtain the reflectance and wavelength data curve of the end member, and then input the reflectance and wavelength data curve of the end member into the pre-trained weight to obtain the type value of the end member and the type prediction probability of the end member;

[0112] A third acquisition module is used to convert the type value of the endmember into an endmember color and generate an endmember type result map;

[0113] A fourth acquisition module is used to process the type prediction probability of the end member to obtain a probability grayscale value and generate a type prediction probability map;

[0114] The fifth acquisition module is used to combine the endmember type result map with the type prediction probability map to obtain hyperspectral instance segmentation data, and process the hyperspectral instance segmentation data to obtain instance segmentation results.

[0115] Example 3

[0116] Example 3 provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that when the processor executes the program, the above-mentioned hyperspectral image instance segmentation method based on deep learning prediction probability is implemented.

[0117] Example 4

[0118] Embodiment 4 A non-transitory computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the above-mentioned hyperspectral image instance segmentation method based on deep learning prediction probability.

[0119] The memory in the embodiment of the present invention is used to store various types of data to support the operation of the electronic device. Examples of such data include: any computer program used to operate on the electronic device.

[0120] The hyperspectral image instance segmentation method based on deep learning predicted probabilities disclosed in the embodiments of the present invention can be applied to or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, the various steps of the hyperspectral image instance segmentation method based on deep learning predicted probabilities can be completed by hardware integrated logic circuits or software instructions in the processor. The processor may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the embodiments of the present invention can be directly implemented and executed by a hardware decoding processor or by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium located in a memory. The processor reads information from the memory and, in conjunction with its hardware, completes the steps of the hyperspectral image instance segmentation method based on deep learning predicted probabilities provided in the embodiments of the present invention.

[0121] In an exemplary embodiment, the electronic device may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), FPGAs, general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0122] It is understood that the memory can be volatile memory or non-volatile memory, or can include both volatile and non-volatile memory. Among them, non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disk, or compact disc read-only memory (CD-ROM); magnetic surface memory can be magnetic disk memory or tape memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM).The memories described in the embodiments of the present invention are intended to include, but are not limited to, these and any other suitable types of memories.

[0123] The above embodiments are merely illustrative of the technical solutions of the present invention. The methods of the present invention are not limited solely to those described in the above embodiments, but are subject to the scope defined by the claims. Any modifications, supplements, or equivalent substitutions made by those skilled in the art based on these embodiments are within the scope of protection claimed in the claims.

Claims

1. A hyperspectral image instance segmentation method based on deep learning prediction probability, characterized in that: include: Acquire a spectral data set, the spectral data set including multiple groups of input variables and multiple groups of corresponding output variables, the input variables including at least a standard reflectance and wavelength data curve, and the output variables including at least a type value; The spectral dataset is input into the spectral classification model for training and verification to obtain pre-training weights; Obtain the hyperspectral raw data result graph and process it to obtain the reflectance and wavelength data curve of the end member. Then input the reflectance and wavelength data curve of the end member into the pre-trained weight to obtain the type value of the end member and the type prediction probability of the end member; Convert the type value of the end member into the end member color and generate the end member type result map; Process the type prediction probability of the end member to obtain a probability gray value and generate a type prediction probability map; The endmember type result map and the type prediction probability map are combined to obtain hyperspectral instance segmentation data, and the hyperspectral instance segmentation data is processed to obtain instance segmentation results; The overlapping particle instance segmentation method processes the hyperspectral instance segmentation data to obtain the instance segmentation result, which specifically includes the following steps: A region growing algorithm is used to process the hyperspectral instance segmentation data to obtain a class instance segmentation result, wherein the class instance segmentation result includes a plurality of class instance masks, and the class instance masks include class instance boundaries and class instance colors; Determine the probability threshold, map all class instance boundaries into the probability layer and take the union to obtain the class instance area; Run the watershed algorithm on the class instance area to obtain the instance segmentation result.

2. The hyperspectral image instance segmentation method based on deep learning prediction probability according to claim 1 is characterized in that: The method for obtaining the standard reflectivity and wavelength data curve is: Obtain standard reflectance and wavelength data curves through the spectral database, or Obtain a standard light intensity and wavelength data curve through a spectral database, perform baseline correction and reflectance calibration on the standard light intensity and wavelength data curve to obtain a standard reflectance and wavelength data curve; The method for processing the hyperspectral raw data result graph to obtain the reflectivity and wavelength data curve of the end member is: The hyperspectral raw data result graph is standardized to obtain the light intensity and wavelength data curve, and the light intensity and wavelength data curve is baseline corrected and reflectance calibrated to obtain the reflectance and wavelength data curve.

3. The hyperspectral image instance segmentation method based on deep learning prediction probability according to claim 1, characterized in that: The spectral dataset was randomly divided into a spectral training set and a spectral validation set according to the validation ratio, and input into the spectral classification model for training and validation. The validation ratio ranged from 0.05 to 0.30, and hyperparameters were set. The hyperparameters included learning rate, batch size, and number of iterations. The spectral classification model was a one-dimensional deep learning network model.

4. The hyperspectral image instance segmentation method based on deep learning prediction probability according to claim 1 is characterized in that The specific method of processing the type prediction probability of the end member to obtain the probability gray value and generate the type prediction probability map is as follows: The type prediction probabilities are linearly stretched and rounded in sequence to obtain probability grayscale values, and a type prediction probability map is generated based on the probability grayscale values. The size of the type prediction probability map is the same as the size of the end member type result map.

5. The hyperspectral image instance segmentation method based on deep learning prediction probability according to claim 1 is characterized in that The specific method of using the region growing algorithm to process the hyperspectral instance segmentation data to obtain the class instance segmentation result is as follows: Determine the seed point selection method, growth criteria and stopping conditions for hyperspectral instance segmentation data; According to the seed point selection method, growth criterion and stopping condition, the region growing algorithm is run in the end member type result map, the seed point is selected as the initial end member, and the end member that meets the growth criterion is searched until the stopping condition is met to obtain the class instance segmentation result.

6. The hyperspectral image instance segmentation method based on deep learning prediction probability according to claim 1, characterized in that The specific method of running the watershed algorithm on the class instance area to obtain the instance segmentation result is: End members that meet the overlapping grain boundary condition in the class instance region are classified as overlapping grain boundaries. The overlapping grain boundary condition is that the end member type value is equal to the standard type value and the probability gray value is less than or equal to the probability threshold.

7. A hyperspectral image instance segmentation system based on deep learning prediction probability, characterized in that: include: A first acquisition module is configured to acquire a spectral dataset, wherein the spectral dataset includes multiple sets of input variables and multiple sets of corresponding output variables, wherein the input variables include at least a standard reflectance and wavelength data curve, and the output variables include at least a type value; The training module is used to input the spectral dataset into the spectral classification model for training and verification to obtain pre-trained weights; The second acquisition module is used to obtain the hyperspectral raw data result graph and process it to obtain the reflectance and wavelength data curve of the end member, and then input the reflectance and wavelength data curve of the end member into the pre-trained weight to obtain the type value of the end member and the type prediction probability of the end member; A third acquisition module is used to convert the type value of the endmember into an endmember color and generate an endmember type result map; A fourth acquisition module is used to process the type prediction probability of the end member to obtain a probability grayscale value and generate a type prediction probability map; a fifth acquisition module, configured to combine the endmember type result map with the type prediction probability map to obtain hyperspectral instance segmentation data, and process the hyperspectral instance segmentation data to obtain an instance segmentation result; The fifth acquisition module is specifically configured to process the hyperspectral instance segmentation data using a region growing algorithm to obtain a class instance segmentation result, wherein the class instance segmentation result includes a plurality of class instance masks, and the class instance masks include class instance boundaries and class instance colors; Determine the probability threshold, map all class instance boundaries into the probability layer and take the union to obtain the class instance area; Run the watershed algorithm on the class instance area to obtain the instance segmentation result.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the hyperspectral image instance segmentation method based on deep learning prediction probability is implemented as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for hyperspectral image instance segmentation based on deep learning prediction probability is implemented.

Images