Image segmentation method and device, electronic device, and storage medium

Abstract

The present disclosure relates to an image segmentation method and device, an electronic device and a storage medium, the method comprising: obtaining a first tissue parameter map of a to-be-tested substance according to a first hyperspectral image, the first tissue parameter map containing content information of a component of the to-be-tested substance; and performing segmentation processing on the first hyperspectral image based on the first tissue parameter map to obtain a target segmentation result of the to-be-tested substance. By obtaining the first tissue parameter map containing the content information of the component of the to-be-tested substance, an image segmentation basis with a physical meaning is obtained, and then the first hyperspectral image is segmented based on the first tissue parameter map, so that the image segmentation is efficient and accurate, and the target segmentation result obtained is more accurate.

Description

Technical Field

[0001] This disclosure relates to the field of image processing, and more particularly to an image segmentation method and apparatus, electronic device and storage medium. Background Technology

[0002] With the development of image processing technology, image data classification and segmentation techniques can be applied to many scenarios. Typically, this is done using deep learning models for image data classification and segmentation.

[0003] In certain scenarios, the accuracy, reliability, and timeliness of image data classification and segmentation results are extremely important. However, after deep learning models are trained, the classification and segmentation results derived solely from data statistics cannot adequately meet the accuracy and reliability requirements of these scenarios.

[0004] If the classification and segmentation results output by the deep learning model are further processed to improve their accuracy, it becomes difficult to meet the requirements for timeliness.

[0005] Therefore, under normal circumstances, it is difficult to balance the accuracy and timeliness of the obtained classification and segmentation results. Summary of the Invention

[0006] In view of this, this disclosure proposes an image segmentation scheme. This disclosure provides target segmentation results that are both accurate and efficient.

[0007] According to one aspect of this disclosure, an image segmentation method is provided, comprising:

[0008] Based on the first hyperspectral image, a first tissue parameter map of the substance to be tested is obtained, wherein the first tissue parameter map contains the content information of the components of the substance to be tested;

[0009] Based on the first tissue parameter map, the first hyperspectral image is segmented to obtain the target segmentation result of the substance to be tested.

[0010] In one possible implementation, the step of segmenting the first hyperspectral image based on the first tissue parameter map to obtain the target segmentation result of the substance to be tested includes:

[0011] Based on the image features and component content at the same location in the first hyperspectral image, the first hyperspectral image is segmented to obtain the target segmentation result of the substance to be tested.

[0012] In one possible implementation, obtaining a first tissue parameter map of the analyte based on the first hyperspectral image includes:

[0013] The first hyperspectral image is input into the inverse model to generate the first tissue parameter map of the substance to be tested.

[0014] The training process of the inverse model includes:

[0015] The first hyperspectral image sample is input into the inverse model to generate the second tissue parameter map;

[0016] The second tissue parameter map is input into the forward model to generate a second hyperspectral image;

[0017] The inverse model is trained based on the differences between the second hyperspectral image and the first hyperspectral image sample.

[0018] In one possible implementation, the step of inputting the second tissue parameter map into the forward model to generate the second hyperspectral image includes:

[0019] Based on the second tissue parameter map, determine the first absorption parameter of the analyte in the first hyperspectral image sample;

[0020] Determine the first scattering parameters of the substance to be tested in the first hyperspectral image sample;

[0021] The second hyperspectral image is generated based on the first absorption parameter and the first scattering parameter.

[0022] In one possible implementation, determining the first absorption parameter of the analyte in the first hyperspectral image sample based on the second tissue parameter map includes:

[0023] Obtain the second absorption parameters of each component of the substance to be tested in the first hyperspectral image sample;

[0024] Based on the second tissue parameter map, the content of the analyte in the first hyperspectral image sample is determined;

[0025] The first absorption parameter is determined based on the content and the second absorption parameter.

[0026] In one possible implementation, determining the first scattering parameter of the analyte in the first hyperspectral image sample includes:

[0027] Based on the second tissue parameter map, obtain the reference wavelength and the second scattering parameters of the substance to be tested in the first hyperspectral image sample at the reference wavelength;

[0028] The first scattering parameter is determined based on the reference wavelength and the second scattering parameter.

[0029] In one possible implementation, the image segmentation method is based on a neural network, which includes an inverse model and a segmentation model. The training process of the neural network includes:

[0030] The second hyperspectral image sample with segmentation labels is input into the inverse model to obtain a third tissue parameter map of the composition of the substance to be tested in the second hyperspectral image sample.

[0031] Using the segmentation model, based on the third tissue parameter map and the second hyperspectral image sample, the first segmentation result of the substance to be tested in the second hyperspectral image sample is obtained;

[0032] The neural network is trained based on the difference between the first segmentation result and the segmentation label.

[0033] According to another aspect of this disclosure, an image segmentation apparatus is provided, comprising:

[0034] The first tissue parameter acquisition unit is used to obtain a first tissue parameter map of the substance to be tested based on a first hyperspectral image. The first tissue parameter map contains the content information of the components of the substance to be tested.

[0035] The target segmentation result acquisition unit is used to perform segmentation processing on the first hyperspectral image based on the first tissue parameter map to obtain the target segmentation result of the substance to be tested.

[0036] In one possible implementation, the target segmentation result acquisition unit includes:

[0037] The target segmentation result acquisition subunit is used to segment the first hyperspectral image based on the image features and the content of the components at the same location in the first hyperspectral image, and obtain the target segmentation result of the substance to be tested.

[0038] In one possible implementation, the first organization parameter acquisition unit includes:

[0039] The first tissue parameter map generation unit is used to input the first hyperspectral image into the inverse model to generate the first tissue parameter map of the substance to be tested.

[0040] The device further includes a reverse model training unit, which is used to train the reverse model. The reverse model training module includes:

[0041] The second tissue parameter map generation unit is used to input the first hyperspectral image sample into the inverse model to generate the second tissue parameter map.

[0042] The second hyperspectral image generation unit is used to input the second tissue parameter map into the forward model to generate a second hyperspectral image;

[0043] The inverse model training subunit is used to train the inverse model based on the differences between the second hyperspectral image and the first hyperspectral image samples.

[0044] In one possible implementation, the second hyperspectral image generation unit includes:

[0045] The first absorption parameter determination unit is used to determine the first absorption parameter of the analyte in the first hyperspectral image sample based on the second tissue parameter map.

[0046] The first scattering parameter determination unit is used to determine the first scattering parameter of the substance to be tested in the first hyperspectral image sample.

[0047] The second hyperspectral generation subunit is used to generate the second hyperspectral image based on the first absorption parameter and the first scattering parameter.

[0048] In one possible implementation, the first absorption parameter determining unit includes:

[0049] The second absorption parameter acquisition unit is used to acquire the second absorption parameters of each component of the substance to be tested in the first hyperspectral image sample.

[0050] The content determination unit is used to determine the content of the analyte in the first hyperspectral image sample based on the second tissue parameter map.

[0051] The first absorption parameter determining subunit is used to determine the first absorption parameter based on the content and the second absorption parameter.

[0052] In one possible implementation, the first scattering parameter determining unit includes:

[0053] The reference wavelength acquisition unit is used to acquire a reference wavelength and a second scattering parameter of the substance to be tested in the first hyperspectral image sample at the reference wavelength, based on the second tissue parameter map.

[0054] The first scattering parameter determination unit is used to determine the first scattering parameter based on the reference wavelength and the second scattering parameter.

[0055] In one possible implementation, the apparatus further includes a neural network training unit for training a neural network, the neural network including an inverse model and a segmentation model, the neural network training unit comprising:

[0056] The third tissue parameter map acquisition unit is used to input the second hyperspectral image sample with segmentation labels into the inverse model to obtain the third tissue parameter map of the composition of the substance to be tested in the second hyperspectral image sample.

[0057] The first segmentation result acquisition unit is used to obtain the first segmentation result of the substance to be tested in the second hyperspectral image sample by using the segmentation model based on the third tissue parameter map and the second hyperspectral image sample;

[0058] A neural network training subunit is used to train the neural network based on the difference between the first segmentation result and the segmentation label.

[0059] According to another aspect of this disclosure, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to implement the above-described method when executing instructions stored in the memory.

[0060] According to another aspect of this disclosure, a non-volatile computer-readable storage medium is provided that stores computer program instructions thereon, wherein the computer program instructions, when executed by a processor, implement the above-described method.

[0061] According to another aspect of this disclosure, a computer program product is provided, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described method.

[0062] In this embodiment of the disclosure, a first tissue parameter map containing the content information of the components of the substance to be tested is obtained based on the first hyperspectral image, thus obtaining an image segmentation basis with physical meaning; then, based on the first tissue parameter map, the first hyperspectral image is segmented, making the image segmentation both efficient and accurate, and also making the obtained target segmentation result more accurate.

[0063] Other features and aspects of this disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. Attached Figure Description

[0064] The accompanying drawings, which are included in and form part of this specification, illustrate exemplary embodiments, features, and aspects of this disclosure together with the specification and serve to explain the principles of this disclosure.

[0065] Figure 1 A flowchart of an image segmentation method according to an embodiment of the present disclosure is shown.

[0066] Figure 2A schematic diagram showing an organization parameter diagram according to an embodiment of the present disclosure is provided.

[0067] Figure 3 A schematic diagram illustrating the noise immunity of a reverse model according to an embodiment of the present disclosure is shown.

[0068] Figure 4 A schematic diagram of a reverse model training mode according to an embodiment of the present disclosure is shown.

[0069] Figure 5A A schematic diagram showing the relationship between the training loss and the number of training iterations of a first model according to an embodiment of the present disclosure is provided.

[0070] Figure 5B A schematic diagram showing the relationship between the training loss and the number of training iterations of a second model according to an embodiment of the present disclosure is provided.

[0071] Figure 6 A block diagram of an image segmentation apparatus according to an embodiment of the present disclosure is shown.

[0072] Figure 7 A block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0073] Figure 8 A block diagram of another electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0074] Various exemplary embodiments, features, and aspects of this disclosure will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0075] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0076] Furthermore, to better illustrate this disclosure, numerous specific details are set forth in the following detailed description. Those skilled in the art will understand that this disclosure can be practiced without certain specific details. In some instances, methods, means, components, and circuits well known to those skilled in the art have not been described in detail in order to highlight the main points of this disclosure.

[0077] Due to its advantages such as high sensitivity, non-contact operation, and fast imaging, hyperspectral imaging technology has been used for object damage assessment, object recognition, and segmentation. A common method is to acquire an image of the substance to be tested using hyperspectral imaging technology, and then input the image into a deep learning network for image segmentation processing.

[0078] In some scenarios, there are extremely high requirements for the accuracy, reliability, and timeliness of hyperspectral image segmentation processing.

[0079] However, deep learning models determine segmentation results solely based on statistical data. The accuracy and reliability of these segmentation results cannot adequately meet the needs of these scenarios.

[0080] Of course, other techniques can be used to further process the segmentation results to improve their accuracy. However, this would significantly reduce the efficiency of object edge recognition.

[0081] Therefore, this application proposes an image segmentation method that balances the accuracy, reliability, and timeliness of hyperspectral image segmentation. This allows for accurate and rapid identification of object edges.

[0082] Figure 1 A flowchart illustrating an image segmentation method according to an embodiment of the present disclosure is shown. Figure 1 As shown, the method includes:

[0083] S11, Based on the first hyperspectral image, obtain a first tissue parameter map of the substance to be tested, wherein the first tissue parameter map contains the content information of the components of the substance to be tested.

[0084] The first hyperspectral image can be a hyperspectral image taken of the substance to be tested. The substance to be tested can be organic or inorganic, or a mixture of organic and inorganic substances. The substance to be tested can have multiple components or a single component. In a medical setting, the substance to be tested can be human tissue, such as liver. In a water quality testing setting, the substance to be tested can be a body of water, such as river water. In a mineral reserve detection setting, the substance to be tested can be a rock mass, such as a rock mass containing iron ore.

[0085] The first tissue parameter map can characterize the parameters of the analyte, which can be obtained through inverse spectroscopy. The parameters characterize the physical properties of the analyte. The parameters may include at least one of the following: the content of each component of the analyte, the scattering parameters of the analyte at a reference wavelength, and the scattering slope of the analyte at the reference wavelength; the scattering includes Mie scattering and / or Rayleigh scattering. The scattering parameters include Mie scattering parameters and / or Rayleigh scattering parameters; the scattering slope is preferably the Mie scattering slope. The content can be the volume fraction of the component. The reference wavelength is a predetermined wavelength, for example, 800 nm. Therefore, the first tissue parameter map can have physical meaning.

[0086] A single first tissue parameter map can have a physical meaning. For example, multiple first tissue parameter maps can be obtained for human tissue. These can include at least one of the following: a first tissue parameter map characterizing the water content in the human tissue; a tissue parameter map characterizing the proportion of oxygenated blood in the human tissue; a tissue parameter map characterizing the scattering parameters of the human tissue at a reference wavelength, and so on.

[0087] S12, based on the first tissue parameter map, the first hyperspectral image is segmented to obtain the target segmentation result of the substance to be tested.

[0088] In this embodiment of the disclosure, the substance to be tested can be segmented based on the image features of the first hyperspectral image and the first tissue parameter map. For example, the first hyperspectral image and the first tissue parameter map can be input into a segmentation model to complete the segmentation of the substance to be tested. This embodiment of the disclosure does not limit the method of segmentation.

[0089] Traditional deep learning segmentation models can efficiently segment images. However, these models primarily rely on statistical data for segmentation. In contrast, in this embodiment, the first tissue parameter map characterizes the physical properties of the substance under test, thus giving the segmentation operation a physical meaning. Therefore, compared to segmentation results obtained using traditional deep learning models, the target segmentation results obtained in this embodiment are more accurate and reliable. Consequently, this embodiment achieves both efficient and accurate target segmentation results, namely, efficiently and accurately determining the edges of the object to be identified.

[0090] In one possible implementation, the step of segmenting the first hyperspectral image based on the first tissue parameter map to obtain the target segmentation result of the substance to be tested includes: segmenting the first hyperspectral image according to the image features at the same location in the first hyperspectral image and the content of the constituent components to obtain the target segmentation result of the substance to be tested.

[0091] In this embodiment of the disclosure, a first position can be defined as the position on the test object corresponding to a single region in a first tissue parameter map. Here, the single region can be a region composed of one or more pixels in the first tissue parameter map. The pixel value of this single region can characterize the parameter value at the first position. For example, when the first tissue parameter map characterizes the content of a certain component in the test substance, the pixel value of the single pixel can characterize the content of that component at the first position. The component may include, for example, water, lipids, bile, oxygenated blood, and deoxygenated blood.

[0092] In this embodiment of the disclosure, during the segmentation of the first hyperspectral image, the substance to be tested can be segmented based on the image features at a certain location in the hyperspectral image and the parameters of the substance to be tested at that location.

[0093] For example, during the segmentation of cancerous tissue, because cancerous tissue consumes more oxygen from blood vessels than normal tissue, the content of oxyhemoglobin in cancerous tissue is lower than that in normal tissue; while the content of deoxyhemoglobin in cancerous tissue is higher than that in normal tissue. Therefore, at various locations in the image, the cancerous tissue can be segmented by referring to the first tissue parameter map representing the oxyhemoglobin content and the first tissue parameter map representing the deoxyhemoglobin content, combined with image features.

[0094] The above are merely examples, and the embodiments disclosed herein do not limit the first tissue parameter map used for segmentation processing.

[0095] Because the segmentation process utilizes not only statistical methods for the same location but also the parameters of the analyte at that location that have physical meaning, the target segmentation results have higher accuracy and reliability.

[0096] In one possible implementation, obtaining a first tissue parameter map of the analyte based on a first hyperspectral image includes: inputting the first hyperspectral image into an inverse model to generate a first tissue parameter map of the analyte; the training process of the inverse model includes: inputting a first hyperspectral image sample into the inverse model to generate a second tissue parameter map; inputting the second tissue parameter map into a forward model to generate a second hyperspectral image; and training the inverse model based on the differences between the second hyperspectral image and the first hyperspectral image sample. The first tissue parameters, the first absorption parameter and the first scattering parameter, are obtained by simultaneously solving for the second absorption parameter and the second scattering parameter.

[0097] The inverse model can perform spectral inversion on the spectral information in the first hyperspectral image to obtain the parameters of each component in the substance to be tested; and then obtain the first tissue parameter map.

[0098] The inverse spectral solution process can involve resolving the spectrum for each wavelength in the first hyperspectral image separately. Since hyperspectral images cover a wide spectral range, typically including 400nm to 1000nm or even wider, using traditional nonlinear solving methods to resolve the spectrum would be time-consuming. In this embodiment, the inverse model can employ deep learning methods to complete the inverse spectral solution.

[0099] Table 1 Comparison of time for inverse spectroscopy using different methods.

[0100] Nonlinear solution 16136.71 Deep learning 0.47

[0101] As can be seen from Table 1, compared with the traditional nonlinear solution method, the inverse model inverse spectrum solution method in this embodiment of the present disclosure has high efficiency, greatly improves the efficiency of obtaining the first tissue parameter map, and thus makes the target segmentation result of the substance to be tested have high timeliness.

[0102] Moreover, compared with nonlinear solution methods, the inverse model in this embodiment has stronger noise resistance.

[0103] Figure 2 The diagram illustrates a tissue parameter map according to an embodiment of the present disclosure. The three figures in the first row are schematic diagrams of the tissue parameter map (true value) without added noise; the three figures in the second row are schematic diagrams of the tissue parameter map generated by nonlinear solution after adding 11dB noise; and the three figures in the third row are schematic diagrams of the tissue parameter map generated by the inverse model of the present disclosure after adding 11dB noise.

[0104] Figure 2 Horizontally, each layer contains three types of tissue parameter maps (tissue parameter map characterizing the bile content in the tested liver tissue, tissue parameter map characterizing the blood content in the tested liver tissue, and tissue parameter map characterizing the scattering parameters of the tested liver tissue at the reference wavelength λ0). Through... Figure 2 A comparison of the three tissue parameter diagrams shows that... Figure 2 Adding noise and using the inverse model to generate tissue parameter maps makes them closer to the actual tissue structure. Figure 2 The tissue parameter plot without added noise (true value). Furthermore, Figure 2 Adding noise to the tissue parameter map generated by the inverse model compared to Figure 2 The tissue parameter map generated by adding noise and using nonlinear solution is less affected by noise interference. Therefore, the inverse model in this embodiment also exhibits excellent noise resistance.

[0105] Figure 3 A schematic diagram illustrating the noise immunity of a reverse model according to an embodiment of the present disclosure is shown.

[0106] Noise, such as Gaussian white noise, is added when generating the first tissue parameter map. To mimic the noise of a natural image, the noise value can be selected within the range of 9dB-18dB, with a smaller value indicating a larger amount of noise added. Then, two sets of first tissue parameter maps are generated using a nonlinear solution model and the inverse model of this embodiment, respectively. Figure 3The diagram shows a comparison of the mean absolute errors (MAEs) of the two sets of first tissue parameter maps with the true values. It can be seen that, under noisy conditions, the MAE of the first tissue parameter map generated by the inverse model in this embodiment is smaller than that generated by the nonlinear solution model, meaning it is closer to the true value. Therefore, the inverse model in this embodiment improves the accuracy of generating the first tissue parameter map. Moreover, as the added noise increases, the MAE of the inverse model shows a smoother trend compared to the nonlinear solution model. This demonstrates that the inverse model in this embodiment has better noise resistance.

[0107] In addition, compared with traditional methods for training inverse models, the present embodiment employs a training method that combines the inverse model with the forward model, which can improve the efficiency of training the inverse model and the accuracy of the trained inverse model.

[0108] The inverse model and the forward model operate on inverse principles. The inverse model can decipher the spectrum from a hyperspectral image to obtain the parameters of the analyte; the forward model can generate a hyperspectral image based on the parameters of each analyte. The forward model can be a pre-trained model. In this embodiment, the forward model is a fixed (calibrated) model.

[0109] The first hyperspectral image sample includes the spectral information of the sample object under test. The second tissue parameter map characterizes the parameters of the sample object under test.

[0110] Figure 4 A schematic diagram illustrating a reverse model training mode according to an embodiment of the present disclosure is shown. Figure 4 As shown, the inverse model performs spectral inversion on the first hyperspectral image sample to obtain a second tissue reference map. The forward model generates a second hyperspectral image based on the second tissue parameter map. Then, the second hyperspectral image and the first hyperspectral image sample are compared to obtain their differences; the parameters of the inverse model are optimized based on these differences; training stops when the differences meet preset conditions, such as falling within a certain preset range.

[0111] Since the first hyperspectral image sample can be an unlabeled sample, the process of training the inverse model can save manpower and time for sample labeling, thus improving the efficiency of inverse model training; furthermore, it reduces the impact of sample labeling errors on the accuracy of the inverse model, thereby improving the accuracy of the inverse model.

[0112] To demonstrate that an inverse model can be trained using the differences between the second hyperspectral image and the first hyperspectral image samples, the following verification was performed in this embodiment:

[0113] In this embodiment of the disclosure, the model composed of the inverse model and the forward model is referred to as the first model. During training, the input of the first model is a first hyperspectral image sample, and the output is a second hyperspectral image. Specifically, the first hyperspectral image is input to the inverse model; the inverse model outputs a second tissue parameter map; the second tissue parameter map is input to the forward model, and the forward model outputs the second hyperspectral image. The difference between the second hyperspectral image and the first hyperspectral image sample can be referred to as the training loss of the first model.

[0114] In this embodiment, a second model can be defined. The second model includes a first forward model and a first inverse model. The input of the second model is a simulated tissue parameter map, and the output is a fourth tissue parameter map. The simulated tissue parameter map is an image composed of simulated component content. During training, the simulated tissue parameter map is input to the first forward model; the first forward model outputs a simulated hyperspectral image, which is then input to the first inverse model. The first inverse model decomposes and outputs the fourth tissue parameter map; the difference between the fourth tissue parameter map and the simulated tissue parameter map is used to train the second model. The difference between the fourth tissue parameter map and the simulated tissue parameter map can be referred to as the training loss of the second model.

[0115] If the aforementioned inverse model is a model that uses deep learning to reverse-engineer the spectrum, then the traditional method for training the inverse model is as follows: First, the nonlinear solution results of the first hyperspectral sample are labeled to obtain the labeled tissue parameter map; then, the difference between the second tissue parameter map output by the inverse model and the labeled tissue parameters is used to train the inverse model. The fourth tissue parameter map in the second model can be compared to the second tissue parameter map; they are images generated by the inverse model (first inverse). The simulated tissue parameter map in the second model can be compared to the labeled tissue parameter map; they are the ground truth of the inverse model (first inverse). Therefore, the training loss of the second model effectively simulates the difference between the second tissue parameter map and the labeled tissue parameters in the traditional training of the inverse model.

[0116] Figure 5A A schematic diagram showing the relationship between the training loss and the number of training iterations of a first model according to an embodiment of the present disclosure is provided.

[0117] Figure 5B A schematic diagram showing the relationship between the training loss and the number of training iterations of a second model according to an embodiment of the present disclosure is provided.

[0118] exist Figure 5A The medium-dark gray curve represents the relationship between training loss and the number of training iterations during the first model's training process; the light gray curve represents the relationship between training loss and the number of training iterations during the testing process. Figure 5BThe dark gray curve represents the relationship between training loss and the number of training iterations during the training of the second model; the light gray curve represents the relationship between training loss and the number of training iterations during the testing process.

[0119] By comparison Figure 5A and Figure 5B It is easy to see that, both during training and testing, as the training loss of the first model decreases, the training loss of the second model also decreases. In other words, the trends in the training loss of the first and second models are essentially synchronized during training. Furthermore, the trends in the training losses of both models are also essentially synchronized during testing. This indicates that the method for training the inverse network in this application shows a trend in sync with the effects of traditional methods; therefore, it proves that the training loss of the first model can be used to train the inverse model.

[0120] In one possible implementation, the step of inputting the second tissue parameter map into a forward model to generate a second hyperspectral image includes: determining a first absorption parameter of the analyte in the first hyperspectral image sample based on the second tissue parameter map; determining a first scattering parameter of the analyte in the first hyperspectral image sample; and generating the second hyperspectral image based on the first absorption parameter and the first scattering parameter.

[0121] In this embodiment of the present disclosure, the first absorption parameter and the first scattering parameter of the analyte can be determined respectively. Then, the forward model generates a second hyperspectral image based on the first absorption parameter and the first scattering parameter. For ease of understanding, as one possible implementation, this embodiment of the present disclosure can use formula (1) to solve for the reflectance spectrum (second hyperspectral image) of the analyte.

[0122]

[0123] Among them, R f (r) represents the reflectance spectrum of the analyte; μ eff Indicates the effective attenuation coefficient. μ a μ′ represents the first absorption parameter of the analyte. s a' represents the first scattering parameter of the analyte; a′ represents the transmission albedo. μ′ t Represents the total transmission parameters, μ′ t =μ a +μ′ s r1 represents the distance from the interface observation point to the light source; r2 represents the distance from the interface observation point to the detector; A represents the internal refraction parameter.

[0124] In this embodiment of the disclosure, by determining the first absorption parameter of the substance to be tested in the first hyperspectral image sample based on the second tissue parameter map; and determining the first scattering parameter of the substance to be tested from the first hyperspectral image sample; and then generating the second hyperspectral image based on the first absorption parameter and the first scattering parameter, the accuracy of the trained inverse model is improved.

[0125] In one possible implementation, determining the first absorption parameter of the analyte in the first hyperspectral image sample based on the second tissue parameter map includes: obtaining the second absorption parameters of each component of the analyte in the first hyperspectral image sample; determining the content of the analyte in the first hyperspectral image sample based on the second tissue parameter map; and determining the first absorption parameter based on the content and the second absorption parameter.

[0126] In this embodiment, the second absorption parameters of each component of the analyte and some general parameters can be used as parameters of the forward model, stored in the forward model either pre-stored or temporarily input. Then, the forward model determines the first absorption parameter based on the content of each component of the analyte and the second absorption parameters.

[0127] As mentioned earlier, the second tissue parameter map can characterize the content of each component in the sample. Therefore, the forward model can obtain the content of each component from the second tissue parameter map. Thus, based on the second tissue parameter map, the content of the analyte in the first hyperspectral image sample can be determined.

[0128] For example, the forward model pre-stores vessel diameter and a second absorption parameter. The second absorption parameter includes at least one of the following candidate parameters: oxyhemoglobin absorption parameter, deoxyhemoglobin absorption parameter, water absorption parameter, lipid absorption parameter, and bile absorption parameter. To obtain a more accurate first absorption parameter, the second absorption parameter may include four of the candidate parameters, or all of them.

[0129] In this embodiment, the blood oxygenation ratio can be determined based on the oxyhemoglobin and deoxyhemoglobin content, or directly obtained from the second parameter graph characterizing the blood oxygenation ratio. Then, vascular uniformity correction parameters can be generated based on the vessel diameter, oxyhemoglobin absorption parameters, and deoxyhemoglobin absorption parameters in the forward model. Next, blood absorption parameters can be determined based on the blood oxygenation ratio, oxyhemoglobin absorption parameters, deoxyhemoglobin absorption parameters, and vascular uniformity correction parameters. Following this, the blood sub-item of liver tissue absorption parameters can be determined based on blood content and blood absorption parameters; the water molecule sub-item of liver tissue absorption parameters can be determined based on water content and water absorption parameters; the lipid sub-item of liver tissue absorption parameters can be determined based on lipid content and lipid absorption parameters; and the bile sub-item of liver tissue absorption parameters can be determined based on bile content and bile absorption parameters. Finally, the first tissue absorption parameter, specifically the liver tissue absorption parameter, can be determined based on the blood sub-item, water molecule sub-item, lipid sub-item, and bile sub-item of liver tissue absorption parameters. For ease of understanding, the solution process for the first tissue absorption parameter can be represented by formulas (2)-(5) in the embodiments of this disclosure.

[0130]

[0131] Wherein, HbO2 represents the content of oxyhemoglobin, Hb represents the content of deoxyhemoglobin, and S represents the oxygen content of the blood.

[0132]

[0133] Among them, C diff (λ) represents the blood vessel uniformity correction parameter at wavelength λ, R vessel Indicates the diameter of the blood vessel. This represents the absorption parameters of oxyhemoglobin at wavelength λ. This represents the absorption parameter of deoxyhemoglobin at wavelength λ.

[0134]

[0135] in, This represents the blood absorption parameter at wavelength λ.

[0136]

[0137] Where B represents blood content, W represents water content, L represents lipid content, and Bi represents bile content. This represents the water absorption parameter at wavelength λ. This represents the lipid absorption parameter at wavelength λ; This represents the bile absorption parameter at wavelength λ. This represents the absorption parameters of liver tissue at wavelength λ.

[0138] In this embodiment of the disclosure, the second absorption parameters of each component of the substance to be tested in the first hyperspectral image sample are obtained; the content of the substance to be tested in the first hyperspectral image sample is determined according to the second tissue parameter map; at the same time, the first absorption parameter is determined according to the content and the second absorption parameter, thereby improving the accuracy of determining the first absorption parameter and thus improving the accuracy of the trained inverse model.

[0139] In one possible implementation, determining the first scattering parameter of the substance to be tested in the first hyperspectral image sample includes: obtaining a reference wavelength and a second scattering parameter of the substance to be tested in the first hyperspectral image sample at the reference wavelength according to the second tissue parameter map; and determining the first scattering parameter based on the reference wavelength and the second scattering parameter.

[0140] According to the Mie scattering principle, the ratio of the scattering parameter of the analyte at a certain wavelength to the scattering parameter of the analyte at a reference wavelength is positively correlated with a power of the ratio of the analyte to the reference wavelength. This power can be expressed as the Mie scattering slope.

[0141] For ease of understanding, the solution process for the first tissue absorption parameter can be represented by formula (6) in the embodiments of this disclosure.

[0142]

[0143] Where, μ′ s (λ) represents the scattering parameters of the analyte at wavelength λ, λ0 represents the reference wavelength, b represents the Mie scattering slope, and μ′ s (λ0) represents the scattering parameters of the substance to be measured at wavelength λ0.

[0144] In this embodiment of the disclosure, by obtaining the reference wavelength and the second scattering parameter of the substance to be tested in the first hyperspectral image sample at the reference wavelength according to the second tissue parameter map; at the same time, the first scattering parameter is determined based on the reference wavelength and the second scattering parameter, thereby improving the accuracy of determining the first scattering parameter and thus improving the accuracy of the trained inverse model.

[0145] In one possible implementation, the image segmentation method is based on a neural network, which includes an inverse model and a segmentation model. The training process of the neural network includes: inputting a second hyperspectral image sample with segmentation labels into the inverse model to obtain a third tissue parameter map of the composition of the substance to be tested in the second hyperspectral image sample; using the segmentation model, based on the third tissue parameter map and the second hyperspectral image sample, obtaining a first segmentation result of the substance to be tested in the second hyperspectral image sample; and training the neural network based on the difference between the first segmentation result and the segmentation labels.

[0146] The inverse model included in the neural network can be a pre-trained inverse model. During the training process of this neural network, the input to the neural network is the second hyperspectral image sample, and the output of the neural network is the first segmentation result.

[0147] Although the parameters of the inverse model and the segmentation model are optimized based on the difference between the first segmentation result and the segmentation label during the training process of the optimized neural network, the parameter optimization process mainly targets the segmentation model since the inverse model has already been trained. Because the input to the segmentation model is the second hyperspectral sample and the third tissue parameter map, the basis for the segmentation operation has physical meaning, making the first segmentation result closer to the true label value. Therefore, compared to simply training the traditional segmentation model, the number of times the neural network is trained is less than the number of times the traditional segmentation model is trained, improving the efficiency of training the neural network in this embodiment. Furthermore, because the segmentation operation has physical meaning, the number of parameters required for the segmentation model is reduced, which also greatly improves the efficiency of training the neural network in this embodiment.

[0148] Furthermore, during the training of the neural network, for the segmentation model in the neural grid, not only are hyperspectral image samples (with annotations) utilized, but also tissue parameter maps generated from these hyperspectral image samples by an inverse model. This reduces the amount of labeled sample data used, thereby reducing the impact of labeling errors on the accuracy of the neural network and improving its accuracy. Moreover, reducing the amount of labeled sample data also reduces sample data preparation time, thus improving the overall training efficiency of the neural network.

[0149] Table 2 shows a comparison of the segmentation performance of the neural network provided in the embodiments of this disclosure with that of the traditional segmentation model.

[0150] Table 2 Comparison of segmentation performance between neural networks and traditional segmentation models

[0151] The neural network disclosed herein 95.24 94.62 95.21 95.34 Traditional segmentation model 94.24 94.17 94.43 94.93

[0152] As can be seen from Table 2, the segmentation performance of the neural network in this disclosure is better than that of traditional segmentation models, especially improving the accuracy of the neural network.

[0153] Figure 6 A block diagram of an image segmentation apparatus according to an embodiment of the present disclosure is shown, such as Figure 6 As shown, the device 60 includes:

[0154] The first tissue parameter acquisition unit 61 is used to obtain a first tissue parameter map of the substance to be tested based on a first hyperspectral image. The first tissue parameter map contains the content information of the components of the substance to be tested.

[0155] The target segmentation result acquisition unit 62 is used to perform segmentation processing on the first hyperspectral image based on the first tissue parameter map to obtain the target segmentation result of the substance to be tested.

[0156] In one possible implementation, the target segmentation result acquisition unit 62 includes:

[0157] The target segmentation result acquisition subunit is used to segment the first hyperspectral image based on the image features and the content of the components at the same location in the first hyperspectral image, and obtain the target segmentation result of the substance to be tested.

[0158] In one possible implementation, the first organization parameter acquisition unit 61 includes:

[0159] The first tissue parameter map generation unit is used to input the first hyperspectral image into the inverse model to generate the first tissue parameter map of the substance to be tested.

[0160] The device further includes a reverse model training unit, which is used to train the reverse model. The reverse model training module includes:

[0161] The second tissue parameter map generation unit is used to input the first hyperspectral image sample into the inverse model to generate the second tissue parameter map.

[0162] The second hyperspectral image generation unit is used to input the second tissue parameter map into the forward model to generate a second hyperspectral image;

[0163] The inverse model training subunit is used to train the inverse model based on the differences between the second hyperspectral image and the first hyperspectral image samples.

[0164] In one possible implementation, the second hyperspectral image generation unit includes:

[0165] The first absorption parameter determination unit is used to determine the first absorption parameter of the analyte in the first hyperspectral image sample based on the second tissue parameter map.

[0166] The first scattering parameter determination unit is used to determine the first scattering parameter of the substance to be tested in the first hyperspectral image sample.

[0167] The second hyperspectral generation subunit is used to generate the second hyperspectral image based on the first absorption parameter and the first scattering parameter.

[0168] In one possible implementation, the first absorption parameter determining unit includes:

[0169] The second absorption parameter acquisition unit is used to acquire the second absorption parameters of each component of the substance to be tested in the first hyperspectral image sample.

[0170] The content determination unit is used to determine the content of the analyte in the first hyperspectral image sample based on the second tissue parameter map.

[0171] The first absorption parameter determining subunit is used to determine the first absorption parameter based on the content and the second absorption parameter.

[0172] In one possible implementation, the first scattering parameter determining unit includes:

[0173] The reference wavelength acquisition unit is used to acquire a reference wavelength and a second scattering parameter of the substance to be tested in the first hyperspectral image sample at the reference wavelength, based on the second tissue parameter map.

[0174] The first scattering parameter determination unit is used to determine the first scattering parameter based on the reference wavelength and the second scattering parameter.

[0175] In one possible implementation, the apparatus further includes a neural network training unit for training a neural network, the neural network including an inverse model and a segmentation model, the neural network training unit comprising:

[0176] The third tissue parameter map acquisition unit is used to input the second hyperspectral image sample with segmentation labels into the inverse model to obtain the third tissue parameter map of the composition of the substance to be tested in the second hyperspectral image sample.

[0177] The first segmentation result acquisition unit is used to obtain the first segmentation result of the substance to be tested in the second hyperspectral image sample by using the segmentation model based on the third tissue parameter map and the second hyperspectral image sample;

[0178] A neural network training subunit is used to train the neural network based on the difference between the first segmentation result and the segmentation label.

[0179] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0180] This disclosure also proposes a computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the above-described method. The computer-readable storage medium can be volatile or non-volatile.

[0181] This disclosure also proposes an electronic device, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to implement the above method when executing the instructions stored in the memory.

[0182] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device performs the above-described method.

[0183] Figure 7 This is a block diagram illustrating an apparatus 800 for image segmentation according to an exemplary embodiment. For example, apparatus 800 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0184] Reference Figure 7 The device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0185] Processing component 802 typically controls the overall operation of device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 802 may include one or more processors 820 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0186] Memory 804 is configured to store various types of data to support the operation of device 800. Examples of such data include instructions for any application or method operating on device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0187] Power supply component 806 provides power to various components of device 800. Power supply component 806 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to device 800.

[0188] Multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0189] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0190] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0191] Sensor assembly 814 includes one or more sensors for providing status assessments of various aspects of device 800. For example, sensor assembly 814 may detect the on / off state of device 800, the relative positioning of components such as the display and keypad of device 800, changes in the position of device 800 or a component of device 800, the presence or absence of user contact with device 800, the orientation or acceleration / deceleration of device 800, and temperature changes of device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0192] Communication component 816 is configured to facilitate wired or wireless communication between device 800 and other devices. Device 800 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0193] In an exemplary embodiment, the apparatus 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0194] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions that can be executed by a processor 820 of the device 800 to perform the above-described method.

[0195] Figure 8 This is a block diagram illustrating an apparatus 1900 for image segmentation according to an exemplary embodiment. For example, apparatus 1900 may be provided as a server or terminal device. (Refer to...) Figure 8The apparatus 1900 includes a processing component 1922, which further includes one or more processors, and memory resources represented by memory 1932 for storing instructions, such as application programs, that can be executed by the processing component 1922. The application programs stored in memory 1932 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1922 is configured to execute instructions to perform the methods described above.

[0196] Device 1900 may also include a power supply component 1926 configured to perform power management of device 1900, a wired or wireless network interface 1950 configured to connect device 1900 to a network, and an input / output (I / O) interface 1958. Device 1900 can operate on an operating system stored in memory 1932, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™, or similar.

[0197] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions that can be executed by a processing component 1922 of the device 1900 to perform the above-described method.

[0198] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0199] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0200] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0201] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0202] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0203] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0204] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0205] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0206] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. An image segmentation method, characterized in that, The image segmentation method is based on a neural network, which includes an inverse model and a segmentation model, including: Based on the first hyperspectral image, a first tissue parameter map of the substance to be tested is obtained. The first tissue parameter map contains the content information of the components of the substance to be tested. The first tissue parameter map characterizes the parameters of the substance to be tested. The parameters characterize the physical properties of the substance to be tested. The parameters are obtained by inverse spectral decomposition through the inverse model. The process of inverse spectral decomposition is to inversely decompose each wavelength in the first hyperspectral image. Based on the first tissue parameter map, the first hyperspectral image is segmented to obtain the target segmentation result of the substance to be tested. The step of segmenting the first hyperspectral image based on the first tissue parameter map to obtain the target segmentation result of the substance to be tested includes: The first hyperspectral image and the first tissue parameter map are input into the segmentation model. Based on the image features at the same location in the first hyperspectral image and the content of the constituent components, the first hyperspectral image is segmented to obtain the target segmentation result of the substance to be tested. The segmentation model is trained using a tissue parameter map generated from a first hyperspectral image sample with annotations and the first hyperspectral image sample. The training process of the neural network includes: The second hyperspectral image sample with segmentation labels is input into the inverse model to obtain a third tissue parameter map of the composition of the substance to be tested in the second hyperspectral image sample. Using the segmentation model, based on the third tissue parameter map and the second hyperspectral image sample, the first segmentation result of the substance to be tested in the second hyperspectral image sample is obtained; The neural network is trained based on the difference between the first segmentation result and the segmentation label.

2. The method according to claim 1, characterized in that, The step of obtaining the first tissue parameter map of the substance to be tested based on the first hyperspectral image includes: The first hyperspectral image is input into the inverse model to generate the first tissue parameter map of the substance to be tested. The training process of the inverse model includes: The first hyperspectral image sample is input into the inverse model to generate the second tissue parameter map; The second tissue parameter map is input into the forward model to generate a second hyperspectral image; The inverse model is trained based on the differences between the second hyperspectral image and the first hyperspectral image sample.

3. The method according to claim 2, characterized in that, The step of inputting the second tissue parameter map into the forward model to generate the second hyperspectral image includes: Based on the second tissue parameter map, determine the first absorption parameter of the analyte in the first hyperspectral image sample; Determine the first scattering parameters of the substance to be tested in the first hyperspectral image sample; The second hyperspectral image is generated based on the first absorption parameter and the first scattering parameter.

4. The method according to claim 3, characterized in that, The step of determining the first absorption parameter of the analyte in the first hyperspectral image sample based on the second tissue parameter map includes: Obtain the second absorption parameters of each component of the substance to be tested in the first hyperspectral image sample; Based on the second tissue parameter map, the content of the analyte in the first hyperspectral image sample is determined; The first absorption parameter is determined based on the content and the second absorption parameter.

5. The method according to claim 3, characterized in that, The determination of the first scattering parameter of the substance to be tested in the first hyperspectral image sample includes: Based on the second tissue parameter map, obtain the reference wavelength and the second scattering parameters of the substance to be tested in the first hyperspectral image sample at the reference wavelength; The first scattering parameter is determined based on the reference wavelength and the second scattering parameter.

6. An image segmentation apparatus, characterized in that, The image segmentation device is functionally based on a neural network, which includes an inverse model and a segmentation model, comprising: The first tissue parameter acquisition unit is used to obtain a first tissue parameter map of the substance to be tested based on the first hyperspectral image. The first tissue parameter map contains the content information of the components of the substance to be tested. The first tissue parameter map characterizes the parameters of the substance to be tested. The parameters characterize the physical properties of the substance to be tested. The parameters are obtained by inverse spectral decomposition through the inverse model. The process of inverse spectral decomposition is to inversely decompose each wavelength in the first hyperspectral image. The target segmentation result acquisition unit is used to perform segmentation processing on the first hyperspectral image based on the first tissue parameter map to obtain the target segmentation result of the substance to be tested; The target segmentation result acquisition unit includes: The target segmentation result acquisition subunit is used to input the first hyperspectral image and the first tissue parameter map into the segmentation model, and to perform segmentation processing on the first hyperspectral image based on the image features at the same position and the content of the components in the first hyperspectral image to obtain the target segmentation result of the substance to be tested. The segmentation model is trained using a tissue parameter map generated from a first hyperspectral image sample with annotations and the first hyperspectral image sample. The device further includes a neural network training unit, the neural network training unit comprising: The third tissue parameter map acquisition unit is used to input the second hyperspectral image sample with segmentation labels into the inverse model to obtain the third tissue parameter map of the composition of the substance to be tested in the second hyperspectral image sample. The first segmentation result acquisition unit is used to obtain the first segmentation result of the substance to be tested in the second hyperspectral image sample by using the segmentation model based on the third tissue parameter map and the second hyperspectral image sample; A neural network training subunit is used to train the neural network based on the difference between the first segmentation result and the segmentation label.

7. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the method of any one of claims 1 to 5 when executing instructions stored in the memory.

8. A non-volatile computer-readable storage medium storing computer program instructions thereon, characterized in that, When the computer program instructions are executed by the processor, they implement the method described in any one of claims 1 to 5.

Images