Training method of disease classification model, disease classification method and system
By combining metasurface light guide elements and disease classification models, and using optical parameter sets to convert spectral images and train models, the problems of high cost and large data volume in hyperspectral imaging are solved, achieving efficient and accurate disease classification.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SHPHOTONICS LTD
- Filing Date
- 2023-12-27
- Publication Date
- 2026-07-07
AI Technical Summary
Hyperspectral imaging is costly and generates large amounts of complex data in medical diagnosis, which hinders its widespread adoption.
By combining metasurface light guide elements with a disease classification model, the optical parameters of the metasurface light guide elements are used to convert spectral images into optical images. The disease classification model is then trained using a neural network, and the optical parameters of the metasurface light guide elements are optimized to achieve accurate disease classification.
It enables efficient and accurate disease classification, reduces the cost and data requirements of hyperspectral imaging, and improves diagnostic efficiency.
Smart Images
Figure CN117710774B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of optical imaging technology, and in particular to a training method for a disease classification model, a disease classification method, and a system. Background Technology
[0002] Medical imaging technology is becoming increasingly advanced, and hyperspectral imaging, in particular, has shown great potential in medical diagnosis due to its ability to provide detailed image information. For example, in the field of dentistry, hyperspectral imaging technology has been used to assist in the diagnosis of oral cancer, dental caries, and other abnormalities in dental tissues.
[0003] However, among related technologies, hyperspectral imaging is expensive, and the amount of hyperspectral data is large and complex, requiring professional data processing and analysis, which is not conducive to its widespread adoption. Summary of the Invention
[0004] In view of this, embodiments of this application provide a training method, a disease classification method, and a system for a disease classification model, so that the combination of metasurface light guide elements and models can accurately predict disease categories.
[0005] The technical solution of this application embodiment is implemented as follows:
[0006] In a first aspect, embodiments of this application provide a training method for a disease classification model. The disease classification model is used in a disease classification system, which includes a photosensitive element and a metasurface light guide element. The photosensitive element includes M sets of pixels for sensing incident light of different wavelengths, and the metasurface light guide element is used to guide incident light of different wavelengths to the M sets of pixels.
[0007] The training method includes:
[0008] Acquire a spectral image set, wherein the spectral image set includes multiple spectral image groups and a category label corresponding to each spectral image group, and each spectral image group includes N spectral images of the same target object obtained by imaging, wherein N is a positive integer;
[0009] Based on the preset M optical parameter groups of the metasurface light guide element, each spectral image group is converted into an optical image group to obtain a training image set, where M is a positive integer;
[0010] The disease classification model is trained using the training image set to obtain a loss value;
[0011] Based on the loss value, update at least one of the M optical parameter sets of the metasurface light guide element, or update at least one of the M optical parameter sets of the metasurface light guide element and both the model parameters of the disease classification model, until the disease classification model converges, thus obtaining the trained disease classification model and the optimal M optical parameter sets of the metasurface light guide element.
[0012] Furthermore, based on the preset M optical parameter groups of the metasurface light guide element, each spectral image group is converted into an optical image group to obtain a training image set, including:
[0013] Each of the spectral image groups is coupled to M optical parameter groups of the metasurface light guide element to obtain an optical image group corresponding to each spectral image group. Each time a spectral image group is coupled to an optical parameter group, an optical image of the corresponding optical image group is obtained.
[0014] Further, each of the spectral image groups is coupled with an optical parameter group to obtain an optical image of the corresponding optical image group, including:
[0015] The optical image of the optical image group is obtained by multiplying each of the spectral images in the spectral image group by an optical parameter in the optical parameter group and summing the products.
[0016] Furthermore, each of the optical parameter groups includes N optical parameters;
[0017] The step of multiplying each spectral image in the spectral image group by an optical parameter from the optical parameter group and then summing the products to obtain an optical image of the optical image group corresponding to the spectral image group includes:
[0018] The spectral images of the N bands of the spectral image group are multiplied one-to-one with the N optical parameters of the optical parameter group, and then summed to obtain an optical image of the optical image group corresponding to the spectral image group.
[0019] Furthermore, M satisfies: M≥N / 4.
[0020] Furthermore, the wavelength range of the N bands is in the range of 400nm to 1000nm.
[0021] Furthermore, the method also includes:
[0022] Based on the optimal M sets of optical parameters of the metasurface light guide element, the arrangement of the nanostructure units on the metasurface light guide element is determined.
[0023] Furthermore, determining the arrangement of nanostructure units on the metasurface light guide element based on the optimal M sets of optical parameters includes:
[0024] Based on the optimal M optical parameter sets of the metasurface optical guide element, M types of nanostructure units are determined;
[0025] Based on each of the aforementioned nanostructure units, a nanostructure unit array is determined, resulting in M nanostructure unit arrays;
[0026] Based on the array of M nanostructure units, the arrangement of the nanostructure units on the metasurface optical guide element is determined.
[0027] Secondly, embodiments of this application provide a disease classification system, including:
[0028] A photosensitive element, the photosensitive element comprising M groups of pixels for sensing incident light of different wavelengths;
[0029] A metasurface light guide element, wherein the metasurface light guide element is determined according to the M optical parameter groups determined by the training method of the disease classification model in any one of the above embodiments, and is used to guide incident light of different wavelengths to the M groups of pixels;
[0030] A processor is configured to acquire M optical images from the photosensitive element and predict the disease category of the target object using a disease classification model trained according to the training method of the disease classification model according to any one of the above embodiments.
[0031] Thirdly, embodiments of this application provide a disease classification method, which is used in the disease classification system described in the second aspect above, the method comprising:
[0032] The target object is photographed using the photosensitive element to obtain M optical images of the target object;
[0033] Using the processor, the disease category of the target object is predicted based on the M optical images.
[0034] Fourthly, embodiments of this application provide a computer device, including:
[0035] At least one processor;
[0036] A memory storing a computer program executable on the processor, characterized in that the processor executes the steps of the method described in the first aspect above when executing the program.
[0037] Fifthly, embodiments of this application provide a computer storage medium storing a computer program that, when executed by a processor, performs the steps of the method described in the first aspect above.
[0038] This application provides a training method, a disease classification method, and a system for a disease classification model. The training method for the disease classification model includes: acquiring a set of spectral images; converting the spectral image set into a training image set based on the optical parameter set of a metasurface light guide element; training the disease classification model using the training image set to obtain a loss value; and updating at least one of the M optical parameter sets of the metasurface light guide element, or updating both the M optical parameter sets of the metasurface light guide element and the model parameters of the disease classification model, based on the loss value, to obtain a trained disease classification model and the optimal optical parameter set of the metasurface light guide element. This application converts spectral images into optical images based on the optical parameter set of the metasurface light guide element, and through model training, enables the combination of the metasurface light guide element and the model to accurately predict disease categories. Attached Figure Description
[0039] Figure 1 This is a flowchart illustrating a training method for a disease diagnosis model provided in an embodiment of this application.
[0040] Figure 2 This is a schematic diagram of a spectral transmittance curve provided in an embodiment of this application.
[0041] Figure 3 This is a schematic diagram illustrating a nanostructure unit of a metasurface optical guide element determined based on an optimal set of optical parameters, as provided in an embodiment of this application.
[0042] Figure 4 This is a schematic diagram illustrating the distribution of nanostructure units on a metasurface optical guide element based on a defined set of M nanostructure units, as provided in an embodiment of this application.
[0043] Figure 5 This is a schematic diagram of the structure of a disease classification system provided in an embodiment of this application.
[0044] Figure 6 This is a flowchart illustrating a disease classification method provided in an embodiment of this application.
[0045] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation
[0046] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0047] Figure 1 This is a flowchart illustrating a training method for a disease classification model provided in an embodiment of this application. The method is used in a disease classification system, which includes a photosensitive element and a metasurface light guide element. The photosensitive element includes M sets of pixels for sensing incident light of different wavelengths, and the metasurface light guide element is used to guide incident light of different wavelengths to the M sets of pixels.
[0048] In this embodiment, the metasurface light guide element is a surface structure with special optical properties, which can achieve various functions by controlling the propagation of light. In an exemplary embodiment of this application, the metasurface light guide element includes multiple nanostructure units. By appropriately designing these multiple nanostructure units, precise deflection of light can be achieved, thereby guiding M incident lights of different wavelengths one-to-one to M sets of pixels.
[0049] like Figure 1 As shown, the training method for the disease classification model includes, but is not limited to, the following steps:
[0050] S201: Obtain a spectral image set, wherein the spectral image set includes multiple spectral image groups and a category label corresponding to each spectral image group, and each spectral image group includes N spectral images of the same target object obtained by imaging, wherein N is a positive integer;
[0051] S202: Based on the preset M optical parameter groups of the metasurface light guide element, each spectral image group is converted into an optical image group to obtain the training image set, where M is a positive integer;
[0052] S203: Train the disease classification model using the training image set to obtain the loss value;
[0053] S204: Based on the loss value, update at least one of the M optical parameter sets of the metasurface light guide element or update at least one of the M optical parameter sets of the metasurface light guide element and both the model parameters of the disease classification model, until the disease classification model converges, and obtain the trained disease classification model and the optimal M optical parameter sets of the metasurface light guide element.
[0054] The disease classification model trained in this application can be used to predict the disease category of a target object. Here, the target object can be a body part, such as the mouth, teeth, knee joint, elbow joint, etc. Disease categories can be divided into various different categories depending on the target object. For example, when the target object is teeth, the disease categories can include: healthy, dental caries, periodontal disease, damaged, etc. Of course, disease categories can also be set as a first category, a second category, a third category, etc., according to a predefined correspondence. This application does not limit the specific names used to distinguish disease categories.
[0055] In this embodiment, when acquiring the spectral image set, it can be obtained from a database of hospitals, clinics, or related institutions that treat or statistically analyze the diseases of the target object. The database has pre-recorded and saved spectral image sets of target objects with different disease categories and their corresponding disease categories (i.e., category labels). Of course, the spectral image set also includes spectral image sets of the target object in a healthy state and their corresponding category labels.
[0056] Understandably, the source of the spectral image set may also be other, and this application does not limit it.
[0057] Each group of spectral images in a spectral image set can be acquired once or multiple times by a spectral acquisition device. For example, for a patient, the target object can be imaged using a spectral acquisition device (such as a hyperspectral camera or a multispectral camera) to obtain spectral images of N bands at once; or, the target object can be imaged multiple times by a spectral acquisition device to obtain spectral images of one band or fewer than N bands each time.
[0058] In the embodiments of this application, the N spectral images of the same spectral image group are spectral images in different bands, so that the spectral information of the spectral images in different bands can fully reflect the internal physical structure, chemical composition and other information of the target object.
[0059] Preferably, the N spectral images may also include spectral images from the near-infrared region, which is not visible.
[0060] In one exemplary embodiment, a spectral image group contains 61 spectral images (i.e., N = 61), and the wavelength range of the 61 spectral images is between 400 and 1000 nm, with a band spacing of 10 nm.
[0061] The category label corresponding to the spectral image group is determined based on the actual situation of the target object. For example, taking a patient's upper first molar as the target object, if the upper first molar has caries, then the category label corresponding to the spectral image group collected for the target object is labeled as caries.
[0062] In step 202, each spectral image group can be converted into an optical image group based on the preset M optical parameter groups of the metasurface light guide element to obtain a training image set.
[0063] In this embodiment, each optical parameter group includes multiple optical parameters, each of which indicates the transmittance of a nanostructure unit of the metasurface light guide element for light of a specific wavelength or band. Multiple optical parameters within the same optical parameter group indicate the transmittance of a nanostructure unit of the metasurface light guide element for light of different wavelengths or bands, and multiple optical parameter groups indicate the transmittance of multiple nanostructure units of the metasurface light guide element for light. Here, the optical parameters can be directly the transmittance, or they can be values converted from the transmittance according to a preset conversion formula.
[0064] In one exemplary embodiment of this application, the number of optical parameters in each optical parameter group is the same as the number of spectral images N in the spectral image group; that is, each optical parameter group includes N optical parameters (e.g., transmittance). Simultaneously, the wavelengths corresponding to these N transmittance parameters can correspond one-to-one with the wavebands corresponding to the N spectral images in the spectral image group.
[0065] Here, a one-to-one correspondence can mean that the wavelength corresponding to the transmittance is within the wavelength range of the corresponding band in the spectral image; or, it can mean that the wavelength corresponding to the transmittance does not exceed the wavelength range threshold of the corresponding band in the spectral image. The wavelength threshold is a preset value, preferably less than the wavelength range of all corresponding bands in the spectral images. When the wavelength corresponding to a transmittance exceeds the wavelength range of a band in a certain spectral image, but does not exceed the wavelength range threshold of that band, the wavelength corresponding to the transmittance can still be correlated with the range of the corresponding band in the spectral image. For example, if the wavelength range of a band corresponding to a spectral image is 500nm to 510nm, and the wavelength threshold is 3nm, then when correlating optical parameters with spectral images, the spectral image can be correlated with the optical parameter (transmittance) at a wavelength of 512nm.
[0066] In an exemplary embodiment of this application, a preset set of optical parameters for the metasurface light guide element can be determined based on a randomly constructed spectral transmittance curve. For example, Figure 2 This is a spectral transmittance curve provided for an embodiment of this application. For example... Figure 2 As shown, the transmittance of the wavelengths corresponding to the frequency bands of the N spectral images in the spectral image group on the spectral transmittance curve can be used as the N optical parameters of the optical parameter group, thereby determining an optical parameter group.
[0067] In some embodiments of this application, step S202 may specifically be as follows:
[0068] S302: Couple each spectral image group with M optical parameter groups of the metasurface light guiding element respectively to obtain an optical image group corresponding to each spectral image group. Here, when converting each spectral image group into an optical image group based on the preset optical parameter groups of the metasurface light guiding element, specifically: First, obtain the preset M optical parameter groups of the metasurface light guiding element; then, couple each spectral image group with the M optical parameter groups of the metasurface light guiding element respectively to obtain an optical image group corresponding to each spectral image group. Specifically, when a spectral image group is coupled with an optical parameter group, an optical image corresponding to it is obtained. Thus, after the spectral image group is coupled with the M optical parameter groups respectively, M optical images can be obtained, and these M optical images form an optical image group corresponding to the spectral image group.
[0069] Here, when converting each spectral image group into an optical image group based on the preset optical parameter groups of the metasurface light guiding element, it can be specifically as follows: First, obtain the preset M optical parameter groups of the metasurface light guiding element; then, couple each spectral image group with the M optical parameter groups of the metasurface light guiding element respectively to obtain an optical image group corresponding to each spectral image group. Specifically, when a spectral image group is coupled with an optical parameter group, an optical image corresponding to it is obtained. Thus, after the spectral image group is coupled with the M optical parameter groups respectively, M optical images can be obtained, and these M optical images form an optical image group corresponding to the spectral image group.
[0070] In some embodiments of the present application, M satisfies: M≥N / 4. Exemplarily, when N is 60, M can be 16.
[0071] In some embodiments of the present application, when the spectral image group in step S302 is coupled with an optical parameter group, it can be specifically as follows: Multiply each spectral image in the spectral image group by an optical parameter of the optical parameter group respectively and then sum them to obtain an optical image of the optical image group corresponding to the spectral image group. In a specific example of the present application, when each optical parameter group includes N optical parameters, that is, the number of optical parameters in each optical parameter group is the same as the number of spectral images in the spectral image group, then when the spectral image group is coupled with an optical parameter group to obtain an optical image, it can be specifically as follows: Multiply the N spectral images of the spectral image group by the N optical parameters of the optical parameter group one by one and then sum them to obtain an optical image.
[0072] In another specific example of the present application, when each optical parameter group includes K optical parameters, K<N, that is, the number of optical parameters in each optical parameter group is less than the number of spectral images in the spectral image group, then when the spectral image group is coupled with an optical parameter group to obtain an optical image, it can be specifically as follows: First, divide the N spectral images of the spectral image group into K first groups, each first group includes at least one spectral image, and each spectral image is only in one first group; then, correspond the K first groups to the K optical parameters respectively; finally, multiply the N spectral images of the spectral image group by the optical parameters corresponding to their respective first groups respectively and then sum them to obtain an optical image.
[0073] In this embodiment, when dividing the N spectral images of a spectral image group into K first groups, multiple spectral images with adjacent bands are divided into the same first group.
[0074] In another specific example of this application, when each optical parameter group includes K' optical parameters, where K'>N, that is, the number of optical parameters in each optical parameter group is greater than the number of spectral images in the spectral image group, the coupling of the spectral image group with an optical parameter group to obtain an optical image can be specifically as follows: First, the K' spectral images of the optical parameter group are divided into N second groups, each second group including at least one optical parameter, and each optical parameter is only in one second group; then, the average value of all optical parameters in each second group is used as the average optical parameter of that second group; finally, the N spectral images of the spectral image group are multiplied one-to-one with the average optical parameters of the N second groups, and then the products are summed to obtain an optical image.
[0075] The following example illustrates the process of converting a spectral image set into a training image set:
[0076] First, obtain a spectral image set, denoted as A. The spectral image set A contains 10,000 spectral image groups B, and each spectral image group B is labeled with a corresponding category label S.
[0077] Each spectral image group B contains 30 (i.e., N = 30) spectral images, and each spectral image group is denoted as: B i (HSI i1 HSI i2 ,……,HSI iN ), where 1≤i≤10000.
[0078] Then, nine preset optical parameter groups (i.e., M=9) are determined for the metasurface light guide element. Each optical parameter group has N optical parameters, and the nine optical parameter groups are denoted as follows:
[0079] Φ1(θ 11 ,θ 12 ,......,θ 1N ),Φ2(θ 21 ,θ 22 ,......,θ 2N ), ......, Φ M (θ M1 ,θ M2 ,......,θ MN ).
[0080] Finally, based on the nine preset optical parameter groups of the metasurface light guide element, each spectral image group is converted into an optical image group D, and the category label of each optical image group D follows the category label of the corresponding spectral image group, thus obtaining the training image set, denoted as C.
[0081] Each optical image group D contains 9 optical images, and each optical image group is denoted as D. i (M i1 M i2 ,……,M iM ), where 1≤i≤10000.
[0082] Here, the optical image and the spectral image satisfy the following relationship:
[0083] Where 1≤i≤10000, 1≤j≤M.
[0084] This specific example is based on a preset set of optical parameters of a metasurface light guide element. Each spectral image set is converted into an optical image set. The converted optical image set is used as the training sample for the disease classification model. This allows the trained disease classification model to predict the disease category of the target object directly based on the optical image set of the target object obtained by the metasurface light guide element, without the need to collect the spectral image of the target object.
[0085] In step 203 above, after converting the spectral image set into a training image set, the disease classification model is further trained using the training image set to obtain the loss value.
[0086] Specifically, when training a disease classification model using a training image set, the optical image set and its corresponding category label in the training image set are used as variables, and at least one of the M optical parameter sets or at least one of the M optical parameter sets and the model parameters of the disease classification model are used as response quantities to train the disease classification model, obtain the predicted disease category, and input it into a preset loss function to calculate the loss value.
[0087] The disease classification model can be a single neural network model or a comprehensive model fused from multiple models. It should be noted that this application primarily focuses on obtaining the disease classification model through training, and does not limit the type of disease classification model.
[0088] In this embodiment of the application, the disease classification model is used to predict the category of optical images, and the loss value is used to measure the classification prediction loss of the disease classification model. The calculation method of the classification prediction loss can refer to related technologies, such as using cross-entropy loss to measure the classification prediction loss. The difference between the classification prediction result of the disease classification model and the category label of the corresponding optical image can be measured based on the cross-entropy loss to obtain the loss value.
[0089] In one exemplary embodiment, the disease classification model is a convolutional neural network (CNN) containing multiple layers and having independent model parameters, specifically designed for processing and analyzing hyperspectral data obtained from metasurfaces. The specific structure of this disease classification model is as follows:
[0090] 3D convolutional layers:
[0091] The first layer is a 3D convolutional layer with a kernel size of (7,3,3). This layer aims to initially extract spatial and spectral features from the input data. Next is a batch normalization layer (nn.BatchNorm3d) to accelerate the training process and improve model stability. Then comes the ReLU activation function, which increases the non-linearity of the network.
[0092] The second 3D convolutional layer has a kernel size of (5,3,3), followed by another batch normalization layer and a ReLU activation function.
[0093] The third 3D convolutional layer has a kernel size of (3,3,3), followed by a batch normalization layer and a ReLU activation function.
[0094] Attention mechanism:
[0095] Two attention mechanisms were used: Channel Attention and Spatial Attention. Channel Attention focuses on the importance of different channels, while Spatial Attention focuses on the importance of different spatial locations in the image.
[0096] 2D convolutional layers:
[0097] After applying the attention mechanism, the data undergoes further feature extraction through a 2D convolutional layer. This convolutional layer has 32 input channels multiplied by the depth of the 3D convolutional layer, 64 output channels, and a kernel size of (3,3). This is followed by a batch normalization layer and a ReLU activation function.
[0098] Fully connected layer:
[0099] The final part of the network is a classifier composed of fully connected layers.
[0100] The first fully connected layer maps the flattened features to 256 nodes, followed by a ReLU activation function and a Dropout layer (with a dropout rate of 0.5). Then, another fully connected layer maps the 256 nodes to 128 nodes, also using ReLU and Dropout. The final fully connected layer maps the features to the final number of categories.
[0101] In this embodiment, the neural network is trained using the train function. The train function initializes a historical list of loss and accuracy, sets the total number of training rounds, the dictionary of optimal network weights, and the record of the highest accuracy. At the same time, the cross-entropy loss function is used as the loss criterion, and the Adam optimizer is used to adjust the network parameters with a learning rate of 0.001.
[0102] During neural network training, the network is set to training mode at each epoch and performs forward propagation, loss calculation, backpropagation, and weight updates on the training data provided by the data loader `train_loader`. Afterward, the network switches to evaluation mode to calculate the current accuracy and adds it to the history. If the current accuracy exceeds the previous best accuracy, the best accuracy and optimal network weights are updated. At the end of each epoch, the average loss, the most recent loss value, and the current accuracy for that epoch are printed. Upon completion of training, the weights with the best accuracy are loaded, the trained network is returned along with the history of loss and accuracy, and the final hypersurface parameters are recorded.
[0103] In this embodiment of the application, after training the disease classification model and obtaining the loss value in step S203, it is further determined whether the current loss value meets the preset conditions in order to determine whether to continue training the disease classification model.
[0104] Specifically, the current loss value is compared with a loss threshold. If the current loss value is greater than the loss threshold, at least one of the M optical parameter sets is updated, or at least one of the M optical parameter sets and both the model parameters of the disease classification model are updated, and training of the disease classification model continues. If the current loss value is less than or equal to the loss threshold, it indicates that the loss function has converged, and the disease classification model training is complete. The loss threshold can be determined based on factors such as the convergence degree of the disease classification model or its classification accuracy. For example, the loss threshold can be set to a classification accuracy of 97% or 98%.
[0105] In this embodiment of the application, the model parameters of the disease classification model can be: weight parameters.
[0106] In some embodiments of this application, after updating at least one of the M optical parameter sets or updating both the M optical parameter sets and the model parameters of the disease classification model, the disease classification model is further trained using the training image set obtained by converting the preset M optical parameter sets based on the metasurface light guide element.
[0107] In other embodiments of this application, after updating at least one of the M optical parameter groups or updating both the M optical parameter groups and the model parameters of the disease classification model, each spectral image group is converted into an optical image group based on the updated M optical parameter groups of the metasurface light guide element to obtain an updated training image set, and the disease classification model is trained using the updated training image set.
[0108] This application embodiment is based on a preset set of M optical parameters for a metasurface light guide element. All spectral images in each spectral image group are converted into a single optical image group with weighted allocation. These spectral images are then configured onto the nanostructure units of the M parameterized metasurface light guide elements to obtain a training image set. This training image set is used as training samples to train a disease classification model. The model parameters and the M optical parameter sets are continuously adjusted, and the parameters of the metasurface light guide elements are serially integrated into the model. Finally, a converged disease classification model and the optimal set of M optical parameters are obtained. This application embodiment effectively combines optical design with neural network algorithm optimization, providing a precise alternative for rapid hyperspectral detection of target disease classification, significantly reducing the problems of large data volume, high cost, and slow speed associated with traditional hyperspectral imaging.
[0109] In some embodiments of this application, the metasurface light guide element includes at least one nanostructure unit, and the training method for the disease classification model further includes:
[0110] S205: Based on the optimal M optical parameter sets of metasurface light guide elements, determine the arrangement of nanostructure units on the metasurface.
[0111] In this embodiment of the application, step S205 may specifically include:
[0112] S2051: Based on the optimal M sets of optical parameters of metasurface optical guide elements, determine M types of nanostructure units;
[0113] S2052: Based on each type of nanostructure unit, determine a nanostructure unit array to obtain M nanostructure unit arrays;
[0114] S2053: Based on an array of M nanostructure units, determine the arrangement of nanostructure units on a metasurface optical guide element.
[0115] The following example illustrates how to determine the arrangement of nanostructure units on a metasurface based on the optimal set of M optical parameters of the metasurface light guide element.
[0116] Figure 3 This is a schematic diagram illustrating a nanostructure unit of a metasurface optical guide element determined based on an optimal set of optical parameters, as provided in an embodiment of this application. Figure 4 A schematic diagram illustrating the distribution of nanostructure units on a metasurface optical guide element based on a given set of M nanostructure units.
[0117] like Figure 3 As shown, firstly, based on an optimal set of optical parameters Φ' l (θ' l1 ,θ' l2 ,......,θ' lN First, a spectral transmittance curve 600 is determined; then, a nanostructure unit 4201 is determined based on the spectral transmittance curve 600. Here, determining a nanostructure unit means determining the geometric structure information of the nanostructure unit, which includes the shape and / or size of the cross-section of the nanostructure unit.
[0118] In a specific example, the disease classification system pre-stores the correspondence between the geometric structure information of multiple nanostructure units and their spectral transmittance curves. Based on this optimal set of optical parameters Φ' l (θ' l1 ,θ' l2 ,......,θ' lN The N optical parameters θ' in ) l1 ,θ' l2 ,......,θ' lN By matching a pre-stored spectral transmittance curve, the corresponding nanostructure unit can be determined based on the geometric structure information of the nanostructure unit corresponding to the spectral transmittance curve.
[0119] Here, the correspondence between the geometric structure information of the nanostructure unit and the spectral transmittance curve can be obtained by simulating the preset nanostructure unit using optical simulation software, obtaining the transmittance of the nanostructure unit in the target band, and then fitting the corresponding spectral transmittance curve.
[0120] In other specific examples, the disease classification system pre-stores the geometric structure information of multiple nanostructure units and the correspondence between a set of multiple band transmittances, based on this optimal set of optical parameters Φ' l (θ' l1 ,θ' l2 ,......,θ' lN The N optical parameters θ' in ) l1 ,θ'l2 ,......,θ' lN By matching a set of pre-stored transmittance across multiple bands, the corresponding nanostructure units can be determined based on the geometric information of the nanostructure units corresponding to the transmittance across multiple bands.
[0121] like Figure 4 As shown, after determining M nanostructure units based on M optimal optical parameter sets, a nanostructure unit array 500 is further obtained based on the arrangement of each nanostructure unit. Finally, the arrangement of nanostructure units on the entire metasurface optical guide element is determined by periodically arranging the M nanostructure unit arrays.
[0122] Figure 5 This is a schematic diagram of the structure of a disease classification system provided in an embodiment of this application. Figure 5 As shown, the disease classification system 400 includes:
[0123] Photosensitive element 410 includes M groups of pixels for sensing incident light of different wavelengths;
[0124] Metasurface light guide element 420, which is determined by M optical parameter groups according to the training method of the disease classification model of any of the above embodiments, is used to guide incident light of different wavelengths to the M groups of pixels.
[0125] The processor 430 is configured to acquire M optical images from the photosensitive element 410 and predict the disease category of the target object using a disease classification model trained according to the training method of the disease classification model according to any of the above embodiments.
[0126] In the embodiments of this application, the photosensitive element 410 can be a complementary metal-oxide-semiconductor (CMOS) type photosensitive element or an electro-coupled device (CCD) type photosensitive element; this application does not specifically limit it in this regard.
[0127] In one implementation, the metasurface light guide element 420 includes a substrate and a plurality of nanostructure units 4202 disposed on the substrate. The nanostructure units 4202 may have a shape and size smaller than the wavelength of the incident light. The nanostructure units 4202 may be disposed on the side of the substrate closer to the photosensitive element 410 or on the side of the substrate farther from the photosensitive element 410. The plurality of nanostructure units on the metasurface light guide element 420 are determined and arranged according to M sets of optical parameters determined by the training method of the disease classification model of any of the above embodiments.
[0128] The disease classification system 400 may include only one metasurface light guide element 420, or it may include multiple metasurface light guide elements 420 stacked together. When the disease classification system 400 includes multiple metasurface light guide elements 420 stacked together, the multiple nanostructure units 4202 of each metasurface light guide element 420 disposed on the substrate 4201 may be disposed on the side closer to the photosensitive element 410, or they may be disposed on the side farther away from the photosensitive element 410.
[0129] The nanostructure unit 4201 can be a three-dimensional structure with width (w) and thickness (t), such as a cuboid shape, a hexahedron shape, a cylinder shape or a disk shape, which is not limited in this application.
[0130] The processor 430 can be a model training device or equipment, such as a computer, server, or server cluster with model training capabilities.
[0131] In an exemplary embodiment of this application, each pixel of the photosensitive element is provided with a corresponding filter unit. Each filter unit can filter light waves so that only light of a preset wavelength band can pass through and be incident on the corresponding pixel, ultimately enabling each group of pixels to sense incident light of different wavelength bands. In this embodiment, when the disease classification system 400 is used to implement the training method of the disease classification model, it first needs to convert each spectral image group into an optical image group based on the preset M optical parameter groups of the metasurface light guide element 410 to obtain a training image set, and then use the training image set to train the disease classification model. When the disease classification system 400 is used to implement the disease classification method, the processor 430 acquires M optical images from the photosensitive element 420 and uses the disease classification model trained according to the training method of any of the above-mentioned disease classification models to predict the disease category of the target object based on the acquired M optical images.
[0132] Figure 6 This is a flowchart illustrating a disease classification method provided in an embodiment of this application. The subject executing this method is the disease classification system described in any of the above embodiments.
[0133] like Figure 6 As shown, the disease classification method includes, but is not limited to, the following steps:
[0134] S401: Using the photosensitive element described in any of the above embodiments, the target object is photographed to obtain M optical images of the target object;
[0135] S402: Using the processor described in any of the above embodiments, predict the disease category of the target object based on M optical images.
[0136] Furthermore, in order to implement the method of the embodiments of this application, the embodiments of this application also provide a computer device, which may be a terminal device or a server. Figure 7 This is merely an exemplary structure of the computer device, not the entire structure; implementation is possible as needed. Figure 7 The structure shown may be part or all of the structure.
[0137] like Figure 7 As shown, the computer device 1000 provided in this embodiment includes at least one processor 1001, a memory 1002, a user interface 1003, and at least one network interface 1004. The various components in the electronic device 1000 are coupled together via a bus system 1005. It can be understood that the bus system 1005 is used to implement communication between these components. In addition to a data bus, the bus system 1005 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 7 The general labeled all buses as Bus System 1005.
[0138] The user interface 1003 may include a monitor, keyboard, mouse, trackball, click wheel, buttons, touchpad, or touch screen.
[0139] The memory 1002 in this embodiment is used to store various types of data to support the operation of the computer device. Examples of such data include any computer program used to operate on the computer device.
[0140] The disease classification model training method or disease classification method disclosed in this application embodiment can be applied to or implemented by the processor 1001. The processor 1001 may be an integrated circuit chip with signal processing capabilities. In the implementation process, the training method or disease classification method of the disease classification model can be completed by the integrated logic circuit of the hardware or by the instructions in the software form of the processor 1001. The processor 1001 mentioned above can be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 1001 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiments of this application can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in memory 1002. The processor 1001 reads the information in memory 1002 and, in conjunction with its hardware, completes the steps of the positioning method provided in the embodiments of this application.
[0141] In an exemplary embodiment, the computer device may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned methods.
[0142] It is understood that memory 1002 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memories.
[0143] It should be noted that this application also provides a computer-readable storage medium storing a computer program executed by the aforementioned data processing apparatus. When the processor executes the computer program, it can perform the data processing method described in the corresponding embodiments 1 or 6 above; therefore, it will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the embodiments of the computer-readable storage medium involved in this application, please refer to the description of the method embodiments of this application.
[0144] It should be noted that: This application also provides a computer program product, which may include a computer program that can be stored in a computer-readable storage medium. The processor of a computer device reads the computer program from the computer-readable storage medium, and the processor can execute the computer program, causing the computer device to perform the aforementioned... Figure 1 or Figure 6 The description of the data processing method in the corresponding embodiments is already provided and will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the computer program product embodiments related to this application, please refer to the description of the method embodiments of this application.
[0145] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.
[0146] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc.
[0147] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.
[0148] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.
Claims
1. A training method for a disease classification model, characterized in that, The disease classification model is used in a disease classification system, which includes a photosensitive element and a metasurface light guide element. The photosensitive element includes M sets of pixels for sensing incident light of different wavelengths, and the metasurface light guide element is used to guide incident light of different wavelengths to the M sets of pixels. The training method includes: Acquire a spectral image set, wherein the spectral image set includes multiple spectral image groups and a category label corresponding to each spectral image group, and each spectral image group includes N spectral images of the same target object obtained by imaging, wherein N is a positive integer; Based on the preset M optical parameter groups of the metasurface light guide element, each spectral image group is converted into an optical image group to obtain a training image set, where M is a positive integer; The disease classification model is trained using the training image set to obtain a loss value; Based on the loss value, at least one of the M optical parameter sets of the metasurface light guide element is updated; alternatively, at least one of the M optical parameter sets of the metasurface light guide element and both the model parameters of the disease classification model are updated until the disease classification model converges, thus obtaining the trained disease classification model and the optimal M optical parameter sets of the metasurface light guide element. The method of converting each spectral image group into an optical image group based on the preset M optical parameter groups of the metasurface light guide element to obtain a training image set includes: coupling each spectral image group with the M optical parameter groups of the metasurface light guide element to obtain an optical image group corresponding to each spectral image group, wherein each time a spectral image group is coupled with an optical parameter group, an optical image of the corresponding optical image group is obtained; The process of coupling each of the spectral image groups with a set of optical parameters to obtain an optical image of the corresponding optical image group includes: multiplying all the spectral images in the spectral image group by an optical parameter from each of the optical parameter groups and then summing the products to obtain an optical image of the optical image group corresponding to the spectral image group; and wherein... Each of the optical parameter groups includes N optical parameters, each optical parameter being used to indicate the transmittance of a nanostructure unit of the metasurface light guide element to a wavelength or band of light; The step of multiplying all the spectral images in the spectral image group with one of the optical parameters in the optical parameter group and then summing the products to obtain an optical image of the optical image group corresponding to the spectral image group includes: multiplying the spectral images of N bands in the spectral image group with the N optical parameters in the optical parameter group one by one and then summing the products to obtain an optical image of the optical image group corresponding to the spectral image group.
2. The method according to claim 1, characterized in that, The M satisfies: .
3. The method according to claim 1, characterized in that, The wavelength range of the N bands is between 400 nm and 1000 nm.
4. The method according to claim 1, characterized in that, The method further includes: Based on the optimal M sets of optical parameters of the metasurface light guide element, the arrangement of the nanostructure units on the metasurface light guide element is determined.
5. The method according to claim 4, characterized in that, The arrangement of nanostructure units on the metasurface light guide element is determined based on the optimal M sets of optical parameters, including: Based on the optimal M optical parameter sets of the metasurface optical guide element, M types of nanostructure units are determined; Based on each of the aforementioned nanostructure units, a nanostructure unit array is determined, resulting in M nanostructure unit arrays; Based on the array of M nanostructure units, the arrangement of the nanostructure units on the metasurface optical guide element is determined.
6. A disease classification system, characterized in that, include: A photosensitive element, the photosensitive element comprising M groups of pixels for sensing incident light of different wavelengths; A metasurface light guide element, wherein the metasurface light guide element is determined by M optical parameter groups as determined by the training method of the disease classification model according to any one of claims 1-5, and is used to guide incident light of different wavelengths to the M groups of pixels; A processor is configured to acquire M optical images from the photosensitive element and predict the disease category of the target object using a disease classification model trained according to any one of claims 1-5.
7. A method for classifying diseases, characterized in that, The disease classification method is used in the disease classification system of claim 6, and the method includes: The target object is photographed using the photosensitive element to obtain M optical images of the target object; Using the processor, the disease category of the target object is predicted based on the M optical images.
8. A computer device, comprising: At least one processor; A memory storing a computer program executable on the processor, characterized in that the processor executes the program and performs the steps of the method as described in any one of claims 1 to 5.
9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it performs the steps of the method as described in any one of claims 1 to 5.