A skin cancer detection method, device, medium and product based on a self-attention neural network

Abstract

This application discloses a skin cancer detection method, device, medium, and product based on a self-attention neural network, relating to the field of medical image processing technology. The method includes: decomposing a polarized speckle image of a patient to be detected into parallel and vertical polarized speckle images; employing a skin cancer detection network model to obtain skin cancer detection results based on these two types of images; a multi-scale token generator in the model divides these two types of images into image patches of different scales; a statistical perception controller determines statistical perception weights based on the distribution differences between the image patches and the global distribution; an attention mechanism module performs collaborative work on intra-channel feature modeling and inter-channel information interaction to obtain parallel and vertical polarization fusion features; and a classifier obtains skin cancer detection results based on the parallel and vertical polarization fusion features. This application can improve the accuracy and robustness of skin cancer detection, providing an efficient and feasible technical solution for intelligent diagnosis of skin cancer based on novel imaging modalities.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and in particular to a method, device, medium and product for skin cancer detection based on a self-attention neural network. Background Technology

[0002] Polarized speckle is a random optical interference pattern. When coherent or partially coherent light (such as light from a laser) is scattered from a turbid medium (such as skin tissue), each photon undergoes a random walk path in the medium, resulting in a corresponding phasor statistical distribution. These phasors are added together to form an interference pattern called speckle, or speckle pattern. The speckle pattern encodes the optical scattering properties of the object in its statistical characteristics, rather than directly imaging the object. Much work on speckle has focused on its spatial intensity distribution, which has been applied in rapid and non-contact surface roughness metrology. In addition to intensity, light also has a polarization state, which refers to the oscillating orientation of its electric field. An extension of intensity speckle also considers its spatial polarization state distribution, referred to here as polarized speckle, an emerging method with significant implications for biomedical applications. The statistical characteristics of intensity speckle are encoded by an intensity distribution (scalar) determined by the distribution information of phasor path lengths in the medium. The characteristics of polarized speckle are encoded through a polarization state distribution (3D vector), determined by the scattering and absorption properties of the tissue. This gives polarized speckle imaging a unique advantage in distinguishing skin diseases (such as skin cancer and moles) that are similar in color or shape but have different internal structures. In this way, speckle can serve as an alternative to or complement to traditional clinical and dermoscopy imaging techniques, which have achieved great success in the field of artificial intelligence. Although polarized speckle patterns possess rich statistical features, they lack interpretable features such as shape or boundaries, making it difficult for traditional techniques to directly extract visual features.

[0003] The statistical characteristics of speckle imaging require AI models to capture nonlocal dependencies and extract subtle patterns, a requirement that goes beyond traditional feature extraction methods. Self-attention mechanisms possess excellent capabilities in capturing global dependencies and nonlocal relationships. These mechanisms, by dynamically allocating attention weights, enable the model to focus on the most relevant features, regardless of their spatial or temporal distribution. This makes self-attention particularly suitable for analyzing polarized speckle patterns, characterized by statistical randomness and nonlocal dependencies. By leveraging the advantages of self-attention, non-obvious statistical features in speckle data can be effectively extracted.

[0004] However, current self-attention models are mainly designed for general image analysis and are not yet able to perform more robust and in-depth feature extraction for the unique statistical characteristics and multidimensional information of polarized speckle images, making it difficult to fully explore the potential statistical regularities in speckle images. Summary of the Invention

[0005] To address the aforementioned problems in the existing technology, this application provides a skin cancer detection method, device, medium, and product based on a self-attention neural network, which can improve the accuracy and robustness of skin cancer detection, thereby providing an efficient and feasible technical solution for intelligent diagnosis of skin cancer based on a novel imaging modality.

[0006] To achieve the above objectives, this application provides the following solution: In a first aspect, this application provides a skin cancer detection method based on a self-attention neural network, comprising: Acquire polarized speckle images of the patient to be tested; The polarized speckle image is decomposed into a parallel polarized speckle image and a vertical polarized speckle image; A skin cancer detection network model is obtained, and using this model, skin cancer detection results are obtained based on the parallel polarization speckle image and the vertical polarization speckle image. The skin cancer detection network model is constructed based on a self-attention neural network and includes: a multi-scale token generator, a statistical perception controller, an attention mechanism module, and a classifier. The multi-scale token generator is used to divide the parallel polarization speckle image and the vertical polarization speckle image into image patches of different scales. The statistical perception controller is used to determine statistical perception weights based on the distribution differences between the image patches of different scales and the global distribution. The attention mechanism module incorporates the statistical perception weights for collaborative work of intra-channel feature modeling and inter-channel information interaction to obtain parallel polarization fusion features and vertical polarization fusion features. The classifier is used to obtain the skin cancer detection results based on the parallel polarization fusion features and the vertical polarization fusion features.

[0007] Secondly, this application provides a skin cancer detection device based on a self-attention neural network, comprising: Image acquisition device, used to acquire polarized speckle images of the patient to be tested; A non-polarizing beam splitter, connected to the image acquisition unit, is used to decompose the polarized speckle image into a parallel polarized speckle image and a vertical polarized speckle image. A processor, connected to the non-polarization beam splitter, is used to acquire a skin cancer detection network model and, using this model, obtain skin cancer detection results based on the parallel polarization speckle image and the vertical polarization speckle image. The skin cancer detection network model is constructed based on a self-attention neural network and includes: a multi-scale token generator, a statistical perception controller, an attention mechanism module, and a classifier. The multi-scale token generator is used to divide the parallel polarization speckle image and the vertical polarization speckle image into image patches of different scales. The statistical perception controller is used to determine statistical perception weights based on the distribution differences between the image patches of different scales and the global distribution. The attention mechanism module incorporates these statistical perception weights for collaborative work of intra-channel feature modeling and inter-channel information interaction, obtaining parallel polarization fusion features and vertical polarization fusion features. The classifier is used to obtain the skin cancer detection results based on the parallel polarization fusion features and the vertical polarization fusion features.

[0008] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the skin cancer detection method based on the self-attention neural network provided above.

[0009] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the skin cancer detection method based on a self-attention neural network provided above.

[0010] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the skin cancer detection method based on a self-attention neural network provided above.

[0011] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides a skin cancer detection method, device, medium, and product based on a self-attention neural network. In the multi-scale token generator of the skin cancer detection network model, speckle images from two channels (parallel polarization and vertical polarization) are used to construct multi-granularity image patch representations (i.e., generating image patches of different scales). An attention mechanism module is used to collaboratively model intra-channel features and facilitate inter-channel information interaction, achieving cross-channel information fusion. Simultaneously, statistically perceptual weights are introduced into the self-attention framework to learn the statistical characteristics of speckle images in different channels and perform weighted optimization of features, thereby improving the skin cancer detection network model's ability to model implicit statistical information and effectively overcoming the difficulty in feature representation caused by the randomness of polarized speckle images. Compared to traditional methods that rely on explicit structural features, this application can more accurately uncover the potential statistical patterns in speckle images, improving the accuracy and robustness of skin cancer detection, and providing an efficient and feasible technical solution for intelligent skin cancer diagnosis based on novel imaging modalities. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0013] Figure 1 A schematic flowchart of a skin cancer detection method based on a self-attention neural network provided in an embodiment of this application; Figure 2 A schematic diagram showing the comparison results of classification performance of different methods in a comparative experiment provided for an embodiment of this application; Figure 3 A schematic diagram showing the comparison of classification performance of different module combinations in an ablation experiment provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a skin cancer detection device based on a self-attention neural network according to an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0014] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0015] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, this application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0016] In one exemplary embodiment, this application provides a skin cancer detection method based on a self-attention neural network. This method is executed by a computer device, specifically a terminal or server, or both. In this embodiment, the method is described using a server as an example. Figure 1 As shown, the method includes: Step 100: Acquire polarized speckle images of the patient to be tested.

[0017] Step 101: Decompose the polarized speckle image into a parallel polarized speckle image and a vertical polarized speckle image.

[0018] Step 102: Obtain the skin cancer detection network model and use it to obtain skin cancer detection results based on parallel polarization speckle images and vertical polarization speckle images. The skin cancer detection network model is constructed based on a self-attention neural network, which includes: a multi-scale token generator, a statistical perception controller, an attention mechanism module, and a classifier. The multi-scale token generator is used to divide the parallel polarization speckle images and vertical polarization speckle images into image patches of different scales. The statistical perception controller is used to determine statistical perception weights based on the distribution differences between image patches of different scales and the global distribution. The attention mechanism module introduces statistical perception weights to perform collaborative work on intra-channel feature modeling and inter-channel information interaction, obtaining parallel polarization fusion features and vertical polarization fusion features. The classifier is used to obtain skin cancer detection results based on the parallel polarization fusion features and vertical polarization fusion features.

[0019] To address the issues of strong randomness, weak semantic information, and difficulty in effectively representing polarized speckle images using traditional shape, boundary, or color features, this application, by implementing steps 100-102, can more accurately uncover the potential statistical patterns in speckle images, improve the accuracy and robustness of skin cancer detection, and thus provide an efficient and feasible technical solution for intelligent skin cancer diagnosis based on a novel imaging modality, facilitating auxiliary assessment for clinicians.

[0020] In an exemplary embodiment of this application, due to the randomness of polarized speckle images, traditional self-attention may be overly sensitive to accidental textures and fail to fully utilize the information in the samples. To address this issue, in step 102 above, a multi-scale token generator is used to extract relevant statistical evidence. Based on this, the process by which the multi-scale token generator divides the parallel polarized speckle image and the vertical polarized speckle image into image blocks of different scales can be described as follows: In the multi-scale token generator, the parallel polarization speckle image and the vertical polarization speckle image are divided into three different scale image patches (fine, medium, and coarse) in a non-overlapping manner. Among them, the fine scale preserves local fluctuations, the medium scale extracts correlations, and the coarse scale summarizes global trends.

[0021] The sizes of the image blocks of three different scales—fine, medium, and coarse—are H1×W1 (e.g., 32×32), H2×W2 (e.g., 64×64), and H3×W3 (e.g., 128×128), respectively, to meet the requirement that the fine scale is smaller than the medium scale and the medium scale is smaller than the coarse scale.

[0022] Furthermore, the formulas for calculating the number of image patches at three different scales—fine, medium, and coarse—are as follows: .

[0023] In the formula, The number of image patches, Image height, This represents the image width. For the first Several scales, among which For fine scale, Mesoscale, This is a coarse scale.

[0024] Furthermore, when the size of the parallel polarized speckle image or the vertical polarized speckle image cannot be divided by the size of the corresponding image block, the edges of the parallel polarized speckle image or the vertical polarized speckle image are zero-filled before dividing the image blocks into different scales.

[0025] Based on the above description, in practical applications, to facilitate subsequent processing, each image patch can be mapped to a feature vector, as follows: .

[0026] In the formula, The mapped feature vector, For the first The first of the scales Image blocks, Mapping function, express 3D real space.

[0027] In an exemplary embodiment of this application, although multi-scale information enriches the features, the learning of random texture features remains unstable. To address this issue, this application designs a statistically perceptual controller in the aforementioned skin cancer detection network model to assign reliability weights, suppress regions with strong randomness and statistical instability, and improve model performance. Based on this, the process by which the statistically perceptual controller determines statistically perceptual weights based on the distribution differences between image patches at different scales and the global distribution may include: Step 1: Based on the EMD (Earth Mover's Distance), use the formula Determine the distribution differences between image patches at different scales and the global distribution. For the first The first of the scales The distribution difference between individual image patches and the global distribution. For the first The first of the scales Distribution of image patches, It is a global distribution.

[0028] Step 2: Based on the distribution difference between image patches and the global distribution, use the formula Determine the statistical perception weights. In the formula, For the first The first of the scales Statistical perceptual weights corresponding to each image patch To adjust the parameters.

[0029] In an exemplary embodiment of this application, by introducing statistically perceptual weights into the attention mechanism, the attention mechanism module provided in this application can include self-attention and cross-channel attention. These two attention mechanisms are not simply concatenated after independent processing; instead, they work collaboratively to achieve feature fusion and interaction by first modeling intra-channel features and then performing inter-channel information interaction. Specifically, the multi-scale token sequences with statistically reliable weights in the obtained parallel polarization image channels and vertical polarization image channels perform self-attention calculations within their respective channels to obtain features within each channel, thereby modeling the relationship between different image blocks within the same polarization channel. Based on this, cross-channel attention calculation is then performed, using the intra-channel features of the two polarization channels as input to achieve cross-channel information interaction and fusion. Therefore, the process of introducing statistically perceptual weights into the attention mechanism module to perform collaborative work of first modeling intra-channel features and then performing inter-channel information interaction to obtain parallel polarization fusion features and vertical polarization fusion features can include: Step 1: In the attention mechanism module, statistical perceptual weights are introduced into the attention mechanism, and statistical perceptual self-attention calculation is performed on each image block to obtain the parallel polarization channel features and the vertical polarization channel features.

[0030] In practical applications, the process of calculating statistical perception self-attention can be described as follows: (1) Weighting features within the same polarization channel (i.e., feature vectors formed by image patch mapping) enhances the modeling ability between features, resulting in: .

[0031] In the formula, For query matrix, This is the transpose of the key matrix. For feature dimension, Here is the self-attention weight matrix: (2) Introduce a statistical reliability weight matrix ,get: .

[0032] In the formula, For statistical perception weights, This is the updated self-attention weight matrix.

[0033] (3) Output characteristics : .

[0034] In the formula, It is a value matrix.

[0035] Based on the above description, the statistically perceived self-attention features of the parallel polarization channel and the vertical polarization channel, namely the parallel polarization channel features and the vertical polarization channel features, are respectively... and They respond differently to scattering information and texture details, and the information they contain is complementary.

[0036] Step 2: Perform cross-channel attention calculation on the parallel polarization channel features and the vertical polarization channel features to obtain the first interactive feature and the second interactive feature.

[0037] In practical applications, the complementary information between parallel polarization channel features and vertical polarization channel features is utilized to calculate cross-channel attention. For example, parallel polarization channel features can be used as the query... Using vertical polarization channel characteristics as a key Sum The attention weights of the parallel polarization channel to the vertical polarization channel are calculated to obtain the cross-channel attention weight matrix. ,have: .

[0038] At this point, a statistical reliability weight matrix is ​​introduced. The attention weight matrix across channels is obtained. ,have: .

[0039] Therefore, the final output feature (i.e., the first interaction feature) after this cross-channel attention calculation is: : .

[0040] Similarly, using the vertical polarization channel characteristics as a query... Parallel polarization channel characteristics as a key Sum Then, another cross-channel attention calculation is performed to finally obtain the second interaction feature. This enables bidirectional information exchange between the two polarization channels.

[0041] Step 3: Fuse the parallel polarization channel features, vertical polarization channel features, first interactive features, and second interactive features to obtain the parallel polarization fusion features and the vertical polarization fusion features.

[0042] In practical applications, the features output by cross-channel attention are not directly fed into the subsequent classifier. Instead, they are added (i.e. fused) to the corresponding channel features updated by statistical perception self-attention, resulting in parallel polarization fused features. and vertical polarization fusion features Therefore, the skin cancer detection network model provided in this application can achieve information fusion to learn scattering information under different polarization states, enhance the stability of classification results, and provide reliable support for subsequent assisted manual diagnosis. Specifically: .

[0043] .

[0044] In an exemplary embodiment of this application, after the above-described extraction and fusion of skin lesion image features, in this embodiment, the process by which the classifier obtains skin cancer detection results based on parallel polarization fusion features and vertical polarization fusion features may include: Step 1: Global average pooling is used to obtain parallel pooling features and vertical pooling features based on parallel polarization fusion features and vertical polarization fusion features. For example, regarding the parallel polarization fusion features... and vertical polarization fusion features Parallel pooling features are obtained by performing global average pooling separately. and vertical pooling features .

[0045] Step 2: Concatenate the parallel pooling features and the vertical pooling features to obtain the concatenated features. The concatenated features are represented as follows: : .

[0046] Step 3: A multilayer perceptron classifier is used to obtain the classification probability based on the concatenated features. The classification probability is used to characterize the skin cancer detection results (including categories such as melanoma and benign nevi).

[0047] This classification probability can provide data support and reference for subsequent skin cancer assessment, assisting clinicians in evaluating skin lesions. In the formula, For classification probability, For multilayer perceptron classifiers, This refers to the Softmax function.

[0048] In one exemplary embodiment of this application, to achieve optimal performance of the skin cancer detection network model, a statistically perceptual self-attention neural network needs to be constructed based on a self-attention neural network before obtaining the skin cancer detection network model, and the constructed statistically perceptual self-attention neural network needs to be trained. Based on this, in this embodiment, to obtain the optimal skin cancer detection network model under real clinical conditions and controlled optical settings, experiments can be conducted on two types of polarized speckle datasets: a clinical dataset and a simulation dataset, to form a dataset. This dataset is then divided into a training set, a test set, and a validation set according to a set ratio (e.g., 7:2:1).

[0049] The dataset includes clinical data from 128 participants, totaling 432 images (149 images of malignant melanoma from 37 participants and 283 images of moles from 91 participants). The simulation dataset contains 4200 images, collected from rough skin with four different skin tones. All images are encoded in 1024p. Images were captured at a resolution of 768. Due to the limited number of participants in the clinical dataset, 5-fold cross-validation was performed on the clinical dataset. These polarization speckle images were decomposed into parallel and perpendicular polarization components by an unpolarization beamsplitter and then fed into the constructed statistical perceptual self-attention neural network. The parallel polarization component image and the perpendicular polarization component image are denoted as follows: and .

[0050] The constructed statistically perceptual self-attention neural network is trained, tested, and validated using the divided training, test, and validation sets until a set number of iterations (e.g., 100) is reached. The trained statistically perceptual self-attention neural network is then used as a skin cancer detection network model.

[0051] In one exemplary embodiment of this application, to verify the effectiveness of the method provided, the classification performance of the skin cancer detection network model (hereinafter referred to as the model) is evaluated using the following metrics: accuracy (ACC), precision, sensitivity, specificity, and area under the curve (AUC). The model provided in this application is compared with other conventional methods through comparative experiments and ablation experiments to demonstrate the effectiveness of the module proposed in this application. Figure 2 and Figure 3 The results shown demonstrate that this application outperforms traditional methods under various evaluation metrics. Ablation experiments also prove that the module proposed in this application produces superior results. The traditional methods used in this embodiment include random forest, Support Vector Machine with Radial Basis Function kernel (SVM-RBF), logistic regression, neural networks, Patch Voting ResNet (PV-ResNet), Dual Vision Transformer (Dual-ViT), Group-Wise Feature Enhancement-and-Fusion Network (GWFEF-Net), Deep Learning-based Laser Speckle Contrast Imaging Model (DL-LSCI), and a model combining a sliding window transformer with Long Short-Term Memory (SwinLSTM).

[0052] Specifically, the method provided in this application (corresponding to) Figure 2 and Figure 3The method described in this paper achieved a sensitivity of 94.1±1.2%, a specificity of 52.5±1.8%, and a precision of 77.7±1.1%. These data indicate that this application can not only efficiently identify malignant melanoma but also significantly reduce the false positive rate and improve the reliability of diagnosis. Furthermore, the method provided in this application, after using the statistically perceptual self-attention (SC-SA) module, improved accuracy by 2% and AUC by 5.4%, demonstrating that the statistically perceptually weighted self-attention mechanism is more robust to processing random speckle image patterns.

[0053] Compared to traditional methods that use spatial or temporal features alone, this application integrates multi-dimensional spatiotemporal features (i.e., parallel and vertical polarization images), effectively improving the model's classification performance. Furthermore, the stability and accuracy of the classification results are enhanced through multi-scale and cross-channel self-attention modules.

[0054] In the ablation experiments, this application used a dual-channel convolutional network as the baseline model. Introducing single-channel self-attention (SC) improved model performance (ACC increased from 68.5% to 70.9%, and AUC increased from 71.7% to 74.1%), indicating that self-attention is helpful for speckle feature modeling. Introducing cross-channel self-attention (CC) alone also improved AUC, indicating complementary information between different polarization channels. Combining single-channel self-attention (SC) and cross-channel attention (i.e., SC+CC) further improved performance (ACC reached 73.4%, and AUC reached 78.2%), demonstrating a synergistic effect between intra-channel modeling and inter-channel fusion. Building on this, introducing a statistical awareness mechanism to weight and optimize single-channel statistic-aware self-attention (SC-SA) significantly improved model performance (ACC reached 77.2%, and AUC reached 83.1%), indicating that statistical awareness can effectively enhance feature representation. Ultimately, the complete model integrating statistical sensing mechanism and multi-channel attention (i.e., introducing statistical sensing cross-channel attention (CC-SA)) achieved the best performance (ACC of 79.1% and AUC of 89.2%), indicating that this application significantly improves the discrimination ability of polarization speckle images through the synergistic effect of statistical sensing weighting and multi-channel feature fusion.

[0055] In summary, the method provided in this application enhances the self-attention learning mechanism by introducing an attention controller based on Earth mobile distance, integrates data from parallel and vertical polarization channels to utilize polarization state information, and combines single-channel information with cross-channel attention weight allocation to perform more robust and in-depth feature extraction on polarization speckle data. Based on polarization speckle images, it enables the extraction and analysis of features related to skin cancer (such as melanoma), thereby helping clinicians to assess skin cancer and reducing the burden on doctors.

[0056] Based on the same inventive concept, this application also provides a skin cancer detection device based on a self-attention neural network for implementing the skin cancer detection method based on a self-attention neural network described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations of one or more embodiments of the skin cancer detection device based on a self-attention neural network provided below can be found in the limitations of the skin cancer detection method based on a self-attention neural network described above, and will not be repeated here.

[0057] In one exemplary embodiment, such as Figure 4 As shown, a skin cancer detection device based on a self-attention neural network is provided, comprising: an image acquisition unit 200, a non-polarization beam splitter 201, and a processor 202. The non-polarization beam splitter 201 is connected to the image acquisition unit 200. The processor 202 is connected to the non-polarization beam splitter 201.

[0058] Image acquisition unit 200 acquires polarized speckle images of the patient to be detected. Non-polarized beam splitter 201 decomposes the polarized speckle images into parallel polarized speckle images and vertical polarized speckle images. Processor 202 acquires a skin cancer detection network model and, using this model, obtains skin cancer detection results based on the parallel and vertical polarized speckle images. The skin cancer detection network model includes a multi-scale token generator, a statistical sensing controller, an attention mechanism module, and a classifier. The multi-scale token generator is used to divide the parallel and vertical polarized speckle images into image patches of different scales. The statistical sensing controller determines statistical sensing weights based on the distribution differences between image patches of different scales and the global distribution. The attention mechanism module introduces statistical sensing weights for collaborative work in intra-channel feature modeling and inter-channel information interaction, obtaining parallel polarization fusion features and vertical polarization fusion features. The classifier obtains skin cancer detection results based on the parallel and vertical polarization fusion features.

[0059] As an optional implementation, the processor 202 used in this application can be a computer or other intelligent electronic device.

[0060] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 5 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores skin cancer detection data based on a self-attention neural network. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a skin cancer detection method based on a self-attention neural network.

[0061] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer equipment to which the present application is applied. Specific computer equipment may include, for example, [the following is a list of possible additional structures]. Figure 5 The diagram shows more or fewer components, or combinations of certain components, or different component arrangements.

[0062] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0063] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0064] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0065] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0066] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (RRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0067] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0068] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0069] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A skin cancer detection method based on a self-attention neural network, characterized in that, include: Acquire polarized speckle images of the patient to be tested; The polarized speckle image is decomposed into a parallel polarized speckle image and a vertical polarized speckle image; A skin cancer detection network model is obtained, and using the skin cancer detection network model, based on the parallel polarized speckle image and the vertical polarized speckle image, the skin cancer detection result is obtained; The skin cancer detection network model is constructed based on a self-attention neural network, comprising: a multi-scale token generator, a statistical perception controller, an attention mechanism module, and a classifier. The multi-scale token generator is used to divide the parallel polarization speckle image and the vertical polarization speckle image into image patches of different scales. The statistical perception controller is used to determine statistical perception weights based on the distribution differences between the image patches of different scales and the global distribution. The attention mechanism module incorporates the statistical perception weights to perform collaborative work of intra-channel feature modeling and inter-channel information interaction, obtaining parallel polarization fusion features and vertical polarization fusion features. The classifier is used to obtain the skin cancer detection result based on the parallel polarization fusion features and the vertical polarization fusion features.

2. The skin cancer detection method based on a self-attention neural network according to claim 1, characterized in that, The process by which the multi-scale token generator divides the parallel polarization speckle image and the vertical polarization speckle image into image blocks of different scales includes: In the multi-scale token generator, the parallel polarized speckle image and the vertical polarized speckle image are divided into three image blocks of different scales—fine, medium, and coarse—in a non-overlapping block format. The sizes of the three image blocks are H1×W1, H2×W2, and H3×W3, respectively. The formula for calculating the number of image blocks of the three scales is as follows: ； In the formula, The number of image patches, Image height, Image width; For the first One scale, Corresponding to fine, medium, and coarse.

3. The skin cancer detection method based on a self-attention neural network according to claim 2, characterized in that, When the size of the parallel polarized speckle image or the size of the vertical polarized speckle image cannot be divided by the size of the corresponding image block, the edges of the parallel polarized speckle image or the edges of the vertical polarized speckle image are zero-filled before the image blocks of different scales are divided.

4. The skin cancer detection method based on a self-attention neural network according to claim 1, characterized in that, The process by which the statistical sensing controller determines statistical sensing weights based on the distribution differences between image patches at different scales and the global distribution includes: Based on EMD distance, the formula is used. Determine the distribution differences between image patches at different scales and the global distribution; For the first The first of the scales The distribution difference between individual image patches and the global distribution. For the first The first of the scales Distribution of image patches, It is a global distribution; Based on the distribution differences between image patches and the global distribution, the formula is used. Determine the statistical perception weights; where, For the first The first of the scales Statistical perceptual weights corresponding to each image patch To adjust the parameters.

5. The skin cancer detection method based on a self-attention neural network according to claim 1, characterized in that, The attention mechanism module introduces the statistical perception weights to perform collaborative work of first modeling features within a channel, and then exchanging information between channels to obtain parallel polarization fusion features and vertical polarization fusion features. The process includes: In the attention mechanism module, statistical perceptual weights are introduced into the attention mechanism, and statistical perceptual self-attention calculation is performed on each image block to obtain parallel polarization channel features and vertical polarization channel features. Cross-channel attention calculations are performed on the parallel polarization channel features and the vertical polarization channel features to obtain the first interactive feature and the second interactive feature; By fusing the parallel polarization channel feature, the vertical polarization channel feature, the first interaction feature, and the second interaction feature, the parallel polarization fusion feature and the vertical polarization fusion feature are obtained.

6. The skin cancer detection method based on a self-attention neural network according to claim 1, characterized in that, The process by which the classifier obtains the skin cancer detection result based on the parallel polarization fusion feature and the vertical polarization fusion feature includes: Global average pooling is employed to obtain parallel pooling features and vertical pooling features based on the parallel polarization fusion features and the vertical polarization fusion features; The concatenated feature is obtained by splicing the parallel pooling feature and the vertical pooling feature; A multilayer perceptron classifier is used to obtain a classification probability based on the concatenated features; the classification probability is used to characterize the skin cancer detection result.

7. A skin cancer detection device based on a self-attention neural network, characterized in that, include: Image acquisition device, used to acquire polarized speckle images of the patient to be tested; A non-polarizing beam splitter, connected to the image acquisition unit, is used to decompose the polarized speckle image into a parallel polarized speckle image and a vertical polarized speckle image. The processor, connected to the non-polarized beam splitter, is used to acquire a skin cancer detection network model and, using the skin cancer detection network model, obtain skin cancer detection results based on the parallel polarized speckle image and the vertical polarized speckle image. The skin cancer detection network model is constructed based on a self-attention neural network, comprising: a multi-scale token generator, a statistical perception controller, an attention mechanism module, and a classifier. The multi-scale token generator is used to divide the parallel polarization speckle image and the vertical polarization speckle image into image patches of different scales. The statistical perception controller is used to determine statistical perception weights based on the distribution differences between the image patches of different scales and the global distribution. The attention mechanism module incorporates the statistical perception weights to perform collaborative work of intra-channel feature modeling and inter-channel information interaction, obtaining parallel polarization fusion features and vertical polarization fusion features. The classifier is used to obtain the skin cancer detection result based on the parallel polarization fusion features and the vertical polarization fusion features.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that the processor executes the computer program to implement the skin cancer detection method based on a self-attention neural network as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the skin cancer detection method based on a self-attention neural network as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the skin cancer detection method based on a self-attention neural network as described in any one of claims 1-6.

Images