Thyroid nodule detection model construction method

Abstract

The invention relates to the technical field of thyroid nodule detection, in particular to a thyroid nodule detection model construction method, which comprises the following steps of: performing multi-modal imaging on a thyroid nodule by using an image enhancement technology and a photoacoustic imaging technology; building a thyroid nodule detection model based on the built multi-modal image data; and optimizing the thyroid nodule detection model by using the loss function, according to the method, various modal data such as ultrasonic images, photoacoustic imaging and gene data are fused, the advantages of each type of data are fully utilized, the possible limitation of single modal data is overcome, and the comprehensiveness and accuracy of detection are improved; a graph neural network is used for gradually extracting high-order relation features between nodule samples, global embedding features are generated, and the complex nonlinear relation of the nodule samples can be captured; through the attention mechanism, the important relationship between nodule samples can be automatically identified and weighted, the feature redundancy is reduced, and the robustness and accuracy of the model are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of thyroid nodule detection, and in particular to a method for constructing a thyroid nodule detection model. Background Art

[0002] Thyroid nodules are one of the common clinical diseases, and their incidence rate is increasing year by year. Evaluating the nature of thyroid nodules through effective detection methods is of great significance for the early diagnosis and treatment of patients. At present, conventional technologies for thyroid nodule detection mainly include ultrasound imaging technology, fine needle aspiration cytology (FNA), thyroid function tests, radionuclide scanning, and genetic testing. However, these conventional technologies still face many technical bottlenecks and difficulties in practical applications, specifically:

[0003] Ultrasound examination can provide information on the size, shape, echo characteristics and presence of calcification of nodules. However, the results of ultrasound examinations depend on the experience of the operator and there is a problem of insufficient consistency in operation. In addition, for tiny nodules, the resolution of existing ultrasound equipment may be insufficient, which may easily lead to missed diagnosis or misdiagnosis, and its imaging characteristics are difficult to directly distinguish between benign and malignant nodules, and other examination methods are needed to assist in diagnosis;

[0004] The distinction between "hot nodules" and "cold nodules" is based on the absorption characteristics of radioactive iodine or technetium. However, the vast majority of malignant nodules are "cold nodules", but most cold nodules are benign, resulting in low specificity of this method. In addition, patients need to be exposed to a certain dose of radioactive substances during the examination, which increases potential health risks. Summary of the invention

[0005] In view of the above-mentioned problems, the present invention is proposed.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: a method for constructing a thyroid nodule detection model, comprising the following steps:

[0007] Multimodal imaging of thyroid nodules is performed using image enhancement technology and photoacoustic imaging technology, specifically:

[0008] For the collected thyroid ultrasound image data, convolutional neural network technology is used to enhance the image to enhance the detail features of the thyroid nodules, and an enhancement verification function is introduced to verify whether the image enhancement is completed. At the same time, pulsed laser is used to irradiate the thyroid tissue, and the generated ultrasonic signal is collected and back-projected to realize the collection of thyroid nodule three-dimensional volume data. The enhanced image data and three-dimensional volume data are fused to form multi-modal joint features.

[0009] A thyroid nodule detection model is built based on the constructed multimodal image data, specifically:

[0010] Each thyroid nodule is taken as a node in the graph data structure, the constructed multimodal image data is used as the node feature, and connections are established based on the feature similarity between nodes. At the same time, an attention mechanism is introduced to update the node features in real time, and classification is performed based on the updated node features to complete the detection of thyroid nodules.

[0011] And, the loss function is used to optimize the thyroid nodule detection model, specifically:

[0012] The model is optimized by constructing a loss function. The constructed loss function is minimized through the weights and bias terms of the thyroid nodule detection model, so that the output results of the thyroid nodule detection model are consistent with the actual results of historical data, thereby optimizing the constructed thyroid nodule detection model.

[0013] As a preferred solution of the method for constructing a thyroid nodule detection model described in the present invention, the enhancement of the detail features of the thyroid nodules is specifically as follows:

[0014] The acquired ultrasound image of the thyroid nodule is recorded as a two-dimensional matrix I(x,y), where (x,y) represents the spatial coordinates of the pixel;

[0015] The convolutional neural network is used to enhance the image of the two-dimensional matrix, specifically:

[0016] By extracting the resolution features in the two-dimensional matrix through the convolution layer, we have:

[0017] F 1 (x,y)=ReLu(W 1 *I(x,y)+b 1 )

[0018] Among them, ReLu represents the activation function of the convolutional layer, W 1 , b 1 Represent the weight and bias of the convolutional layer respectively, I(x,y) represents the two-dimensional matrix of the input ultrasound image, F 1 (x,y) represents the extracted resolution feature;

[0019] Performing high-dimensional mapping on the extracted resolution features, we have:

[0020] F 2 (x,y)=ReLu(W 2 *F 1 (x,y)+b 2 )

[0021] Among them, ReLu represents the activation function of the convolutional layer, W 2 , b2 Represent the weight and bias of the mapping layer, F 1 (x, y) represents the extracted resolution feature, F 2 (x, y) represents the resolution feature after high-dimensional mapping;

[0022] Based on the mapped resolution features, the resolution enhancement of the ultrasound image is completed, then,

[0023] I′(x,y)=W 3 *F 2 (x,y)+b 3

[0024] Among them, W 3 , b 3 Represent the weight and bias of the enhancement layer, F 2 (x, y) represents the resolution feature after high-dimensional mapping, and I′(x, y) represents the two-dimensional matrix of the enhanced ultrasound image.

[0025] As a preferred solution of the method for constructing a thyroid nodule detection model described in the present invention, the verification of whether the image is enhanced is as follows:

[0026] Set the enhancement function f(I);

[0027] The enhanced image data is divided into multiple small blocks, and the size of each small block is equal, I′(x,y)={I 1 ′(x,y),I 2 ′(x,y),...,I n ′(x,y)}, where I 1 ′(x,y) represents the first small square divided by the enhanced image data;

[0028] The unenhanced image data is divided into the same number of small blocks, and the size of each small block is equal, I(x,y) = {I 1 (x,y),I 2 (x,y),...,I n (x,y)}, where I 1 (x, y) represents the first small square divided by the unenhanced image data;

[0029] Using the enhancement function to calculate the sum of the normal differences of each small block in the image data, we have:

[0030]

[0031] Where I′(x, y) represents the two-dimensional matrix of the enhanced ultrasound image, I(x, y) represents the unenhanced image data, and I i′(x,y) represents the i-th small block of the enhanced image data. i (x, y) represents the i-th small block of the unenhanced image data, i represents the index of the small block, n represents the total number of small blocks, f(I) represents the output result of the enhancement function, and the image data is judged whether the enhancement is completed according to the output result of the function, then,

[0032] When the output of the function satisfies the formula f(I)≥(I′(x,y)-I(x,y)) 2 When n, it means that the current image data is fully enhanced;

[0033] When the output of the function satisfies the formula f(I)<(I′(x,y)-I(x,y)) 2 When n is reached, it means that the current image data is not fully enhanced. The weights and bias items of each layer of the convolutional neural network are adjusted to re-enhance the image data until the formula f(I)≥(I′(x,y)-I(x,y)) is satisfied. 2 ·n.

[0034] As a preferred embodiment of the method for constructing a thyroid nodule detection model described in the present invention, the collection of the thyroid nodule three-dimensional volume data is specifically as follows:

[0035] A three-dimensional data matrix P(x, y, t) is formed based on the photoacoustic signal, where x and y represent the horizontal and vertical spatial positions, and t represents the time dimension;

[0036] Through the propagation speed of the photoacoustic wave, the time dimension is mapped to the depth z=c·t in the z-axis direction, where c represents the propagation speed of the photoacoustic wave;

[0037] Mapping the converted photoacoustic signal to three-dimensional space, we have P(x,y,z)=P(x,y,t·c);

[0038] Back-project the three-dimensional spatial data of all detectors to complete the collection of three-dimensional volume data, then we have:

[0039] V p (x,y,z)=∫P(θ,t)·R(θ,t)dθ

[0040] Where θ represents the angle distribution of the detector, is the angle at which the pulsed laser irradiates the thyroid tissue, R(θ,t) represents the back-projection kernel function, P(θ,t) represents the three-dimensional spatial data obtained when the detector irradiates the thyroid at different angles θ, and V p (x, y, z) represents the formed three-dimensional volume data.

[0041] As a preferred solution of the method for constructing a thyroid nodule detection model described in the present invention, the establishment of a connection based on the feature similarity between nodes is specifically as follows:

[0042] Treat each thyroid nodule as a node v in the graph data structure i , and the multimodal image data corresponding to the current thyroid nodule is used as the node feature vector F i ;

[0043] The weight of the edge is determined according to the similarity between node samples, then,

[0044]

[0045] Among them, F i 、F j Respectively represent the node v i and node v j The node features, σ represents the Gaussian kernel width, w ij Represents node v i , and node v j The weight of the edge between .

[0046] As a preferred solution of the method for constructing a thyroid nodule detection model described in the present invention, the real-time update node features are specifically as follows:

[0047] For each node's feature vector F i Performing linear transformation, we have:

[0048] F i ′=WF i

[0049] Among them, F i Represents the feature vector of the node, F i ′ represents the node feature vector after linear transformation, W represents the feature transformation matrix, which is used to control the linear transformation of the feature vector;

[0050] Connect the transformed node feature vectors, and we have:

[0051] H ij =[F i ′||F′ j ]

[0052] Among them, || represents the feature connection operation, and F i ′ and F′ j Spliced ​​into a vector, H ij represents the concatenated feature vector;

[0053] By calculating the attention score of the connected feature vector through a single-layer neural network, we have:

[0054] e ij =ReLu(a T H ij )

[0055] Among them, a represents the attention parameter vector, T represents the transposed matrix, and e ij represents the calculated attention score, which indicates the feature correlation between nodes;

[0056] The dynamic attention weight is set according to the calculated feature correlation, then we have

[0057]

[0058] Among them, a ij Represents the dynamic attention weight set for node v i Affected by neighbor node v j The degree of influence of node i, N(i) represents the set of all neighbor nodes of node i, k represents the sequence number of the neighbor node, e ik Represents node v i With node v k The correlation between the features;

[0059] Aggregate the features of neighboring nodes according to the attention weights to update the node v i The characteristics of

[0060] F i ″=sigmoid(∑ j∈N(i) a ij F j ′)

[0061] Among them, a ij Represents the dynamic attention weight set for node v i Affected by neighbor node v j The degree of influence, sigmoid represents the activation function, N(i) represents the set of all neighbor nodes of node i, F j ′ represents node v j The linear transformation result, F i ″ represents node v i Updated feature vector.

[0062] As a preferred embodiment of the method for constructing a thyroid nodule detection model described in the present invention, the detection of thyroid nodules is specifically as follows:

[0063] For node v i The updated feature vector embeds the global features through multi-layer stacking, then we have:

[0064]

[0065] in, represents the dynamic attention weight of the lth layer, sigmoid represents the activation function, W l represents the feature transformation matrix of the lth layer, Represents node v j In the feature of the l-1 layer, F i l Represents node v i Features after updating at layer l;

[0066] After multiple layers of stacking, the final stacking result is embedded with global features, and then,

[0067]

[0068] Among them, l represents the sequence number of the layer, L represents the upper limit of the layer, and g i Represents node v i The graph structure consists of nodes and edges corresponding to the nodes. i L Represents node v i The embedded global features are used to detect thyroid nodules. The specific detection is as follows:

[0069] By setting the decision function for thyroid nodule detection, we have:

[0070]

[0071] Among them, w and b represent the weight and bias of the decision function respectively, and F i L Represents node v i The global feature of the embedding, p i Represents node v i The probability that the corresponding thyroid nodule is a malignant nodule ranges from 0 to 1 and is used to detect thyroid nodules. Then,

[0072] If the calculated result satisfies the formula p i ≥0.5, indicating that the thyroid nodule corresponding to the current node is a malignant nodule;

[0073] If the calculated result satisfies the formula p i <0.5, indicating that the thyroid nodule corresponding to the current node is a benign nodule.

[0074] As a preferred solution of the method for constructing a thyroid nodule detection model described in the present invention, the optimization of the thyroid nodule detection model using a loss function is specifically as follows:

[0075] Construct the loss function Loss, then we have,

[0076]

[0077] Among them, N represents the total number of samples involved in model training and optimization, y i represents the output result of the historical data of thyroid nodules in the model, p i Represents the output result of the embedded features in the model;

[0078] By adjusting y i and p i The value results of are used to optimize the thyroid nodule detection model, specifically:

[0079] For y in the loss function i log(p i ), by adjusting the weights and biases in the model, we can achieve y i and p i If we adjust both at the same time, we have

[0080] If the output result of the historical data of thyroid nodules in the model satisfies the formula y i =1, indicating that the current thyroid nodule is a malignant nodule. By adjusting the weights and bias terms in the model, the output of the model satisfies p i =1, to achieve y i log(p i ) to achieve model optimization;

[0081] For the loss function (1-y i )log(1-p i ), by adjusting the weights and biases in the model, we can achieve y i and p i If we adjust both at the same time, we have

[0082] If the output result of the historical data of thyroid nodules in the model satisfies the formula y i =0, indicating that the current thyroid nodule is a benign nodule. By adjusting the weights and bias terms in the model, the output of the model satisfies 1-p i =0, to achieve (1-y i )log(1-p i ) to achieve model optimization.

[0083] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of a method for constructing a thyroid nodule detection model when executing the computer program.

[0084] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for constructing a thyroid nodule detection model.

[0085] Beneficial effects of the present invention:

[0086] The present invention integrates multiple modal data such as ultrasound images, photoacoustic imaging, and genetic data, making full use of the advantages of each data, overcoming the limitations that may exist in single modal data (such as the resolution problem of ultrasound images, the locality of genetic data, etc.), and improving the comprehensiveness and accuracy of detection;

[0087] Use graph neural networks to gradually extract high-order relationship features between nodule samples and generate global embedding features that can capture complex nonlinear relationships between nodes (nodule samples);

[0088] Through the attention mechanism, important relationships between nodule samples can be automatically identified and weighted, reducing feature redundancy and improving the robustness and accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0090] Figure 1 The figure is a schematic diagram of the overall method steps of a method for building a thyroid nodule detection model of the present invention. DETAILED DESCRIPTION

[0091] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0092] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0093] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0094] The present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0095] Meanwhile, in the description of the present invention, it should be noted that the terms "first, second or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0096] In the present invention, unless otherwise clearly specified and limited, the terms "install, connect, connect" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0097] Example 1

[0098] Reference Figure 1 , as an embodiment of the present invention, provides a method for constructing a thyroid nodule detection model, comprising the following steps:

[0099] S1: Multimodal imaging of thyroid nodules using image enhancement technology and photoacoustic imaging.

[0100] Specifically, the multimodal imaging of thyroid nodules using image enhancement technology and photoacoustic imaging technology uses ultrasound images to capture the morphology and physical characteristics of thyroid nodules, and uses photoacoustic imaging technology to capture biological information. The results captured by the two imaging technologies are processed through feature extraction and fusion to form a multimodal feature representation, thereby improving the dimension of thyroid nodule detection and diagnosis.

[0101] Furthermore, the image enhancement technology is to use a convolutional neural network to enhance the ultrasound image of the thyroid nodule to enhance the detail features of the thyroid nodule. The specific implementation is as follows:

[0102] The acquired ultrasound image of the thyroid nodule is recorded as a two-dimensional matrix I(x,y), where (x,y) represents the spatial coordinates of the pixel;

[0103] The convolutional neural network is used to enhance the image of the two-dimensional matrix, specifically:

[0104] By extracting the resolution features in the two-dimensional matrix through the convolution layer, we have:

[0105] F 1 (x,y)=ReLu(W 1 *I(x,y)+b 1 )

[0106] Among them, ReLu represents the activation function of the convolutional layer, W 1 , b 1 Represent the weight and bias of the convolutional layer respectively, I(x,y) represents the two-dimensional matrix of the input ultrasound image, F 1 (x,y) represents the extracted resolution feature;

[0107] Performing high-dimensional mapping on the extracted resolution features, we have:

[0108] F 2 (x,y)=ReLu(W 2 *F 1 (x,y)+b 2 )

[0109] Among them, ReLu represents the activation function of the convolutional layer, W 2 , b 2 Represent the weight and bias of the mapping layer, F 1 (x, y) represents the extracted resolution feature, F 2 (x, y) represents the resolution feature after high-dimensional mapping;

[0110] Based on the mapped resolution features, the resolution enhancement of the ultrasound image is completed, then,

[0111] I′(x,y)=W 3 *F 2 (x,y)+b 3

[0112] Among them, W 3 , b 3 Represent the weight and bias of the enhancement layer, F 2 (x, y) represents the resolution feature after high-dimensional mapping, and I′(x, y) represents the two-dimensional matrix of the enhanced ultrasound image.

[0113] It should be noted that in order to ensure the comprehensive enhancement of the ultrasound image, an enhancement verification function is introduced. According to the output result of the function, it is verified whether the image is enhanced. The specific verification is as follows:

[0114] Set the enhancement function f(I);

[0115] The enhanced image data is divided into multiple small blocks, and the size of each small block is equal, I′(x,y)={I 1 ′(x,y),I 2 ′(x,y),...,I n ′(x,y)}, where I 1 ′(x,y) represents the first small square divided by the enhanced image data;

[0116] The unenhanced image data is divided into the same number of small blocks, and the size of each small block is equal, I(x,y) = {I 1 (x,y),I 2 (x,y),...,I n (x,y)}, where I 1 (x, y) represents the first small square divided by the unenhanced image data;

[0117] Using the enhancement function to calculate the sum of the normal differences of each small block in the image data, we have:

[0118]

[0119] Where I′(x, y) represents the two-dimensional matrix of the enhanced ultrasound image, I(x, y) represents the unenhanced image data, and I i ′(x,y) represents the i-th small block of the enhanced image data. i (x, y) represents the i-th small block of the unenhanced image data, i represents the index of the small block, n represents the total number of small blocks, f(I) represents the output result of the enhancement function, and the image data is judged whether the enhancement is completed according to the output result of the function, then,

[0120] When the output of the function satisfies the formula f(I)≥(I′(x,y)-I(x,y)) 2 When n, it means that the current image data is fully enhanced;

[0121] When the output of the function satisfies the formula f(I)<(I′(x,y)-I(x,y)) 2 When n is reached, it means that the current image data is not fully enhanced. The weights and bias items of each layer of the convolutional neural network are adjusted to re-enhance the image data until the formula f(I)≥(I′(x,y)-I(x,y)) is satisfied. 2 ·n.

[0122] Furthermore, the photoacoustic imaging technology is to irradiate the thyroid tissue with a pulsed laser, and the laser energy is absorbed in the thyroid tissue and converted into heat energy, which increases the local temperature, causes the thyroid tissue to expand and compress, and generates ultrasonic signals. The signals generated by all detectors are back-projected to complete the collection of three-dimensional volume data. The specific implementation is as follows:

[0123] A three-dimensional data matrix P(x, y, t) is formed based on the photoacoustic signal, where x and y represent the horizontal and vertical spatial positions, and t represents the time dimension;

[0124] Through the propagation speed of the photoacoustic wave, the time dimension is mapped to the depth z=c·t in the z-axis direction, where c represents the propagation speed of the photoacoustic wave;

[0125] Mapping the converted photoacoustic signal to three-dimensional space, we have P(x,y,z)=P(x,y,t·c);

[0126] Back-project the three-dimensional spatial data of all detectors to complete the collection of three-dimensional volume data, then we have:

[0127] V p (x,y,z)=∫P(θ,t)·R(θ,t)dθ

[0128] Where θ represents the angle distribution of the detector, is the angle at which the pulsed laser irradiates the thyroid tissue, R(θ,t) represents the back-projection kernel function, P(θ,t) represents the three-dimensional spatial data obtained when the detector irradiates the thyroid at different angles θ, and V p (x, y, z) represents the formed three-dimensional volume data.

[0129] Furthermore, the multimodal imaging is to perform multimodal fusion of the enhanced image data and the three-dimensional volume data, as follows:

[0130] The enhanced image data is dimensionally aligned with the three-dimensional volume data through the spatial transformation matrix, then,

[0131] I′(x,y,z)=T·V p (x,y,z)

[0132] Where T represents the spatial transformation matrix, which is used to transform two-dimensional image data into three-dimensional data, V p (x, y, z) represents the formed three-dimensional volume data, and I′(x, y, z) represents the converted three-dimensional image data;

[0133] And perform feature extraction on the 3D volume data and 3D image data, then we have:

[0134] The features F of the three-dimensional volume data are extracted by 3D convolutional neural network. pAnd the feature F of the 3D image data I ;

[0135] And the extracted features are fused to form joint features, then we have,

[0136] F=α·F p +β·F I

[0137] Among them, F p represents the features of the extracted 3D volume data, F I It represents the features of the extracted three-dimensional image data, α and β represent weight factors, which are set by the implementers according to the actual application scenario, and F represents the fused joint feature vector, which is multimodal image data.

[0138] S2: Build a thyroid nodule detection model based on the constructed multimodal image data.

[0139] The thyroid nodule model is constructed based on the graph attention network mechanism. Each thyroid nodule is taken as a node in the graph data structure, and the constructed multimodal image data is taken as the feature of the node. The connection is established based on the feature similarity between the nodes. At the same time, the attention mechanism is introduced to update the node features in real time. The node features are classified according to the updated node features to complete the detection of thyroid nodules. The specific implementation is as follows:

[0140] Treat each thyroid nodule as a node v in the graph data structure i , and the multimodal image data corresponding to the current thyroid nodule is used as the node feature vector F i ;

[0141] The weight of the edge is determined according to the similarity between node samples, then,

[0142]

[0143] Among them, F i 、F j Respectively represent the node v i and node v j The node characteristics of σ represent the Gaussian kernel width, which is set by the implementer according to the actual application scenario. ij Represents node v i , and node v j The weight of the edge between .

[0144] Furthermore, by introducing the attention mechanism, each white edge in the graph data structure is assigned a corresponding dynamic weight a ij , and update the node feature vector according to the dynamic weight. The specific implementation is as follows:

[0145] For each node's feature vector F i Performing linear transformation, we have:

[0146] F i ′=WF i

[0147] Among them, F i Represents the feature vector of the node, F i ′ represents the node feature vector after linear transformation, W represents the feature transformation matrix, which is used to control the linear transformation of the feature vector;

[0148] Connect the transformed node feature vectors, and we have:

[0149] H ij =[F i ′||F′ j ]

[0150] Among them, || represents the feature connection operation, and F i ′ and F′ j Spliced ​​into a vector, H ij represents the concatenated feature vector;

[0151] By calculating the attention score of the connected feature vector through a single-layer neural network, we have:

[0152] e ij =Re Lu(a T H ij )

[0153] Among them, a represents the attention parameter vector, which is used to calculate the interaction between features. The specific value is set by the implementer according to the actual application scenario. T represents the transposed matrix, e ij represents the calculated attention score, which indicates the feature correlation between nodes;

[0154] The dynamic attention weight is set according to the calculated feature correlation, then we have

[0155]

[0156] Among them, a ij Represents the dynamic attention weight set for node v i Affected by neighbor node v j The degree of influence of node i, N(i) represents the set of all neighbor nodes of node i, k represents the sequence number of the neighbor node, e ik Represents node v i With node v k The correlation between the features;

[0157] Aggregate the features of neighboring nodes according to the attention weights to update the node v i The characteristics of

[0158] F i ″=sigmoid(∑ j∈N(i) a ij F j ′)

[0159] Among them, a ij Represents the dynamic attention weight set for node v i Affected by neighbor node v j The degree of influence, sigmoid represents the activation function, N(i) represents the set of all neighbor nodes of node i, F j ′ represents node v j The linear transformation result, F i ″ represents node v i Updated feature vector.

[0160] Furthermore, the detection of thyroid nodules is based on the updated feature vectors. Through the multi-layer stacking of the multi-layer graph attention network, the feature relationship between high-order neighbors is gradually captured, thereby extracting the global graph structure features. The detection of thyroid nodules is realized according to the extracted global graph structure features. The specific implementation is as follows:

[0161] For node v i The updated feature vector embeds the global features through multi-layer stacking, then we have:

[0162]

[0163] in, represents the dynamic attention weight of the lth layer, sigmoid represents the activation function, W l represents the feature transformation matrix of the lth layer, Represents node v j In the feature of the l-1 layer, F i l Represents node v i Features after updating at layer l;

[0164] After multiple layers of stacking, the final stacking result is embedded with global features, and then,

[0165]

[0166] Among them, l represents the sequence number of the layer, L represents the upper limit of the layer, and g i Represents node v i The graph structure consists of nodes and edges corresponding to the nodes. iL Represents node v i The embedded global features are used to detect thyroid nodules. The specific detection is as follows:

[0167] By setting the decision function for thyroid nodule detection, we have:

[0168]

[0169] Among them, w and b represent the weight and bias of the decision function respectively. The specific values ​​are set by the implementer according to the actual application scenario. i L Represents node v i The global feature of the embedding, p i Represents node v i The probability that the corresponding thyroid nodule is a malignant nodule ranges from 0 to 1 and is used to detect thyroid nodules. Then,

[0170] If the calculated result satisfies the formula p i ≥0.5, indicating that the thyroid nodule corresponding to the current node is a malignant nodule;

[0171] If the calculated result satisfies the formula p i <0.5, indicating that the thyroid nodule corresponding to the current node is a benign nodule.

[0172] S3: Use the loss function to optimize the thyroid nodule detection model.

[0173] Specifically, the optimization of the thyroid nodule detection model is to optimize the model by constructing a loss function, and the specific optimization is as follows:

[0174] The classification result p for each node i , using the historical data of thyroid nodules to train and optimize the model, specifically:

[0175] Construct the loss function Loss, then we have,

[0176]

[0177] Among them, N represents the total number of samples involved in model training and optimization, y i represents the output result of the historical data of thyroid nodules in the model, p i Represents the output result of the embedded features in the model;

[0178] By adjusting y i and p i The value results of are used to optimize the thyroid nodule detection model, specifically:

[0179] For y in the loss functioni log(p i ), by adjusting the weights and biases in the model, we can achieve y i and p i If we adjust both at the same time, we have

[0180] If the output result of the historical data of thyroid nodules in the model satisfies the formula y i =1, indicating that the current thyroid nodule is a malignant nodule. By adjusting the weights and bias terms in the model, the output of the model satisfies p i =1, to achieve y i log(p i ) is maximized, thereby reducing model loss and achieving model optimization;

[0181] For the loss function (1-y i )log(1-p i ), by adjusting the weights and biases in the model, we can achieve y i and p i If we adjust both at the same time, we have

[0182] If the output result of the historical data of thyroid nodules in the model satisfies the formula y i =0, indicating that the current thyroid nodule is a benign nodule. By adjusting the weights and bias terms in the model, the output of the model satisfies 1-p i =0, to achieve (1-y i )log(1-p i ) is maximized, thereby reducing model loss and achieving model optimization;

[0183] Through y i and p i , and adjust them at the same time to achieve the minimization of the loss function, so that the output results of the thyroid nodule detection model are consistent with the actual results of the historical data, indicating that the optimization of the constructed thyroid nodule detection model is completed.

[0184] Furthermore, if the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer,

[0185] The server, or network device, etc.) performs all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0186] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0187] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0188] Additionally, in order to provide a concise description of exemplary embodiments, all features of an actual embodiment (ie, those features that are not relevant to the best mode presently contemplated for carrying out the invention or those that are not relevant to implementing the invention) may not be described.

[0189] It will be appreciated that in the development of any actual implementation, as in any engineering or design project, numerous implementation-specific decisions may be made. Such a development effort may be complex and time-consuming, but will be a routine task of design, fabrication, and production for those of ordinary skill having the benefit of this disclosure without undue experimentation.

[0190] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for constructing a thyroid nodule detection model, characterized in that: The following steps are included: Multimodal imaging of thyroid nodules is performed using image enhancement technology and photoacoustic imaging technology, specifically: For the collected thyroid ultrasound image data, convolutional neural network technology is used to enhance the image to enhance the detail features of the thyroid nodules, and an enhancement verification function is introduced to verify whether the image enhancement is completed. At the same time, pulsed laser is used to irradiate the thyroid tissue, and the generated ultrasonic signal is collected and back-projected to realize the collection of thyroid nodule three-dimensional volume data. The enhanced image data and three-dimensional volume data are fused to form multi-modal joint features. A thyroid nodule detection model is built based on the constructed multimodal image data, specifically: Each thyroid nodule is taken as a node in the graph data structure, the constructed multimodal image data is used as the node feature, and connections are established based on the feature similarity between nodes. At the same time, an attention mechanism is introduced to update the node features in real time, and classification is performed based on the updated node features to complete the detection of thyroid nodules. And, the loss function is used to optimize the thyroid nodule detection model, specifically: The model is optimized by constructing a loss function. The constructed loss function is minimized through the weights and bias terms of the thyroid nodule detection model, so that the output results of the thyroid nodule detection model are consistent with the actual results of historical data, thereby optimizing the constructed thyroid nodule detection model.

2. A method for constructing a thyroid nodule detection model according to claim 1, characterized in that: The enhancement of the detailed features of the thyroid nodules is specifically as follows: The acquired ultrasound image of the thyroid nodule is recorded as a two-dimensional matrix I(x,y), where (x,y) represents the spatial coordinates of the pixel; The convolutional neural network is used to enhance the image of the two-dimensional matrix, specifically: By extracting the resolution features in the two-dimensional matrix through the convolution layer, we have: F1(x,y)=ReLu(W1*I(x,y)+b1) Where ReLu represents the activation function of the convolutional layer, W1 and b1 represent the weight and bias term of the convolutional layer respectively, I(x, y) represents the two-dimensional matrix of the input ultrasound image, and F1(x, y) represents the extracted resolution feature; Performing high-dimensional mapping on the extracted resolution features, we have: F2(x,y)=ReLu(W2*F1(x,y)+b2) Among them, ReLu represents the activation function of the convolutional layer, W2 and b2 represent the weight and bias term of the mapping layer respectively, F1(x, y) represents the extracted resolution feature, and F2(x, y) represents the resolution feature after high-dimensional mapping; Based on the mapped resolution features, the resolution enhancement of the ultrasound image is completed, then, I′(x,y)=W3*F2(x,y)+b3 Among them, W3 and b3 represent the weight and bias term of the enhancement layer respectively, F2(x, y) represents the resolution feature after high-dimensional mapping, and I′(x, y) represents the two-dimensional matrix of the enhanced ultrasound image.

3. A method for constructing a thyroid nodule detection model according to claim 2, characterized in that: The verification of whether the image enhancement is completed is as follows: Set the enhancement function f(I); The enhanced image data is divided into multiple small blocks, and the size of each small block is equal, I′(x,y)={I1′(x,y),I2′(x,y),...,I n ′(x,y)}, where I1′(x,y) represents the first small block of the enhanced image data division; The unenhanced image data is divided into the same number of small blocks, and the size of each small block is equal, I(x,y) = {I1(x,y),I2(x,y),...,I n (x, y)}, where I1(x, y) represents the first small block divided by the unenhanced image data; Using the enhancement function to calculate the sum of the normal differences of each small block in the image data, we have: Where I′(x, y) represents the two-dimensional matrix of the enhanced ultrasound image, I(x, y) represents the unenhanced image data, and I i ′(x,y) represents the i-th small block of the enhanced image data. i (x, y) represents the i-th small block of the unenhanced image data, i represents the index of the small block, n represents the total number of small blocks, f(I) represents the output result of the enhancement function, and the image data is judged whether the enhancement is completed according to the output result of the function, then, When the output of the function satisfies the formula f(I)≥(I′(x,y)-I(x,y)) 2 When n, it means that the current image data is fully enhanced; When the output of the function satisfies the formula f(I)<(I′(x,y)-I(x,y)) 2 When n is reached, it means that the current image data is not fully enhanced. The weights and bias items of each layer of the convolutional neural network are adjusted to re-enhance the image data until the formula f(I)≥(I′(x,y)-I(x,y)) is satisfied. 2 · Until n.

4. A method for constructing a thyroid nodule detection model according to claim 3, characterized in that: The collection of the three-dimensional volume data of the thyroid nodule is specifically as follows: A three-dimensional data matrix P(x, y, t) is formed based on the photoacoustic signal, where x and y represent the horizontal and vertical spatial positions, and t represents the time dimension; Through the propagation speed of the photoacoustic wave, the time dimension is mapped to the depth z=c·t in the z-axis direction, where c represents the propagation speed of the photoacoustic wave; Mapping the converted photoacoustic signal to three-dimensional space, we have P(x,y,z)=P(x,y,t·c); Back-project the three-dimensional spatial data of all detectors to complete the collection of three-dimensional volume data, then we have: V p (x,y,z)=∫P(θ,t)·R(θ,t)dθ Where θ represents the angle distribution of the detector, is the angle at which the pulsed laser irradiates the thyroid tissue, R(θ,t) represents the back-projection kernel function, P(θ,t) represents the three-dimensional spatial data obtained when the detector irradiates the thyroid at different angles θ, and V p (x, y, z) represents the formed three-dimensional volume data.

5. A method for constructing a thyroid nodule detection model according to claim 4, characterized in that: The specific method of establishing a connection based on feature similarity between nodes is as follows: Treat each thyroid nodule as a node v in the graph data structure i , and the multimodal image data corresponding to the current thyroid nodule is used as the node feature vector F i ; The weight of the edge is determined according to the similarity between node samples, then, Among them, F i 、F j Respectively represent the node v i and node v j The node features, σ represents the Gaussian kernel width, w ij Represents node v i , and node v j The weight of the edge between .

6. A method for constructing a thyroid nodule detection model according to claim 5, characterized in that: The real-time update node features are as follows: For each node's feature vector F i Performing linear transformation, we have: F i ′=WF i Among them, F i Represents the feature vector of the node, F i ′ represents the node feature vector after linear transformation, W represents the feature transformation matrix, which is used to control the linear transformation of the feature vector; Connect the transformed node feature vectors, and we have: H ij =[F i ′||F’ j ] Among them, || represents the feature connection operation, and F i ′ and F' j Spliced ​​into a vector, H ij represents the concatenated feature vector; By calculating the attention score of the connected feature vector through a single-layer neural network, we have: have been ij =ReLu(a T A ij ) Among them, a represents the attention parameter vector, T represents the transposed matrix, and e ij represents the calculated attention score, which indicates the feature correlation between nodes; The dynamic attention weight is set according to the calculated feature correlation, then we have Among them, a ij Represents the dynamic attention weight set for node v i Affected by neighbor node v j The degree of influence of node i, N(i) represents the set of all neighbor nodes of node i, k represents the sequence number of the neighbor node, e ik Represents node v i With node v k The correlation between the features; Aggregate the features of neighboring nodes according to the attention weights to update the node v i The characteristics of F i ″=sigmoid(∑ j∈N(i) a ij F j ′) Among them, a ij Represents the dynamic attention weight set for node v i Affected by neighbor node v j The degree of influence, sigmoid represents the activation function, N(i) represents the set of all neighbor nodes of node i, F j ′ represents node v j The linear transformation result, F i ″ represents node v i Updated feature vector.

7. A method for constructing a thyroid nodule detection model according to claim 6, characterized in that: The detection of the thyroid nodules is specifically as follows: For node v i The updated feature vector embeds the global features through multi-layer stacking, then we have: in, represents the dynamic attention weight of the lth layer, sigmoid represents the activation function, W l represents the feature transformation matrix of the lth layer, Represents node v j In the feature of the l-1 layer, F i l Represents node v i Features after updating at layer l; After multiple layers of stacking, the final stacking result is embedded with global features, and then, Among them, l represents the sequence number of the layer, L represents the upper limit of the layer, and g i Represents node v i The graph structure consists of nodes and edges corresponding to the nodes. i L Represents node v i The embedded global features are used to detect thyroid nodules. The specific detection is as follows: By setting the decision function for thyroid nodule detection, we have: Among them, w and b represent the weight and bias of the decision function respectively, and F i L Represents node v i The global feature of the embedding, p i Represents node v i The probability that the corresponding thyroid nodule is a malignant nodule ranges from 0 to 1 and is used to detect thyroid nodules. Then, If the calculated result satisfies the formula p i ≥0.5, indicating that the thyroid nodule corresponding to the current node is a malignant nodule; If the calculated result satisfies the formula p i <0.5, indicating that the thyroid nodule corresponding to the current node is a benign nodule.

8. A method for constructing a thyroid nodule detection model according to claim 7, characterized in that: The optimization of the thyroid nodule detection model using the loss function is specifically as follows: Construct the loss function Loss, then we have, Among them, N represents the total number of samples involved in model training and optimization, y i represents the output result of the historical data of thyroid nodules in the model, p i Represents the output result of the embedded features in the model; By adjusting y i and p i The value results of are used to optimize the thyroid nodule detection model, specifically: For y in the loss function i log(p i ), by adjusting the weights and biases in the model, we can achieve y i and p i If we adjust both at the same time, we have If the output result of the historical data of thyroid nodules in the model satisfies the formula y i =1, indicating that the current thyroid nodule is a malignant nodule. By adjusting the weights and bias terms in the model, the output of the model satisfies p i =1, to achieve y i log(p i ) to achieve model optimization; For the loss function (1-y i )log(1-p i ), by adjusting the weights and biases in the model, we can achieve y i and p i If we adjust both at the same time, we have If the output result of the historical data of thyroid nodules in the model satisfies the formula y i =0, indicating that the current thyroid nodule is a benign nodule. By adjusting the weights and bias terms in the model, the output of the model satisfies 1-p i =0, to achieve (1-y i )log(1-p i ) to achieve model optimization.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images