Intelligent auxiliary detection system and method for CT image fracture based on channel cross attention mechanism

Abstract

The application provides a CT image fracture intelligent auxiliary detection system based on a channel cross attention mechanism, comprising an image acquisition module configured to acquire CT image data; an image preprocessing module connected with the image acquisition module and configured to perform preprocessing on the CT image to obtain a CT image data set; a feature extraction module connected with the image preprocessing module and configured to store a neural network model, acquire the CT image data set, and extract features from different channels after feature extraction; and a fracture detection module configured to determine whether a fracture exists and the location, type and severity of the fracture according to preset parameters based on the features determined by the neural network model. The application can greatly improve the detection efficiency and accuracy and reduce human errors by automatically analyzing the CT image, thereby providing more timely and reliable diagnosis services for patients.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of medical image recognition, and particularly relates to a CT image fracture auxiliary detection system and method based on a channel cross attention mechanism. BACKGROUND

[0002] In the medical field, accurate and rapid detection of fractures is crucial for patient diagnosis and treatment. Traditional fracture detection methods mainly rely on manual interpretation of CT images by doctors. However, this approach has many limitations.

[0003] On the one hand, manual interpretation of CT images is very time-consuming and labor-intensive. Doctors need to view a large number of CT image slices one by one, carefully analyzing every detail. This not only requires a high level of professional knowledge and experience from doctors, but also consumes a lot of time, especially when facing a large number of patients, which may lead to low diagnostic efficiency and prolonged patient waiting time.

[0004] On the other hand, the accuracy of manual detection is easily affected by subjective factors of doctors. Different doctors' experience levels, fatigue levels, and other factors can lead to differences in diagnostic results. Even experienced doctors may miss or misdiagnose after a long period of work.

[0005] With the continuous development of medical imaging technology, the resolution of CT images is becoming higher and higher, and the amount of image data is also increasing rapidly. This further increases the difficulty of manual analysis by doctors. At the same time, modern medicine has higher requirements for the accuracy and timeliness of fracture detection.

[0006] In order to solve these problems, there is an urgent need for an intelligent fracture detection system and method. Using advanced computer technology and artificial intelligence algorithms, automatic analysis of CT images can greatly improve detection efficiency and accuracy, reduce human error, and provide more timely and reliable diagnostic services for patients.

[0007] In view of this, there is an urgent need for a CT image fracture intelligent auxiliary detection system and method based on a channel cross attention mechanism. SUMMARY

[0008] To this end, the present application provides a CT image fracture intelligent auxiliary detection system and method based on a channel cross attention mechanism, which realizes automatic analysis of CT images.

[0009] The first aspect of the present application provides a CT image fracture intelligent auxiliary detection system based on a channel cross attention mechanism, comprising:

[0010] An image acquisition module configured to acquire CT image data from at least one database or real-time image terminal;

[0011] An image preprocessing module is connected with the image acquisition module and configured to perform preprocessing of the CT image data to obtain a CT image data set;

[0012] A feature extraction module is connected with the image preprocessing module and configured to store a neural network model to obtain a plurality of groups of channel features of each CT image in the CT image data set;

[0013] A fracture detection module is connected with the feature extraction module and configured to store a sample data set and obtain a plurality of groups of channel features extracted by the neural network model to perform a judgment of a fracture,

[0014] When performing the judgment of the fracture, the feature extraction module and the fracture detection module are configured to perform the following steps:

[0015] A first fitting feature with the highest probability in the CT image data set is obtained from each of the channel features;

[0016] A second fitting feature with the highest probability in the CT image data set is obtained after combining at least two of the channel features;

[0017] A matching degree of the first fitting feature is matched from the sample data set, and a first judgment is performed to determine a bone type of the CT image data;

[0018] A matching degree of the second fitting feature is matched from the sample data set, and a second judgment is performed to determine whether there is a fracture in the bone type obtained by the first judgment;

[0019] A plurality of attention convolution kernels are generated from a random target position of the second fitting feature to obtain a plurality of groups of convolution feature maps under each of the attention convolution kernels;

[0020] The convolution feature maps are combined, and a third fitting feature is obtained by rounding and fitting from an average vector of each of the convolution feature maps to a vector interval determined by the sample data set;

[0021] A matching degree of the third fitting feature is matched from the sample data set, and a third judgment is performed to determine a fracture type;

[0022] The third fitting feature is written into each of the first fitting features to obtain a pixel ratio of the third fitting feature in the first fitting feature within a preset pixel interval under each channel feature, a quantization value of the pixel ratio is obtained, and a severity of the fracture is determined according to the quantization value.

[0023] Further, the image processing module includes first preprocessing and second preprocessing when performing the preprocessing;

[0024] The first preprocessing includes denoising, contrast enhancement and normalization.

[0025] The second preprocessing obtains the CT image data after the first preprocessing, and cuts the CT image into a plurality of groups of sub-images of a preset pixel size according to the image acquisition time of the CT image to generate a CT image data set.

[0026] Further, the channel features at least include texture channel features and pixel value channel features.

[0027] Further, when processing the texture channel features, the processor is configured to perform the following steps:

[0028] Obtain the CT image data set after preprocessing, and obtain pixel vector features from at least a first direction and a second direction from one of the image data;

[0029] Obtain another image data from the CT image data set, which has a similarity to the pixel vector features in the first direction and the second direction within a preset interval;

[0030] Obtain a plurality of maximum value pixels of the maximum range interval of the vector change rate of all the CT image data in the first direction and the second direction;

[0031] Merge all the maximum value pixels to obtain an instance boundary of the maximum vector change rate interval;

[0032] According to the classifier, obtain a first channel and a second channel corresponding to the first direction and the second direction under the texture channel features, and generate a first image and a second image;

[0033] Downsample the pixels in the first image and the second image other than the instance boundary according to the pixel value normalization size, and repeat the steps before generating the first image and the second image, and reduce the change rate range of the vector change rate interval to a preset value each time the loop is executed until the first image and the second image cannot be downsampled;

[0034] Write the first image and the second image into the instance segmentation image;

[0035] Obtain the position of the first image in the instance segmentation image, and the direction of the vector occurrence;

[0036] Obtain the distribution area and the total number of intervals of the distribution of the second image in the instance segmentation image;

[0037] Fit the first image and the second image to obtain a texture first fitting feature.

[0038] Further, the neural network model is a Vision-LSTM architecture, which includes a CNN and an LSTM-based RNN architecture,

[0039] The CNN includes a convolutional layer, a pooling layer, and a fully connected layer;

[0040] The channel cross-attention mechanism module is located after the convolutional layer and before the fully connected layer, and is configured to weight the features of different channels and highlight the features related to the fracture.

[0041] The channel cross-attention mechanism module includes a channel attention submodule and a spatial attention submodule: the channel attention submodule is configured to calculate the importance weight of different channels, obtain the feature descriptor of each channel through global average pooling and global maximum pooling operations, and then pass through a multi-layer perceptron and an activation function to obtain the channel attention weight.

[0042] The spatial attention submodule is configured to calculate the importance weight of different spatial positions by performing convolutional operations on the feature map to obtain the spatial attention weight.

[0043] The channel attention weight and the spatial attention weight are multiplied to obtain the final attention weight, which is used to weight the feature map and highlight the features related to the fracture.

[0044] The RNN includes an input gate, a forget gate, and an output gate, which are configured to obtain the image sequence processed by the CNN, process the image sequence with a preset weight, and return it to the CNN for storage.

[0045] Further, when the RNN obtains a sequence of pixel continuous texture channel features, it is specifically configured to perform the following steps:

[0046] Write the first fitting feature obtained from the pixel value channel feature, and cut the first fitting feature into a strip-shaped pixel sequence at a preset pixel interval,

[0047] Obtain a first vector sequence of first vectors composed of pixel values in the strip-shaped pixel sequence;

[0048] Obtain a second vector sequence of second vectors composed of texture continuity in the texture channel feature;

[0049] Write the first vector sequence and the second vector sequence into the RNN at the same time, and delete the vectors outside the vector size threshold interval in the corresponding positions of the first vector sequence in the interval vector set within the preset continuity threshold in the first direction of the second vector sequence.

[0050] Obtain a combined sequence, each unit of the combined sequence including a first fitting set of first vectors and second vectors.

[0051] performing, in parallel, in the RNN, the set of interval vectors within the continuity threshold of the second direction, writing the set of interval vectors to a first fitting set, obtaining a second fitting set;

[0052] obtaining, from the second fitting set, a single cell in which the first vector does not exist, obtaining an average product of the vector product of the single cell and multiple cells in the first direction and the second direction, writing a third vector to the position of the single cell, and reconstructing the first fitting set to generate the second fitting feature.

[0053] Further, when the fracture detection module determines that there is a fracture, it is further configured to perform the following steps:

[0054] According to the position of the fracture feature obtained in the CT image data set, the fracture position is determined.

[0055] The second aspect of the application provides a CT image fracture intelligent auxiliary detection method based on channel cross attention mechanism, comprising the following steps:

[0056] Obtaining a pre-processed CT image data set, and obtaining pixel vector features from a first direction and a second direction from one of the image data;

[0057] Obtaining another image data from the CT image data set, which has a pixel feature similarity in the first direction and the second direction within a pre-interval;

[0058] Obtaining a plurality of pixel points with maximum vector change rate in the first direction and the second direction from all the image data;

[0059] Merging all the pixel points to obtain the boundary of the maximum vector change, and obtaining an instance segmentation image;

[0060] Obtaining a normalized first channel pixel value of the entire pixel set according to the classifier, and extracting a first image of the pixel set;

[0061] Obtaining a normalized second channel pixel value of the entire pixel set according to the classifier, and extracting a second image of the pixel set;

[0062] Writing the first image and the second image into the instance segmentation image;

[0063] Obtaining the position of the first image in the instance segmentation image and the direction of the vector occurrence;

[0064] Obtaining the distribution area and the total number of intervals of the second image in the instance segmentation image;

[0065] fitting the first image and the second image to obtain a texture first fitting feature;

[0066] writing a pixel first fitting feature from a pixel value channel feature, cutting the pixel first fitting feature into a strip pixel sequence at a preset pixel interval,

[0067] obtaining a first vector sequence of first vectors composed of pixel values in the strip pixel sequence;

[0068] obtaining a second vector sequence of second vectors composed of texture continuity in the texture channel feature;

[0069] simultaneously writing the first vector sequence and the second vector sequence into the RNN, obtaining a vector size threshold interval outside a vector in the first vector sequence corresponding to a position in an interval vector set within a preset continuity threshold in the second direction from the second vector sequence;

[0070] obtaining a combination sequence, each unit of the combination sequence including a first fitting set of first vectors and second vectors;

[0071] performing an interval vector set within a continuity threshold in the second direction in the RNN in parallel, writing the interval vector set into the first fitting set to obtain a second fitting set;

[0072] obtaining a single unit from the second fitting set in which the first vector does not exist, obtaining an average product of vector products of multiple units of the single unit in the first direction and the second direction, writing a third vector to the position of the single unit, and generating the second fitting feature after reconstructing the first fitting set;

[0073] obtaining a first fitting feature with the highest probability in the CT image data set from each of the channel features;

[0074] combining at least two of the channel features, and obtaining a second fitting feature with the highest probability in the CT image data set after average pooling;

[0075] matching a matching degree of the first fitting feature from a sample data set, and determining a bone type of the CT image data by a first determination;

[0076] matching a matching degree of the second fitting feature from a sample data set, and determining whether a fracture exists in the bone type obtained by the first determination by a second determination;

[0077] obtaining a plurality of groups of convolution feature maps under a plurality of groups of attention convolution kernels from a random target position of the second fitting feature;

[0078] combine the convolution feature maps, round off from the average vector difference of each convolution feature map and fit a third fitted feature;

[0079] match the matching degree about the third fitted feature from the sample data set, determine the type of the fracture by the third determination;

[0080] overwrite the third fitted feature to the first fitted feature, determine the pixel ratio of the third fitted feature in the first fitted feature by the third determination, obtain the quantization value about the pixel ratio, and determine the severity of the fracture according to the quantization value.

[0081] The third aspect of the present application provides an electronic device.

[0082] The fourth aspect of the present application is a computer readable storage medium, which stores a computer program, and the program is executed by a processor to realize the functions of the system.

[0083] The above technical solution of the present application has the following advantages compared with the prior art:

[0084] The present application can greatly improve the detection efficiency and accuracy by using advanced computer technology and artificial intelligence algorithm to automatically analyze CT images, and reduce human error, and provide more timely and reliable diagnostic services for patients. BRIEF DESCRIPTION OF DRAWINGS

[0085] Figure 1 is a module connection diagram of the system provided by the first embodiment of the present application;

[0086] Figure 2 is a module connection diagram of the electronic device provided by the embodiment of the present application.

[0087] Among them, 30, processor; 31, memory; 32, communication interface; 33, bus. DETAILED DESCRIPTION

[0088] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0089] Embodiment one

[0090] The first aspect of the present application provides a CT image fracture intelligent auxiliary detection system based on channel cross attention mechanism, comprising:

[0091] an image acquisition module configured to acquire CT image data from at least one database or a real-time image terminal;

[0092] an image preprocessing module connected with the image acquisition module and configured to perform preprocessing of the CT image data to obtain a CT image data set;

[0093] a feature extraction module connected with the image preprocessing module and configured to store a neural network model to obtain a plurality of groups of channel features of each image in the CT image data set;

[0094] a fracture detection module connected with the feature extraction module and configured to store a sample data set and obtain a plurality of groups of channel features extracted by the neural network model to perform a judgment on a fracture;

[0095] when the first judgment and the second judgment are performed, the feature extraction module and the fracture detection module are configured to perform the following steps:

[0096] obtain a first fitting feature with the highest probability in the CT image data set from each of the channel features;

[0097] combine at least two of the channel features, and obtain a second fitting feature with the highest probability in the CT image data set after average pooling;

[0098] match a matching degree of the first fitting feature from the sample data set to determine a bone type of the CT image data by the first judgment;

[0099] match a matching degree of the second fitting feature from the sample data set to determine whether there is a fracture in the bone type obtained by the first judgment by the second judgment;

[0100] obtain a plurality of groups of convolution feature maps under a plurality of groups of attention convolution kernels from a random target position of the second fitting feature;

[0101] combine the convolution feature maps, take an integer from an average vector difference of each of the convolution feature maps, and fit a third fitting feature;

[0102] match a matching degree of the third fitting feature from the sample data set to determine a type of the fracture by a third judgment;

[0103] overwrite the third fitting feature into the first fitting feature, determine a pixel ratio of the third fitting feature in the first fitting feature by the third judgment, obtain a quantization value of the pixel ratio, and determine a severity of the fracture according to the quantization value.

[0104] Further, the image processing module includes first preprocessing and second preprocessing when performing preprocessing.

[0105] The first preprocessing includes denoising, contrast enhancement and normalization;

[0106] The second preprocessing obtains the CT image data obtained by the first preprocessing, and generates a CT image data set by cutting the CT image into a plurality of groups of sub-images of a preset pixel size according to the image acquisition time of the CT image.

[0107] Further, when the pixel continuous texture channel feature is obtained, the following steps are configured to be performed:

[0108] Obtain the CT image data set after preprocessing, and obtain pixel vector features from the first direction and the second direction from one of the image data;

[0109] Obtain another image data from the CT image data set, which has a pixel similarity in the first direction and the second direction within a preset interval;

[0110] Obtain a plurality of pixel points with the maximum vector change rate in the first direction and the second direction in all the image data;

[0111] Merge all the pixel points to obtain the boundary of the maximum vector change, and obtain an instance segmentation image;

[0112] Obtain a normalized first channel pixel value from the classifier, and extract a first image of the pixel set;

[0113] Obtain a normalized second channel pixel value from the classifier, and extract a second image of the pixel set;

[0114] Write the first image and the second image into the instance segmentation image;

[0115] Obtain the position of the first image in the instance segmentation image, and the direction of the vector occurrence;

[0116] Obtain the distribution area of the second image in the instance segmentation image, and the total number of distribution intervals;

[0117] Fit the first image and the second image to obtain a texture first fitting feature.

[0118] Further, the neural network model is a Vision-LSTM architecture, which includes a CNN and an LSTM-based RNN architecture,

[0119] The CNN includes a convolutional layer, a pooling layer and a fully connected layer;

[0120] The channel cross-attention mechanism module is located after the convolutional layer and before the fully connected layer, and is used for weighting features of different channels to highlight the features related to the fracture;

[0121] The channel cross-attention mechanism module includes a channel attention sub-module and a spatial attention sub-module: the channel attention sub-module is used for calculating importance weights of different channels, obtaining feature descriptors of each channel through global average pooling and global maximum pooling operations, and then obtaining channel attention weights through a multi-layer perceptron and an activation function;

[0122] The spatial attention sub-module is used for calculating importance weights of different spatial positions, and the spatial attention weights are obtained by performing convolutional operation on the feature map;

[0123] The channel attention weights and the spatial attention weights are multiplied to obtain the final attention weights, which are used for weighting the feature map to highlight the features related to the fracture;

[0124] The RNN includes an input gate, a forget gate and an output gate, which are configured to obtain the image sequence processed by the CNN, process the image sequence with a preset weight, and return the image sequence to the CNN for storage.

[0125] Further, when the RNN obtains the sequence of pixel continuous texture channel features, the following steps are specifically configured to be performed:

[0126] The pixel first fitting features obtained from the pixel value channel features are written, and the pixel first fitting features are cut into a strip-shaped pixel sequence at a preset pixel interval,

[0127] A first vector sequence of first vectors composed of pixel values in the strip-shaped pixel sequence is obtained;

[0128] A second vector sequence of second vectors composed of texture continuity in the texture channel features is obtained;

[0129] The first vector sequence and the second vector sequence are written into the RNN at the same time, and the vector size threshold outside the vector in the corresponding position of the first vector sequence in the interval vector set within the preset continuity threshold in the second direction of the second vector sequence is deleted;

[0130] A combined sequence is obtained, each unit of the combined sequence including a first fitting set of the first vector and the second vector;

[0131] The interval vector set within the continuity threshold in the second direction is executed in the RNN in parallel, and the interval vector set is written into the first fitting set to obtain a second fitting set;

[0132] obtaining a single cell in which the first vector does not exist from the second fitting set, obtaining an average product of the vector product of the single cell and multiple cells in the first direction and the second direction, writing a third vector to the position of the single cell, and generating the second fitting feature after reconstructing the first fitting set.

[0133] Further, when the fracture detection module determines that a fracture exists, the fracture detection module is further configured to perform the following steps:

[0134] According to the position of the fracture feature obtained in the CT image data set, the position of the fracture is determined.

[0135] The second aspect of the application provides a CT image fracture intelligent auxiliary detection method based on a channel cross-attention mechanism, comprising the following steps:

[0136] Obtaining a pre-processed CT image data set, and obtaining pixel vector features from a first direction and a second direction from one of the image data sets;

[0137] Obtaining another image data from the CT image data set, which has a similarity in pixel features in the first direction and the second direction within a predetermined interval;

[0138] Obtaining a plurality of pixel points with maximum vector change rates in the first direction and the second direction from all the image data;

[0139] Merging all the pixel points to obtain the boundary of the maximum vector change, and obtaining an instance segmentation image;

[0140] Obtaining a normalized first channel pixel value from the classifier, and extracting a first image from the pixel set;

[0141] Obtaining a normalized second channel pixel value from the classifier, and extracting a second image from the pixel set;

[0142] Writing the first image and the second image into the instance segmentation image;

[0143] Obtaining the position of the first image in the instance segmentation image, and the direction of the vector occurrence;

[0144] Obtaining the distribution area and the total number of distribution intervals of the second image in the instance segmentation image;

[0145] Fitting the first image and the second image to obtain a texture first fitting feature;

[0146] Writing a pixel first fitting feature obtained from a pixel value channel feature, and cutting the pixel first fitting feature into a strip-shaped pixel sequence at a predetermined pixel interval,

[0147] obtaining a first vector sequence of first vectors composed of pixel values in the strip pixel sequence;

[0148] obtaining a second vector sequence of second vectors composed of texture continuity in the texture channel feature;

[0149] simultaneously writing the first vector sequence and the second vector sequence into the RNN, obtaining a set of interval vectors within a preset continuity threshold in the second direction, and deleting vectors outside a vector size threshold of corresponding positions in the first vector sequence in the set of interval vectors;

[0150] obtaining a combined sequence, each unit of the combined sequence including a first fitting set of first vectors and second vectors;

[0151] performing, in the RNN, a set of interval vectors within a continuity threshold in the second direction in parallel, writing the set of interval vectors into a first fitting set, and obtaining a second fitting set;

[0152] obtaining a single unit from the second fitting set in which the first vector does not exist, obtaining an average product of vector products of multiple units in the first direction and the second direction of the single unit, writing a third vector to a position of the single unit, and generating the second fitting feature after reconstructing the first fitting set;

[0153] obtaining a first fitting feature with the highest probability in the CT image data set from each of the channel features;

[0154] combining at least two of the channel features, and obtaining a second fitting feature with the highest probability in the CT image data set after average pooling;

[0155] matching a matching degree with respect to the first fitting feature from a sample data set, and determining a bone type of the CT image data by a first determination;

[0156] matching a matching degree with respect to the second fitting feature from a sample data set, and determining whether a fracture exists in the bone type obtained by the first determination by a second determination;

[0157] obtaining a plurality of groups of convolution feature maps under a plurality of groups of attention convolution kernels from randomly targeted positions of the second fitting feature;

[0158] combining the convolution feature maps, and fitting a third fitting feature from an average vector difference of each of the convolution feature maps;

[0159] matching a matching degree with respect to the third fitting feature from a sample data set, and determining a type of the fracture by a third determination;

[0160] The third fitting feature is overwritten into the first fitting feature, a third fitting feature to pixel ratio in the first fitting feature is determined by a third determination, a quantization value about the pixel ratio is obtained, and the severity of the fracture is determined according to the quantization value.

[0161] The second preprocessing provided in the embodiments of the present disclosure specifically includes the following steps:

[0162] First, the original image is divided into N x N small blocks. For example, for an image with a size of 256 x 256, it can be divided into 16 x 16 small blocks. Each small block is given a unique number from 1 to N 2 . Then, the image blocks are shuffled by hash values.

[0163] pi(i) = index of i in sorted order of {h(seed+i) | i = 1, 2,..., N 2}

[0164] Where h is a hash function and seed is a random seed.

[0165] In the embodiments of the present disclosure, a CNN is used to calculate the features related to the segments. Each block of pixels is normalized so that the pixel value varies between -1 and 1. For an 8-bit depth image, if the color depth of the image is higher (such as 16-bit or 32-bit), it is first converted to an 8-bit image and then normalized. And make sure the image is in the standard RGB color space.

[0166] f i = CNN(x i )

[0167] Where x i is the i-th image block, and f i is the feature vector extracted by CNN.

[0168] Image block relevance determination: the cosine similarity is used to determine the relevance between image blocks. The higher the similarity, the stronger the picture relevance. First, calculate the similarity between the feature vectors f i , f j and the feature vectors f i and f j of any selected image blocks. Cosine similarity or Euclidean distance can be used: similarity

[0169]

[0170] Feature combination: To capture the complex dependencies between two image patches, including spatial and semantic relationships, we achieve this goal by connecting two features in the combination layer. The combination of features f1 and f2 uses the Kronecker product, providing a rich feature representation for the subsequent layers:

[0171]

[0172] where denotes the Kronecker product, which combines the features of two patches.

[0173] Probability matrix generation: For the central patch, the network computes the probability of each other patch with its relative position, resulting in a probability matrix P. The process of generating the probability matrix P is as follows:

[0174] Fully connected layer processing: The combined features φ ij are input into a fully connected layer, which performs a linear transformation to obtain an intermediate vector z ij :

[0175]

[0176] where w is the weight matrix and b is the bias vector.

[0177] Activation function application: The ReLU activation function is applied to the result of the linear transformation:

[0178] a ij = ReLU(z ij ) = max(0, z ij )

[0179] Softmax transformation: The activated vector a ij is converted into a probability distribution using the Softmax function,

[0180] The Softmax function ensures that each position probability in the output is between 0 and 1, and the sum is 1. The output vector P ij represents the position probabilities, where each element P ij，k represents the probability that image patch i is in the kth position relative to image patch j.

[0181]

[0182] Optimal assignment finding: Using a greedy algorithm, the highest probability elements are selected from the probability matrix P, ensuring that only one element is selected from each row and column, thus finding the optimal assignment:

[0183]

[0184] where σ is the permutation of selecting elements from the matrix such that the sum is maximized.

[0185] Embodiment Two

[0186] In combination Figure 2 As shown in FIG. 1, the electronic device provided by the embodiments of the present disclosure includes a processor 30 and a memory 31. Optionally, the electronic device can further include a communication interface 32 and a bus 33. The processor 30, the communication interface 32 and the memory 31 can communicate with each other through the bus 33. The communication interface 32 can be used for information transmission. The processor 30 can invoke the logical instructions in the memory 31 to execute the method of Embodiment One.

[0187] The embodiments of the present disclosure provide a storage medium, which stores computer executable instructions configured to execute the method of Embodiment One.

[0188] The storage medium described above can be a transitory computer readable storage medium or a non-transitory computer readable storage medium. The non-transitory storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and the like, which can store program codes, and can also be a transitory storage medium.

[0189] The above description and drawings are illustrative of embodiments of the present disclosure and are not intended to be limiting. Other embodiments can include structural, logical, electrical, process, and other changes. Embodiments are merely representative of possible variations. Individual components and functions are optional unless explicitly required, and the order of operations can be varied. Portions and features of some embodiments can be included in, or substituted for, those of other embodiments. Also, words used in this document and claims are words of description, not limitation. As used in the description and claims herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Similarly, the term "and / or" as used herein refers to any one or more of the associated listed items, optionally including zero of the associated listed items. Additionally, the term "comprising" and variations thereof as used herein are intended to be open-ended terms that specify the presence of the stated features, elements, steps, operations, integers, and / or components, but do not preclude the presence or addition of one or more other features, elements, steps, operations, integers, components, and / or groups thereof. The term "consisting of" as used herein is intended to be a closed term that specifies the presence of the stated features, elements, steps, operations, integers, and / or components, but does not preclude the presence or addition of one or more other features, elements, steps, operations, integers, components, and / or groups thereof. Unless otherwise expressly stated, mechanisms of the present disclosure can be implemented in either hardware, software, or a combination thereof. The description herein assumes that the mechanisms are implemented in software, unless specifically stated otherwise. The description assumes that the software is implemented in a high level procedural or object-oriented programming language that is directed to computer software and / or firmware applications. However, it will be apparent to those of ordinary skill in the art that each of the mechanisms can be implemented in a number of different computing and / or processing environments, and / or implemented in hardware and / or circuitry, without departing from the scope of the present disclosure. Accordingly, the description is not intended to limit the scope of the present disclosure, but instead is intended to cover all such mechanisms within the scope of the present disclosure.

[0190] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods for each specific application to realize the described functions, but such implementation should not be considered beyond the scope of the embodiments of the present disclosure. The skilled person can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices, apparatuses, and units can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0191] The diagrams of the flow and block diagrams show the architecture, functionality, and operation of possible implementations of apparatuses, methods and computer program products according to embodiments of the present disclosure. In this regard, each block in the flow and block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logic functions. In some alternative implementations, the actions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession can in fact be executed substantially concurrently or the actions of a block can be performed in the reverse order, depending upon the functionality involved. These diagrams of the flow and block diagrams are also intended to include any connected data storage and data processing artifacts and structures that can affect the operation of the subject matter described. If warranted, specific data storage artifacts can be shown in a block diagram and / or a flow diagram and referred to in the accompanying text. Conversely, no indication of such data storage artifacts should not be construed to imply that such data storage is not a possible implementation. In some alternative implementations, the actions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession can in fact be executed substantially concurrently or the actions of a block can be performed in the reverse order, depending upon the functionality involved. The description of a flow or block diagram of a process, method, or computer program product should not be construed to mean that all of the actions or steps are required to be performed in the order presented, nor that they are performed at all.

Claims

1. A CT image fracture intelligent auxiliary detection system based on channel cross-attention mechanism, characterized in that, include: The image acquisition module is configured to acquire CT image data from at least one database or real-time imaging terminal. An image preprocessing module, connected to the image acquisition module, is configured to perform preprocessing of the CT image data to obtain a CT image dataset. The feature extraction module, connected to the image preprocessing module, is configured to store a neural network model to obtain multiple sets of channel features for each CT image in the CT image dataset; the channel features include at least texture channel features and pixel value channel features. The fracture detection module, connected to the feature extraction module, is configured to store sample datasets and acquire multiple sets of channel features extracted by the neural network model, and to perform fracture detection. When determining a fracture, the feature extraction module and the fracture detection module are configured to perform the following steps: From each of the channel features, obtain the first fitted feature with the highest probability in the CT image dataset; By combining at least two of the channel features, a second fitted feature with the highest probability in the CT image dataset is obtained; Match the degree of matching with the first fitted feature from the sample dataset, and determine the bone type of the CT image data through the first judgment; Match the degree of matching with respect to the second fitted feature from the sample dataset, and determine whether there is a fracture in the bone type obtained by the first judgment through the second judgment; Multiple attention convolution kernels are generated from the random target positions of the second fitted features to obtain multiple sets of convolution feature maps under each attention convolution kernel; By combining the convolutional feature maps, a third fitted feature is obtained by rounding down the average vector of each convolutional feature map to the vector interval determined by the sample dataset. Match the degree of matching with the third fitting feature from the sample dataset, and determine the fracture type by the third judgment; write the third fitting feature into each of the first fitting features, and obtain the ratio of the third fitting feature in the first fitting feature within a preset pixel interval under each channel feature, obtain the quantized value of the pixel ratio, and determine the severity of the fracture based on the quantized value.

2. The intelligent auxiliary detection system for CT image fractures based on channel cross-attention mechanism according to claim 1, characterized in that, When the image preprocessing module performs preprocessing, it includes a first preprocessing and a second preprocessing. The first preprocessing includes noise reduction, contrast enhancement, and normalization; After the second preprocessing obtains the CT image data obtained by the first preprocessing, the CT image is cropped into multiple sub-images of preset pixel size according to the image acquisition time of the CT image to generate a CT image dataset.

3. The intelligent auxiliary detection system for CT image fractures based on channel cross-attention mechanism according to claim 1, characterized in that, When processing the texture channel features, it is configured to perform the following steps: Obtain a preprocessed CT image dataset, and extract pixel vector features from at least the first and second directions from one of the image datasets; Obtain another image data from the CT image dataset whose pixel vector features are similar to those in the first and second directions within a preset range; Obtain multiple maximum pixel values ​​for the maximum range of vector change rate in the first and second directions for all the CT image data; merge all the maximum pixel values ​​to obtain the instance boundary for the maximum vector change rate range; Based on the classifier, the first and second channels corresponding to the first and second directions under the texture channel features are obtained, and a first image and a second image are generated. The pixels in the first and second images, except for the instance boundaries, are downsampled according to the normalized pixel value size, and the steps before generating the first and second images are repeated. In each loop, the range of the vector change rate interval is reduced to a preset value so that the first and second images cannot be downsampled. Write the first image and the second image into the instance segmentation image; obtain the position of the first image in the instance segmentation image and the direction of the vector. Obtain the distribution area and the total number of distribution intervals of the second image in the instance segmented image; fit the first image and the second image to obtain the first texture fitting feature.

4. The intelligent auxiliary detection system for CT image fractures based on channel cross-attention mechanism according to claim 3, characterized in that, The neural network model is a Vision-LSTM architecture, which includes CNN and LSTM-based RNN architecture. The CNN includes convolutional layers, pooling layers, and fully connected layers; Among them, the channel cross-attention mechanism module is located after the convolutional layer and before the fully connected layer. It is used to weight the features of different channels and highlight the features related to fracture. The channel cross-attention mechanism module includes a channel attention submodule and a spatial attention submodule: the channel attention submodule is used to calculate the importance weights of different channels. It obtains the feature descriptor of each channel through global average pooling and global max pooling operations, and then passes it through a multilayer perceptron and activation function to obtain the channel attention weights; The spatial attention submodule is used to calculate the importance weights of different spatial locations. It obtains the spatial attention weights by performing a convolution operation on the feature map. The channel attention weights and spatial attention weights are multiplied to obtain the final attention weights, which are used to weight the feature map and highlight fracture-related features. The RNN includes an input gate, a forget gate, and an output gate. It is configured to acquire the image sequence processed by the CNN, process the image sequence with preset weights, return it to the CNN, and store it.

5. The intelligent auxiliary detection system for CT image fractures based on channel cross-attention mechanism according to claim 4, characterized in that, When the RNN obtains the sequence of pixel continuous texture channel features, it is specifically configured to perform the following steps: Write the first fitting feature of the pixel obtained from the pixel value channel feature, and cut the first fitting feature of the pixel into a strip pixel sequence with a preset pixel interval to obtain the first vector sequence of the first vector composed of the pixel values ​​in the strip pixel sequence; Obtain a second vector sequence composed of texture continuity in the texture channel features; simultaneously write the first vector sequence and the second vector sequence into the RNN, obtain the set of interval vectors in the first direction of the second vector sequence within a preset continuity threshold, and delete the vectors outside the vector size threshold interval at the corresponding positions in the first vector sequence; A combined sequence is obtained, each unit of which includes a first fitting set of a first vector and a second vector; an interval vector set within a continuity threshold in the second direction is executed in parallel in the RNN, and the interval vector set is written into the first fitting set to obtain a second fitting set; From the second fitting set, obtain a single unit that does not exist in the first vector, obtain the average product of the vector products of multiple units of the single unit in the first direction and the second direction, write the third vector to the position of the single unit, and generate the second fitting feature after reconstructing the first fitting set.

6. The intelligent auxiliary detection system for CT image fractures based on channel cross-attention mechanism according to claim 1, characterized in that, When the fracture detection module determines that a fracture exists, it is also configured to perform the following steps: determine the fracture location based on the location of the acquired fracture features in the CT image dataset.

7. A CT image intelligent assisted detection method based on channel cross-attention mechanism, characterized in that, Includes the following steps: Obtain the preprocessed CT image dataset, and extract pixel vector features from one of the image datasets from the first and second directions; Obtain another image data from the CT image dataset whose pixel feature similarity with the first direction and the second direction is within a pre-range; obtain multiple pixels with the maximum value of the vector change rate in the first direction and the second direction from all the image data; Merge all pixels to obtain the boundary of the maximum vector change, and obtain the instance segmentation image; obtain the set of all pixels with normalized first channel pixel values ​​according to the classifier, and extract the first image of the pixel set; Based on the set of all pixels with normalized second channel pixel values ​​obtained by the classifier, extract the second image of the pixel set; write the first image and the second image into the instance segmentation image; Obtain the position of the first image in the instance segmented image and the direction of the vector; obtain the distribution area and the total number of distribution intervals of the second image in the instance segmented image; The first and second images are fitted to obtain the first texture fitting feature; Write the first fitted pixel feature obtained from the pixel value channel feature, and cut the first fitted pixel feature into a strip pixel sequence at a preset pixel interval. Obtain a first vector sequence of the first vector composed of pixel values ​​in the strip pixel sequence; Obtain the second vector sequence of the second vector composed of texture continuity in the texture channel features; Simultaneously, the first vector sequence and the second vector sequence are written into the RNN to obtain the set of interval vectors in the first direction of the second vector sequence within a preset continuity threshold. Vectors outside the threshold interval of the vector size at the corresponding position in the first vector sequence are then deleted. A combined sequence is obtained, wherein each unit of the combined sequence includes a first fitted set of a first vector and a second vector; In the RNN, the set of interval vectors within the continuity threshold in the second direction is executed in parallel, and the set of interval vectors is written into the first fitting set to obtain the second fitting set; From the second fitting set, obtain a single unit that does not exist in the first vector, obtain the average product of the vector products of the single unit with respect to the multiple units in the first direction and the second direction, write the third vector to the position of the single unit, and generate the second fitting feature after reconstructing the first fitting set. From each of the channel features, obtain the first fitted feature with the highest probability in the CT image dataset; By combining at least two of the channel features and performing average pooling, a second fitted feature with the highest probability in the CT image dataset is obtained. Match the degree of matching with the first fitted feature from the sample dataset, and determine the bone type of the CT image data by the first judgment; Match the degree of matching with respect to the second fitted feature from the sample dataset, and determine whether there is a fracture in the bone type obtained by the first judgment by the second judgment; Using the random target position of the second fitted feature as the attention convolution kernel, multiple sets of convolution feature maps under the attention convolution kernel are obtained; the convolution feature maps are combined, and the average vector difference of each convolution feature map is rounded down and fitted to the third fitted feature. The matching degree of the third fitting feature is matched from the sample dataset, and the type of fracture is determined by the third judgment; the third fitting feature is overwritten into the first fitting feature, and the pixel ratio of the third fitting feature to the first fitting feature is determined by the third judgment, and a quantized value of the pixel ratio is obtained, and the severity of the fracture is determined based on the quantized value.

8. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, it implements the method as described in claim 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in claim 7.

Images