Hip joint impact syndrome intelligent diagnosis method based on multi-modal data fusion

Abstract

The invention provides a hip joint impact syndrome intelligent diagnosis method, system and device based on multi-modal data fusion and a computer readable storage medium. The intelligent diagnosis method for the hip joint impact syndrome based on multi-modal data fusion comprises the following steps: acquiring a hip joint X-ray image and health condition text information of a patient; wherein the health condition text information comprises general information, medical history information, symptom information and physical examination information of the patient; inputting the hip joint X-ray image and the health condition text information into a preset hip joint impact syndrome diagnosis model, and outputting a hip joint impact syndrome diagnosis result; according to the hip joint impact syndrome diagnosis model, text features and image features are extracted and fused for diagnosis. According to the embodiment of the invention, the hip joint impingement syndrome can be diagnosed more accurately.

Description

Technical Field

[0001] This application belongs to the field of intelligent diagnosis of femoroacetabular impingement syndrome, and particularly relates to an intelligent diagnosis method, system, device and computer-readable storage medium for femoroacetabular impingement syndrome based on multi-modal data fusion. Background Art

[0002] Currently, surgeons can determine the specific degree of cam impingement based on the standard measurement results obtained from medical images. More specifically, surgeons analyze and diagnose cam impingement symptoms based on the relationship between the femoral head and neck, or the alpha angle, that is, visually determined from images provided by X-ray devices, computed tomography (CT) devices, magnetic resonance imaging (MRI) devices, and so on.

[0003] However, it is inaccurate for doctors to diagnose femoroacetabular impingement syndrome by visually observing the relationship between the femoral head and neck.

[0004] Therefore, how to diagnose femoroacetabular impingement syndrome more accurately is a technical problem that those skilled in the art urgently need to solve. Summary of the Invention

[0005] The embodiments of this application provide an intelligent diagnosis method, system, device and computer-readable storage medium for femoroacetabular impingement syndrome based on multi-modal data fusion, which can diagnose femoroacetabular impingement syndrome more accurately.

[0006] In a first aspect, the embodiments of this application provide an intelligent diagnosis method for femoroacetabular impingement syndrome based on multi-modal data fusion, including:

[0007] Obtain the hip X-ray image and health status text information of the patient; wherein, the health status text information includes: general information, medical history information, symptom information, and physical examination information of the patient;

[0008] Input the hip X-ray image and health status text information into a preset femoroacetabular impingement syndrome diagnosis model, and output the femoroacetabular impingement syndrome diagnosis result;

[0009] Among them, the femoroacetabular impingement syndrome diagnosis model extracts and fuses text features and image features for diagnosis.

[0010] Optionally, the femoroacetabular impingement syndrome diagnosis result includes:

[0011] Cam type, pincer type or mixed type.

[0012] Among them, the cam type occurs at the junction of the femoral head and the neck. Due to the abnormal shape of the femoral head, a non-spherical bulge is formed in the femoral head and neck region. When the hip joint moves, this bulge will make abnormal contact with the acetabular rim, especially when the hip joint is flexed or rotated, resulting in wear or damage to the articular cartilage and acetabular labrum.

[0013] The pincer type is due to excessive coverage of the acetabulum, resulting in excessive contact between the acetabular rim and the femoral head. In this case, the bone mass or labrum at the acetabular rim will squeeze the femoral head during the movement of the hip joint, especially when the hip joint is abducted or rotated, resulting in damage to the labrum.

[0014] The combined type is a combination of the cam type and the pincer type, and the patient has both cam-type and pincer-type anatomical abnormalities.

[0015] Optionally, in the health status text information:

[0016] General information: including gender, age, sports enthusiasts, whether smoking, etc.;

[0017] Medical history information: including history of intramuscular injection in the buttocks, knee joint history, ankle joint history, other injuries, etc.;

[0018] Symptom information: including the location of hip pain, type of hip pain, hip symptoms, the impact of symptoms on life, etc.;

[0019] Physical examination information: including whether the lengths of the two lower limbs are equal, atrophy of the left gluteal muscles, left gluteal scar, range of motion of the left hip joint, etc.

[0020] Optionally, input the hip joint X-ray image and the health status text information into a preset diagnosis model for femoroacetabular impingement syndrome, and output the diagnosis result of femoroacetabular impingement syndrome, including:

[0021] The text encoder converts the health status text information into a series of text feature vectors;

[0022] The image encoder converts the hip joint X-ray image into a series of image feature vectors;

[0023] Fuse the text feature vectors and the image feature vectors to generate a feature matrix; among them, each element represents the combination of the image feature vector and the text feature vector;

[0024] Based on the feature matrix, output the diagnosis result of femoroacetabular impingement syndrome.

[0025] Optionally, the image encoder converts the hip joint X-ray image into a series of image feature vectors, including:

[0026] Extract features from the hip joint X-ray image to obtain a feature map;

[0027] Sum the obtained feature maps after dilated convolution operations with different dilation rates element-wise;

[0028] Perform a per-channel convolution on the summation result to fuse features of different scales, followed by global average pooling, a fully connected layer to compress the number of feature channels, a fully connected layer to restore the number of channels, and a Sigmoid function for non-linear activation to obtain an image feature vector.

[0029] Optionally, the text encoder converts the health condition text information into a series of text feature vectors, including:

[0030] Convert the health condition text information into text feature word vectors using the ALBERT technique;

[0031] Use a text feature extraction network to extract features from the text feature word vectors to obtain optimal feature vectors;

[0032] Among them, the text feature extraction network includes a convolutional neural network with an attention mechanism and a bidirectional GRU neural network;

[0033] Extract local feature vectors and global feature vectors of the text using the convolutional neural network and the bidirectional GRU network respectively, and finally fuse the feature vectors of the two to output optimal feature vectors.

[0034] Optionally, extract local feature vectors and global feature vectors of the text using the convolutional neural network and the bidirectional GRU network respectively, and finally fuse the feature vectors of the two to output optimal feature vectors, including:

[0035] Use the mini-batch gradient descent method to input the text feature word vectors into the convolutional neural network. The convolutional layer extracts various different feature expressions of the feature words through convolutional kernels of different sizes. The pooling layer compresses and reduces the dimension of the features output by the convolutional layer, retains the main features of the feature words, and finally the pooling layer outputs the extracted features to the fully connected layer to obtain local features;

[0036] At the same time, use the mini-batch gradient descent method to also input the text feature word vectors into the bidirectional GRU neural network to obtain global features.

[0037] In a second aspect, an intelligent diagnosis system for femoroacetabular impingement syndrome based on multi-modal data fusion provided by an embodiment of the present application includes:

[0038] An image-text information acquisition module for acquiring hip X-ray images and health condition text information of a patient; among them, the health condition text information includes: general information, medical history information, symptom information, and physical examination information of the patient;

[0039] A hip impingement syndrome diagnosis module, which is used to input hip X-ray images and health condition text information into a preset hip impingement syndrome diagnosis model and output a hip impingement syndrome diagnosis result;

[0040] Among them, the hip impingement syndrome diagnosis model extracts and fuses text features and image features for diagnosis.

[0041] In a third aspect, an embodiment of the present application provides an electronic device, which includes: a processor and a memory storing computer program instructions;

[0042] When the processor executes the computer program instructions, it implements an intelligent diagnosis method for hip impingement syndrome based on multi-modal data fusion.

[0043] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are executed by a processor, an intelligent diagnosis method for hip impingement syndrome based on multi-modal data fusion is implemented.

[0044] The intelligent diagnosis method, system, device and computer-readable storage medium for hip impingement syndrome based on multi-modal data fusion in the embodiments of the present application can diagnose hip impingement syndrome more accurately.

[0045] The intelligent diagnosis method for hip impingement syndrome based on multi-modal data fusion includes:

[0046] Obtain hip X-ray images and health condition text information of a patient; among them, the health condition text information includes: general information, medical history information, symptom information, and physical examination information of the patient;

[0047] Input the hip X-ray images and health condition text information into a preset hip impingement syndrome diagnosis model and output a hip impingement syndrome diagnosis result;

[0048] Among them, the hip impingement syndrome diagnosis model extracts and fuses text features and image features for diagnosis. Description of the Drawings

[0049] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for the description of the specific embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0050] Figure 1It is a schematic flowchart of an intelligent diagnosis method for femoroacetabular impingement syndrome based on multi-modal data fusion provided by an embodiment of the present application;

[0051] Figure 2 It is a schematic structural diagram of a femoroacetabular impingement syndrome diagnosis model provided by an embodiment of the present application;

[0052] Figure 3 It is a schematic flowchart of an image encoder for obtaining an image feature vector provided by an embodiment of the present application;

[0053] Figure 4 It is a schematic flowchart of a convolutional neural network for obtaining local features provided by an embodiment of the present application;

[0054] Figure 5 It is a schematic flowchart of a bidirectional GRU neural network for obtaining global features provided by an embodiment of the present application;

[0055] Figure 6 It is a schematic structural diagram of an intelligent diagnosis system for femoroacetabular impingement syndrome based on multi-modal data fusion provided by an embodiment of the present application;

[0056] Figure 7 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed Embodiments

[0057] The features and exemplary embodiments of various aspects of the present application will be described in detail below. For the purpose of making the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without some of these specific details. The following description of the embodiments is only intended to provide a better understanding of the present application by showing examples of the present application.

[0058] It should be noted that in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover a non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, elements defined by the statement "comprising..." do not preclude the existence of additional identical elements in the process, method, article or device comprising the said elements.

[0059] To solve the problems of the prior art, the embodiments of the present application provide an intelligent diagnosis method, system, device and computer-readable storage medium for femoroacetabular impingement syndrome based on multimodal data fusion. First, the intelligent diagnosis method for femoroacetabular impingement syndrome based on multimodal data fusion provided by the embodiments of the present application will be introduced below.

[0060] Figure 1 The flowchart of the intelligent diagnosis method for femoroacetabular impingement syndrome based on multimodal data fusion provided by an embodiment of the present application is shown. As Figure 1 shown, the intelligent diagnosis method for femoroacetabular impingement syndrome based on multimodal data fusion includes:

[0061] S101. Obtain the hip X-ray image and health condition text information of the patient; wherein, the health condition text information includes: general information, medical history information, symptom information, physical examination information of the patient;

[0062] S102. Input the hip X-ray image and health condition text information into a preset femoroacetabular impingement syndrome diagnosis model, and output the femoroacetabular impingement syndrome diagnosis result;

[0063] Among them, the femoroacetabular impingement syndrome diagnosis model extracts and fuses text features and image features for diagnosis.

[0064] In one embodiment, the femoroacetabular impingement syndrome diagnosis result includes:

[0065] Cam type, pincer type or combined type.

[0066] Among them, the cam type occurs at the junction of the femoral head and the neck. Due to the abnormal shape of the femoral head, a non-spherical bulge is formed in the femoral head and neck region; when the hip joint moves, this bulge will make abnormal contact with the acetabular rim, especially when the hip joint is flexed or rotated, resulting in wear or damage to the articular cartilage and acetabular labrum;

[0067] The pincer type is due to the excessive coverage of the acetabulum, resulting in excessive contact between the acetabular rim and the femoral head; in this case, the bone mass or labrum at the acetabular rim will squeeze the femoral head during the movement of the hip joint, especially when the hip joint is abducted or rotated, resulting in damage to the labrum;

[0068] The combined type is a combination of the cam type and the pincer type, and the patient has both cam-type and pincer-type anatomical abnormalities.

[0069] In one embodiment, in the health condition text information:

[0070] General information: includes gender, age, sports enthusiast, whether smoking, etc.;

[0071] Medical history information: including hip intramuscular injection history, knee joint history, ankle joint history, other injuries, etc.;

[0072] Symptom information: including the location of hip pain, the type of hip pain, hip symptoms, the impact of symptoms on life, etc.;

[0073] Physical examination information: including whether the lower limbs are of equal length, left gluteal muscle atrophy, left gluteal scar, range of motion of the left hip joint, etc.

[0074] Figure 2 It is a schematic structural diagram of a hip impingement syndrome diagnosis model provided by an embodiment of the present application.

[0075] In one embodiment, the hip X-ray image and the health status text information are input into a preset hip impingement syndrome diagnosis model, and the hip impingement syndrome diagnosis result is output, including:

[0076] The text encoder converts the health status text information into a series of text feature vectors;

[0077] The image encoder converts the hip X-ray image into a series of image feature vectors;

[0078] The text feature vectors and the image feature vectors are fused to generate a feature matrix; wherein, each element represents the combination of the image feature vector and the text feature vector;

[0079] Based on the feature matrix, the hip impingement syndrome diagnosis result is output.

[0080] Figure 3 It is a schematic flowchart of the image encoder obtaining image feature vectors provided by an embodiment of the present application;

[0081] In one embodiment, the image encoder converts the hip X-ray image into a series of image feature vectors, including:

[0082] Feature extraction is performed on the hip X-ray image to obtain a feature map;

[0083] Several feature maps output after performing dilated convolution operations with different dilation sizes are element-wise added;

[0084] The sum result is further subjected to per-channel convolution to fuse features of different scales, and then through global average pooling, a fully connected layer to compress the number of feature channels, a fully connected layer to restore the number of channels, and non-linear activation of the Sigmoid function to obtain the image feature vectors.

[0085] Figure 4 It is a schematic flowchart of a convolutional neural network obtaining local features provided by an embodiment of the present application;

[0086] In one embodiment, the text encoder converts the health condition text information into a series of text feature vectors, including:

[0087] Converting the health condition text information into text feature word vectors using the ALBERT technique;

[0088] Using a text feature extraction network to extract features from the text feature word vectors to obtain optimal feature vectors;

[0089] Among them, the text feature extraction network includes a convolutional neural network with an attention mechanism and a bidirectional GRU neural network;

[0090] Respectively using the convolutional neural network and the bidirectional GRU network to extract the local feature vector and the global feature vector of the text, and finally fusing the two feature vectors to output the optimal feature vector.

[0091] Figure 5 It is a schematic flowchart of the process for the bidirectional GRU neural network to obtain global features provided by an embodiment of the present application;

[0092] In one embodiment, respectively using the convolutional neural network and the bidirectional GRU network to extract the local feature vector and the global feature vector of the text, and finally fusing the two feature vectors to output the optimal feature vector, including:

[0093] Using the mini-batch gradient descent method to input the text feature word vectors into the convolutional neural network. The convolutional layer extracts various different feature expressions of the feature words through convolutional kernels of different sizes. The pooling layer compresses and reduces the dimension of the features output by the convolutional layer, retains the main features of the feature words, and finally the pooling layer outputs the extracted features to the fully connected layer to obtain local features;

[0094] At the same time, using the mini-batch gradient descent method to also input the text feature word vectors into the bidirectional GRU neural network to obtain global features.

[0095] Figure 6 It is a schematic structural diagram of an intelligent diagnosis system for femoroacetabular impingement syndrome based on multi-modal data fusion provided by an embodiment of the present application.

[0096] The intelligent diagnosis system for femoroacetabular impingement syndrome based on multi-modal data fusion includes:

[0097] An image text information acquisition module 601, configured to acquire the hip X-ray image and health condition text information of the patient; among them, the health condition text information includes: general information of the patient, medical history information, symptom information, and physical examination information;

[0098] The hip impingement syndrome diagnosis module 602 is configured to input hip X-ray images and health condition text information into a preset hip impingement syndrome diagnosis model, and output a hip impingement syndrome diagnosis result;

[0099] Among them, the hip impingement syndrome diagnosis model extracts and fuses text features and image features for diagnosis.

[0100] Figure 7 The structural schematic diagram of the electronic device provided by the embodiment of the present application is shown.

[0101] The electronic device may include a processor 701 and a memory 702 storing computer program instructions.

[0102] Specifically, the above-mentioned processor 701 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0103] The memory 702 may include a mass storage for data or instructions. By way of example and not limitation, the memory 702 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disc, a magneto-optical disc, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In a suitable case, the memory 702 may include a removable or non-removable (or fixed) medium. In a suitable case, the memory 702 may be internal or external to the electronic device. In a specific embodiment, the memory 702 may be a non-volatile solid state memory.

[0104] In one embodiment, the memory 702 may be a read only memory (ROM). In one embodiment, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory or a combination of two or more of these.

[0105] The processor 701 reads and executes the computer program instructions stored in the memory 702 to implement any one of the above-mentioned hip impingement syndrome intelligent diagnosis methods based on multi-modal data fusion.

[0106] In one example, the electronic device may further include a communication interface 703 and a bus 710. Among them, as Figure 7As shown, a processor 701, a memory 702, and a communication interface 703 are connected via a bus 710 and communicate with each other.

[0107] The communication interface 703 is mainly used to implement communication between various modules, systems, units, and / or devices in the embodiments of the present application.

[0108] The bus 710 includes hardware, software, or both, and couples the components of the electronic device to each other. By way of example and not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses or a combination of two or more of these. In suitable cases, the bus 710 may include one or more buses. Although the embodiments of the present application describe and illustrate specific buses, the present application contemplates any suitable bus or interconnect.

[0109] In addition, in combination with the above-described method for intelligent diagnosis of femoroacetabular impingement syndrome based on multimodal data fusion in the embodiments, the embodiments of the present application may provide a computer-readable storage medium to implement. Computer program instructions are stored on the computer-readable storage medium; when the computer program instructions are executed by a processor, any one of the above-described methods for intelligent diagnosis of femoroacetabular impingement syndrome based on multimodal data fusion is implemented.

[0110] It should be clear that the present application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications, and additions, or change the order between steps after understanding the spirit of the present application.

[0111] The functional modules shown in the above-described structural block diagrams can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, and so on. When implemented in software, the elements of the present application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via a data signal carried in a carrier wave. A "machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical discs, hard disks, fiber optic media, radio frequency (RF) links, and so on. The code segment can be downloaded via a computer network such as the Internet, an intranet, and so on.

[0112] It should also be noted that the exemplary embodiments mentioned in the present application describe some methods or systems based on a series of steps or systems. However, the present application is not limited to the order of the above steps, that is, the steps can be executed in the order mentioned in the embodiments, can be different from the order in the embodiments, or several steps can be executed simultaneously.

[0113] Aspects of the present application have been described above with reference to the flowcharts and / or block diagrams of methods, systems, and computer program products according to embodiments of the present application. It should be understood that each block in the flowcharts and / or block diagrams, and the combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing system to produce a machine such that these instructions executed by the processor of the computer or other programmable data processing system enable the implementation of the functions / actions specified in one or more blocks of the flowcharts and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It should also be understood that each block in the block diagrams and / or flowcharts, and the combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by dedicated hardware that performs the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0114] As described above, this is only the specific implementation manner of the present application. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, modules, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein. It should be understood that the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should all be covered within the protection scope of the present application.

Claims

1. An intelligent diagnosis method for hip impingement syndrome based on multimodal data fusion, characterized in that: include: Obtaining the patient's hip joint X-ray image and health status text information; wherein the health status text information includes: the patient's general information, medical history information, symptom information, and physical examination information; Input the hip joint X-ray image and health status text information into the preset hip joint impingement syndrome diagnosis model, and output the hip joint impingement syndrome diagnosis result; Among them, the hip impingement syndrome diagnosis model extracts and fuses text features and image features for diagnosis.

2. The intelligent diagnosis method for hip impingement syndrome based on multimodal data fusion according to claim 1 is characterized in that: Hip impingement syndrome diagnosis results include: Cam type, clamp type or mixed type. Among them, the cam type occurs at the junction of the femoral head and the neck. Due to the abnormal shape of the femoral head, a non-spherical bulge is formed in the femoral head and neck area. When the hip joint moves, this bulge will come into abnormal contact with the edge of the acetabulum, especially when the hip joint is flexed or rotated, resulting in wear or damage to the articular cartilage and acetabular labrum. The pincer type is caused by excessive coverage of the acetabulum, resulting in excessive contact between the acetabular rim and the femoral head. In this case, the bone of the acetabular rim or labrum will squeeze the femoral head during hip movement, especially during abduction or rotation of the hip, causing damage to the labrum. The mixed type is a combination of the cam and pincer types, with the patient having both cam and pincer anatomical abnormalities.

3. The intelligent diagnosis method for hip impingement syndrome based on multimodal data fusion according to claim 1 is characterized in that: In the health status text information: General information: including gender, age, sports enthusiast, smoking status, etc.; Medical history information: including history of hip muscle injection, knee joint history, ankle joint history, other injuries, etc.; Symptom information: including the location of hip pain, type of hip pain, hip symptoms, and the impact of symptoms on life; Physical examination information: including whether the two lower limbs are of equal length, left hip muscle atrophy, left hip scar, left hip joint mobility, etc.

4. The intelligent diagnosis method for hip impingement syndrome based on multimodal data fusion according to claim 1 is characterized in that: The hip joint X-ray image and health status text information are input into the preset hip joint impingement syndrome diagnosis model, and the hip joint impingement syndrome diagnosis results are output, including: The text encoder converts the health status text information into a series of text feature vectors; The image encoder converts the hip joint X-ray image into a series of image feature vectors; The text feature vector and the image feature vector are fused to generate a feature matrix, wherein each element represents the combination of the image feature vector and the text feature vector; Based on the feature matrix, the diagnosis results of hip impingement syndrome are output.

5. The intelligent diagnosis method for hip impingement syndrome based on multimodal data fusion according to claim 4 is characterized in that: The image encoder converts the hip joint X-ray image into a series of image feature vectors, including: Extract features from hip joint X-ray images to obtain feature maps; Add the element-by-element sum of several feature maps outputted by dilated convolution operations with different dilated sizes; The summed result is convolved channel by channel to fuse features of different scales, and then goes through global average pooling, fully connected layer compression of the number of feature channels, fully connected layer restoration of the number of channels and Sigmoid function nonlinear activation to obtain the image feature vector.

6. The intelligent diagnosis method for hip impingement syndrome based on multimodal data fusion according to claim 5 is characterized in that: The text encoder converts the health status text information into a series of text feature vectors, including: Use ALBERT technology to convert health status text information into text feature word vectors; Use the text feature extraction network to extract the text feature word vector and obtain the optimal feature vector; Among them, the text feature extraction network includes a convolutional neural network with attention mechanism and a bidirectional GRU neural network; Convolutional neural network and bidirectional GRU network are used to extract local feature vectors and global feature vectors of text respectively, and finally the feature vectors of the two are fused to output the optimal feature vector.

7. The intelligent diagnosis method for hip impingement syndrome based on multimodal data fusion according to claim 6 is characterized in that: The local feature vector and global feature vector of the text are extracted using convolutional neural network and bidirectional GRU network respectively. Finally, the feature vectors of the two are fused to output the optimal feature vector, including: The text feature word vector is input into the convolutional neural network using the mini-batch gradient descent method. The convolution layer extracts multiple different feature expressions of the feature word through convolution kernels of different sizes. The pooling layer compresses and reduces the dimension of the output features of the convolution layer to retain the main features of the feature word. Finally, the pooling layer outputs the extracted features to the fully connected layer to obtain local features. At the same time, the text feature word vector is input into the bidirectional GRU neural network using the mini-batch gradient descent method to obtain the global features.

8. An intelligent diagnosis system for hip impingement syndrome based on multimodal data fusion, characterized in that: The system comprises: The image text information acquisition module is used to acquire the patient's hip joint X-ray image and health status text information; wherein the health status text information includes: the patient's general information, medical history information, symptom information, and physical examination information; Hip impingement syndrome diagnosis module, used to input hip joint X-ray images and health status text information into a preset hip impingement syndrome diagnosis model, and output hip impingement syndrome diagnosis results; Among them, the hip impingement syndrome diagnosis model extracts and fuses text features and image features for diagnosis.

9. An electronic device, characterized in that: The electronic device comprises: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the intelligent diagnosis method for hip impingement syndrome based on multimodal data fusion as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the intelligent diagnosis method for hip impingement syndrome based on multimodal data fusion as described in any one of claims 1-7.

Images