A case analysis method and system based on deep learning

Abstract

The present application provides a case analysis method and system based on deep learning. The method includes: obtaining a target data set containing image data and text data; extracting features from the image data in the target data set based on a pre-trained convolutional neural network; inputting the extracted feature vectors into a pre-trained natural language processing model for analyzing the text data in the target data set; generating new feature vectors by linear interpolation fusion based on the image feature vectors processed by the convolutional neural network and the text feature vectors processed by the natural language processing model; expanding the target data set based on the generated new feature vectors, and training a deep learning model based on the expanded target data set to obtain a trained deep learning model; inputting the case to be analyzed as input into the trained deep learning model, and obtaining the case analysis results output by the trained deep learning model.

Description

Technical Field

[0001] This application relates to the field of medical information processing, and specifically, to a case analysis method and system based on deep learning. Background Art

[0002] With the explosive growth of medical data in recent years, efficient processing and utilization of this data has become a research priority. Traditional case analysis relies primarily on physicians' experience and judgment. While intuitive and reliable, this approach is often inefficient when faced with large amounts of case data and prone to human error. Therefore, developing automated and intelligent case analysis methods is particularly important.

[0003] Existing case analysis methods mostly focus on a single type of data, such as image data or text data, and rarely utilize both types for comprehensive analysis. For example, for medical image analysis, models such as convolutional neural networks (CNNs) are often used for feature extraction; while for medical record analysis, natural language processing (NLP) techniques, such as pre-trained models like BERT, are more often used. However, using any single data type alone cannot fully reflect the entire case information. For example, relying solely on image data may fail to capture important textual information such as the medical history, while relying solely on text data may miss key visual cues.

[0004] Therefore, the present application provides a case analysis method and system based on deep learning to solve one of the above technical problems. Summary of the Invention

[0005] The purpose of this application is to provide a case analysis method and device based on deep learning, which can solve at least one of the technical problems mentioned above. The specific solution is as follows:

[0006] According to the specific implementation of the present application, in a first aspect, the present application provides a case analysis method based on deep learning, comprising:

[0007] Acquire a target data set containing image data and text data; wherein the image data and the text data are respectively obtained based on material cases; perform feature extraction on the image data in the target data set based on a pre-trained convolutional neural network; input the extracted feature vector into a pre-trained natural language processing model for analyzing the text data in the target data set; generate a new feature vector through linear interpolation fusion based on the image feature vector processed by the convolutional neural network and the text feature vector processed by the natural language processing model; expand the target data set based on the generated new feature vector, and train a deep learning model based on the expanded target data set to obtain a trained deep learning model; input the case to be analyzed as input into the trained deep learning model to obtain a case analysis result output by the trained deep learning model.

[0008] In one embodiment, the convolutional neural network is a pre-trained ResNet50 model.

[0009] In one embodiment, the ResNet50 model adopts the following settings: the preprocessing box of the ResNet50 model is configured with a normalization module, and the data input to the preprocessing box is processed by the normalization module and output as pixel values with normalized features, and the value range of the pixel values conforms to 0 to 1; the ResNet50 model is configured with a rotation module, a scaling module, a flipping module, and a randomization module, and the randomization module performs random and repeatable extraction and parallel arrangement in the rotation module, the scaling module, and the flipping module to obtain a module array; the module array is used to output image feature vectors with different degrees of generalization based on multiple different output heads matching the number of columns; the ResNet50 model is configured with an architecture fine-tuning module for increasing or decreasing the number of convolution layers according to the characteristics of the medical images of the case to be analyzed; wherein, the randomization module performs random and repeatable extraction and parallel arrangement based on the number of convolution layers to make the number of convolution layers consistent with the number of rows of the module array.

[0010] In one embodiment, the natural language processing model is a pre-trained BERT model.

[0011] In one embodiment, the BERT model adopts the following settings: the BERT model performs first-level pre-training based on general text data and second-level pre-training based on medical field text data; wherein the medical field text data is configured with a customized word segmenter based on a medical vocabulary; the BERT model is configured with a network layer dynamic connection module, which is used to query the matching fully connected layer and output layer according to the medical field when the medical field to which the case to be analyzed belongs is determined based on text recognition, and set the queried fully connected layer and output layer as layer connections according to the preconfigured connection method; the BERT model is configured with a loop termination module, which is used to terminate the loop when the loop round of the BERT model reaches the round threshold, and instruct the dynamic connection module to reset the layer connection according to the upper-level medical field of the medical field to which the case to be analyzed belongs that is currently identified; the BERT model is configured with a startup module, which is used to start the loop when the layer connection is set.

[0012] In one embodiment, the linear interpolation fusion includes: selecting the feature vectors of two different cases to be analyzed in the target data set as reference points; calculating the linear interpolation coefficient based on the distance between the two feature vectors; and generating a new feature vector based on the linear interpolation coefficient.

[0013] In one embodiment, the expansion of the target dataset includes: fusing feature maps of image data of the same dimension in the target dataset through improved linear interpolation, and reconstructing them back to image space through a decoder to generate a new medical image; and expanding the text data in the target dataset based on the new medical image processed by improved linear interpolation in combination with a pre-trained natural language processing model.

[0014] In one embodiment, the linear interpolation formula includes: F mixed =α·F A +β·F B Among them, F mixed Indicates the result after interpolation, F A 、F B They represent the feature maps of image data of the same dimension, α and β represent the basic interpolation parameters, and β is used to control the nonlinearity of the parameters.

[0015] In one embodiment, the linear interpolation formula includes: F mixed =α′·F A +β′·F B Among them, F mixed Indicates the result after interpolation, F A 、F BThey represent the feature maps of image data of the same dimension, α′ and β′ represent the optimized interpolation parameters, which are obtained using the following formula: ; ; Wherein, C represents the complexity factor of the case to be analyzed, T represents the weight of the type of case to be analyzed, D represents the time weight, and α and β represent the basic interpolation parameters respectively.

[0016] According to the specific implementation of the present application, in a second aspect, the present application provides a case analysis system based on deep learning, characterized in that it includes:

[0017] A data acquisition module is used to acquire a target data set containing image data and text data; wherein the image data and the text data are respectively data obtained based on material cases; an image feature extraction module is used to extract features of image data in the target data set based on a pre-trained convolutional neural network; a text feature analysis module is used to input the extracted feature vectors into a pre-trained natural language processing model for analyzing the text data in the target data set; a feature fusion module is used to generate new feature vectors through linear interpolation fusion based on the image feature vectors processed by the convolutional neural network and the text feature vectors processed by the natural language processing model; a data set expansion module is used to expand the target data set based on the generated new feature vectors, and train a deep learning model based on the expanded target data set to obtain a trained deep learning model; an input-output module is used to input the case to be analyzed as input into the trained deep learning model to obtain the case analysis results output by the trained deep learning model.

[0018] Compared with the prior art, the above-mentioned scheme of the embodiment of the present application has at least the following beneficial effects: The present application provides a case analysis method and system based on deep learning. The method integrates image data and text data. On the one hand, it uses a pre-trained convolutional neural network to extract image features, which can effectively capture key information in medical images. On the other hand, it uses a pre-trained natural language processing (NLP) model to analyze text data, automatically understand the semantics and contextual information in medical record text, and then analyze cases from multiple perspectives, improving the accuracy and completeness of case analysis. This method overcomes the limitations of single-modal information, enhances the model representation capability, and thus makes case analysis based on deep learning more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flowchart of a case analysis method based on deep learning according to an embodiment of the present application is shown;

[0020] Figure 2 A flow chart of a linear interpolation fusion method is shown;

[0021] Figure 3 A flow chart of a method for expanding a target data set is shown;

[0022] Figure 4 A module block diagram of a deep learning-based case analysis system according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0023] To make the objectives, technical solutions, and advantages of this application more clear, this application will be further described in detail below with reference to the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0024] The terms used in the examples of this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms "a," "the," and "the" used in the examples of this application and the appended claims are also intended to include plural forms, and unless the context clearly indicates otherwise, "a plurality" generally includes at least two.

[0025] It should be understood that the term "and / or" used herein is merely a description of the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the associated objects are in an "or" relationship.

[0026] It should be understood that although the terms first, second, third, etc. may be used to describe in the embodiments of the present application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, without departing from the scope of the embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first.

[0027] Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting." Similarly, depending on the context, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)."

[0028] It should also be noted that the terms "include," "comprises," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a product or device comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such product or device. In the absence of further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the product or device comprising the element.

[0029] It should be noted in particular that any symbols and / or numbers in the specification that are not marked in the accompanying drawings are not drawing marks.

[0030] The optional embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0031] The embodiment provided in this application is an embodiment of a case analysis method based on deep learning.

[0032] The following combination Figure 1 The embodiments of the present application are described in detail.

[0033] Figure 1 A flowchart of a case analysis method based on deep learning according to an embodiment of the present application is shown. Figure 1 As shown, the process includes the following steps S101 to S106.

[0034] In step S101 , a target data set including image data and text data is acquired.

[0035] Among them, the image data and text data are data obtained based on the material cases.

[0036] In step S102, feature extraction is performed on the image data in the target data set based on the pre-trained convolutional neural network.

[0037] In step S103 , the extracted feature vector is input into a pre-trained natural language processing model for analyzing the text data in the target dataset.

[0038] In step S104, a new feature vector is generated by linear interpolation fusion based on the image feature vector processed by the convolutional neural network and the text feature vector processed by the natural language processing model.

[0039] In step S105, the target data set is expanded based on the generated new feature vector, and a deep learning model is trained based on the expanded target data set to obtain a trained deep learning model.

[0040] In step S106, the case to be analyzed is input as input to the trained deep learning model to obtain the case analysis results output by the trained deep learning model.

[0041] This application provides a case analysis method and system based on deep learning. The method integrates image data and text data. On the one hand, it uses a pre-trained convolutional neural network to extract image features, effectively capturing key information in medical images. On the other hand, it uses a pre-trained natural language processing (NLP) model to analyze text data, automatically understanding the semantics and contextual information in medical record text, and then analyzing cases from multiple perspectives, improving the accuracy and completeness of case analysis. This method overcomes the limitations of single-modal information, enhances the model's representation capabilities, and thus enables case analysis based on deep learning to have higher precision.

[0042] In this application, the target dataset can be expanded based on the generated new feature vectors. Based on this approach, on the one hand, it can effectively alleviate the problem of data scarcity in the medical field, and on the other hand, the expanded dataset helps to improve the diversity and robustness of model training, thereby improving the generalization ability of the model.

[0043] In one embodiment, the convolutional neural network can be a pre-trained ResNet50 model.

[0044] As a feasible implementation, the ResNet50 model adopts the following ResNet50 model settings 1 to ResNet50 model settings 3.

[0045] ResNet50 model setup 1: The ResNet50 model's preprocessing block is configured with a normalization module. The data input to the preprocessing block is processed by the normalization module and output as pixel values with normalized features. The pixel value range is 0 to 1.

[0046] In this application, the purpose of setting one of the ResNet50 model is to standardize the input image data, ensuring that all image data has the same scale range and eliminating the effects caused by differences in brightness, contrast, and other aspects of the original image. This setting improves the stability and convergence speed of model training, preventing vanishing or exploding data gradients. It also allows the model to focus more on the structural information of the image rather than absolute intensity values, helping to improve the model's generalization ability.

[0047] ResNet50 model setup 2: The ResNet50 model is configured with a rotation module, a scaling module, a flipping module, and a randomization module. The randomization module randomly and repeatedly extracts data from the rotation module, scaling module, and flipping module and arranges them in parallel to form a module array. The module array is used to output image feature vectors with different generalization levels based on multiple output heads that match the number of columns.

[0048] In this application, the purpose of the second setting of the ResNet50 model is to increase the diversity of the data set by performing a series of enhancement operations on the image data, such as rotation, scaling, flipping, etc., to simulate more possible clinical scenarios, and to use a randomization module to select and arrange the enhancement operations to generate image feature vectors with different degrees of generalization. Based on this setting, the robustness of the model to image changes can be further improved, so that it can still maintain good performance when facing images of different angles, sizes or directions. In addition, the data output of multiple different output heads can play a role in expanding the data set, thereby reducing the risk of overfitting and improving the performance of the model on new data.

[0049] ResNet50 Model Setup 3: The ResNet50 model is configured with an architecture fine-tuning module, which increases or decreases the number of convolutional layers based on the characteristics of the medical images being analyzed. The randomization module randomly and repeatedly extracts layers based on the number of convolutional layers and arranges them in parallel to ensure that the number of layers matches the number of rows in the module array.

[0050] In this application, the purpose of setting three of the ResNet50 model is to adjust the depth of the ResNet50 model according to a specific type of medical image, such as increasing or decreasing the number of convolutional layers. In addition, in some embodiments, it also includes personalized settings of the convolutional layer structure. In this application, a randomization module is used to select the number of convolutional layers and ensure that it is consistent with the number of rows in the module array. Based on this setting, it is possible to meet the needs of adaptive data processing and analysis for cases in different medical fields. For example, for case images in some specific medical fields, richer detailed descriptions are required, and deeper networks are used to capture subtle features. Accordingly, for some specific medical fields that do not focus on the analysis of case images (for example, more analysis is required through text information), coarse data can be provided by reducing the number of convolutional layers and simplifying the convolutional layer structure, reducing unnecessary computational complexity, avoiding overfitting, and ensuring that the model can learn enough features.

[0051] For example, the natural language processing model can be a pre-trained BERT model.

[0052] As a feasible implementation, the BERT model may adopt the following BERT model settings 1 to BERT model settings 4.

[0053] BERT Model Setup 1: The BERT model undergoes primary pre-training on general text data and secondary pre-training on medical text data. The medical text data is configured with a customized word segmenter based on a medical vocabulary.

[0054] In this application, the purpose of setting up the BERT model is to perform a first-level pre-training on general text data so that the model can capture a wide range of language structures and patterns, and to perform a second-level pre-training on medical field text data so that the model can better understand the professional terms and context in the medical field. Based on this setting, on the one hand, the model's ability to understand medical text is enhanced, improving its applicability and accuracy in the medical field. On the other hand, the customized word segmenter can process medical-specific vocabulary more accurately, further improving model performance.

[0055] BERT model setting 2: The BERT model is equipped with a network layer dynamic connection module. When the medical field to be analyzed is determined based on text recognition, the BERT model queries for matching fully connected layers and output layers according to the medical field, and sets the queried fully connected layers and output layers as layer connections according to the preconfigured connection method.

[0056] In this application, the purpose of the second BERT model setting is to dynamically select appropriate fully connected layers and output layers based on the medical field to which the input text belongs, to adapt to the needs of different medical fields. This setting can increase the flexibility of the model, enabling it to better process text data from different medical fields, thereby achieving field-specific optimization and improving model accuracy.

[0057] BERT model setting three: The BERT model is configured with a loop termination module, which is used to terminate the loop when the BERT model's loop rounds reach the round threshold, and instruct the dynamic connection module to reset the layer connection based on the medical field of the parent medical field of the currently identified case to be analyzed.

[0058] In this application, the purpose of setting three of the BERT model is to automatically terminate the training process after the model's cyclic training reaches a predetermined number of rounds. If satisfactory results are not obtained in the current medical field, it will fall back to the previous medical field and reconfigure the network connection to continue training. Based on this setting, on the one hand, unnecessary training rounds can be avoided and computing resources can be saved. On the other hand, retraining in the previous medical field can help solve the problem of insufficient data in specific fields and improve the generalization ability of the model.

[0059] BERT model setup 4: The BERT model is configured with a startup module, which is used to start the loop after the layer connections are set up.

[0060] In this application, the purpose of BERT model setting four is to start the training or inference process of the model after the network layer connection setting is completed. It is a startup node prepared for BERT model setting two and BERT model setting three, so that the model can correctly enter the data processing flow after internal structural adjustment, ensuring the complete functionality of the model.

[0061] For example, the present application may perform linear interpolation fusion in the following manner.

[0062] Figure 2 A flow chart of a linear interpolation fusion method is shown in FIG. Figure 2 As shown, the process includes the following steps S201 to S203.

[0063] In step S201 , feature vectors of two different cases to be analyzed in the target data set are selected as reference points.

[0064] In step S202 , a linear interpolation coefficient is calculated according to the distance between two eigenvectors.

[0065] In step S203, a new feature vector is generated based on the linear interpolation coefficients.

[0066] For example, the present application may expand the target dataset in the following manner.

[0067] Figure 3 A flow chart of a method for expanding a target data set is shown. Figure 3 As shown, the process includes the following steps S301 to S302.

[0068] In step S301, the image data of the same dimension in the target data set is fused with feature maps through improved linear interpolation, and reconstructed back to the image space through the decoder to generate a new medical image.

[0069] In step S302 , the text data in the target dataset is expanded based on the new medical image processed by the improved linear interpolation and combined with the pre-trained natural language processing model.

[0070] In one embodiment, an improved linear interpolation formula may be used to perform linear interpolation calculations. The improved linear interpolation formula may be, for example, F mixed =α·F A +β·F B .

[0071] Among them, F mixed Indicates the result after interpolation, F A 、F BThey represent the feature maps of image data of the same dimension, α and β represent the basic interpolation parameters, and β is used to control the nonlinearity of the parameters. When β=0, the improved linear interpolation degenerates to the standard linear interpolation. When β>0, the interpolation result will be biased towards F in a nonlinear way according to the value of α. A or F B .

[0072] This application can use the above improved linear interpolation formula for linear interpolation calculation, which allows the model to have greater flexibility in fusing image features and can adjust F according to specific circumstances. A and F B When β > 0, the bias of the interpolation results can be controlled by adjusting the value of α, which improves sensitivity when dealing with complex or specific cases. Furthermore, by introducing nonlinear control parameters, the model can better adapt to the characteristics of different cases, improving the effectiveness of feature fusion and the accuracy of case analysis.

[0073] In another embodiment, an optimized linear interpolation formula may be used to perform linear interpolation calculation. The optimized linear interpolation formula may be, for example, F mixed =α′·F A +β′·F B .

[0074] Among them, F mixed Indicates the result after interpolation, F A 、F B They represent the feature maps of image data of the same dimension, and α′ and β′ represent the optimized interpolation parameters. Among them, α′ is calculated by the formula We get β′ by the formula Where C represents the complexity factor of the case to be analyzed, T represents the weight of the case type to be analyzed, D represents the time weight, and α and β represent the basic interpolation parameters.

[0075] Among them, when C=0 and D=0, the improved linear interpolation degenerates into the standard linear interpolation. When C≠0 or D≠0, the interpolation result will be biased towards F in a nonlinear way according to the values of C and D. A or F B .

[0076] This application can use the above optimized linear interpolation formula for linear interpolation calculation, which can automatically adjust the interpolation parameters according to the complexity, type and time factors of the case to be analyzed, so that the interpolation process is more in line with the actual situation. When C≠0 or D≠0, the interpolation result will be biased towards F in a nonlinear manner according to the values of C and D. A or F BThis means that the model can be adjusted according to the specific attributes of the case, improving the relevance and accuracy of the analysis results. By optimizing the interpolation parameters, the model can better handle diverse case data, improve the quality of feature fusion, and thus improve the overall effectiveness of case analysis.

[0077] In the present application, the linear interpolation step can be performed based on one of the above-mentioned linear interpolation formulas. Among them, compared with the improved linear interpolation formula, the optimized linear interpolation formula has higher accuracy and higher computational cost. Therefore, in actual use, the two can be selected based on the preference of case analysis, such as selecting the improved linear interpolation formula to ensure timeliness, or selecting the optimized linear interpolation formula to ensure analysis accuracy.

[0078] The present application also provides a system embodiment that is consistent with the above embodiment, which is used to implement the method steps of the above embodiment. The explanation based on the same name meaning is the same as the above embodiment, and has the same technical effect as the above embodiment, which will not be repeated here.

[0079] like Figure 4 As shown, the present application provides a case analysis system 400 based on deep learning, including a data acquisition module 401, an image feature extraction module 402, a text feature analysis module 403, a feature fusion module 404, a data set expansion module 405, and an input and output module 406.

[0080] The data acquisition module 401 is used to obtain a target data set containing image data and text data. The image data and text data are data obtained based on the material cases. The image feature extraction module 402 is used to extract features of the image data in the target data set based on the pre-trained convolutional neural network. The text feature analysis module 403 is used to input the extracted feature vectors into the pre-trained natural language processing model for analyzing the text data in the target data set. The feature fusion module 404 is used to generate new feature vectors by linear interpolation fusion based on the image feature vectors processed by the convolutional neural network and the text feature vectors processed by the natural language processing model. The data set expansion module 405 is used to expand the target data set based on the generated new feature vectors, and train the deep learning model based on the expanded target data set to obtain the trained deep learning model. The input-output module 406 is used to input the case to be analyzed as input into the trained deep learning model to obtain the case analysis results output by the trained deep learning model.

[0081] In one embodiment, the convolutional neural network is a pre-trained ResNet50 model.

[0082] In one embodiment, the ResNet50 model adopts the following settings: the preprocessing box of the ResNet50 model is configured with a normalization module, and the data input to the preprocessing box is processed by the normalization module and output as pixel values with normalized features, and the value range of the pixel values conforms to 0 to 1. The ResNet50 model is configured with a rotation module, a scaling module, a flipping module, and a randomization module. The randomization module performs random and repeatable extraction and parallel arrangement in the rotation module, the scaling module, and the flipping module to obtain a module array. The module array is used to output image feature vectors with different degrees of generalization based on multiple different output heads that match the number of columns. The ResNet50 model is configured with an architecture fine-tuning module, which is used to increase or decrease the number of convolution layers according to the characteristics of the medical images of the case to be analyzed. Among them, the randomization module performs random and repeatable extraction and parallel arrangement based on the number of convolution layers to make the number of convolution layers consistent with the number of rows of the module array.

[0083] In one embodiment, the natural language processing model is a pre-trained BERT model.

[0084] In one embodiment, the BERT model adopts the following settings: the BERT model performs first-level pre-training based on general text data and second-level pre-training based on medical field text data. Among them, the medical field text data is configured with a customized word segmenter based on a medical vocabulary. The BERT model is configured with a network layer dynamic connection module, which is used to query the matching fully connected layer and output layer according to the medical field when the medical field to be analyzed is determined based on text recognition, and set the queried fully connected layer and output layer as layer connections according to the preconfigured connection method. The BERT model is configured with a loop termination module, which is used to terminate the loop when the loop round of the BERT model reaches the round threshold, and instruct the dynamic connection module to reset the layer connection according to the medical field of the upper level of the medical field to which the currently identified case to be analyzed belongs. The BERT model is configured with a startup module, which is used to start the loop when the layer connection is set.

[0085] In one embodiment, feature fusion module 404 performs linear interpolation fusion in the following manner: Feature vectors of two different cases to be analyzed in the target dataset are selected as reference points. A linear interpolation coefficient is calculated based on the distance between the two feature vectors. A new feature vector is generated based on the linear interpolation coefficient.

[0086] In one embodiment, feature fusion module 404 augments the target dataset by fusing feature maps of image data of the same dimension within the target dataset using improved linear interpolation, reconstructing the image data back into image space using a decoder to generate a new medical image. Based on the new medical image processed by improved linear interpolation and combined with a pre-trained natural language processing model, the text data within the target dataset is augmented.

[0087] In one embodiment, the linear interpolation formula includes: F mixed =α·F A +β·F B Among them, F mixed Indicates the result after interpolation, F A 、F B They represent the feature maps of image data of the same dimension, α and β represent the basic interpolation parameters, and β is used to control the nonlinearity of the parameters. When β=0, the improved linear interpolation degenerates to the standard linear interpolation. When β>0, the interpolation result will be biased towards F in a nonlinear way according to the value of α. A or F B .

[0088] In one embodiment, the linear interpolation formula includes: F mixed =α′·F A +β′·F B Among them, F mixed Indicates the result after interpolation, F A 、F B They represent the feature maps of image data of the same dimension, α′ and β′ represent the optimized interpolation parameters, which are obtained using the following formula: . Where C represents the complexity factor of the case to be analyzed, T represents the weight of the case type to be analyzed, D represents the time weight, and α and β represent the basic interpolation parameters.

[0089] Regarding the system in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0090] Although operations are described in a particular order in the drawings, this should not be understood as requiring that the operations be performed in the particular order shown or in serial order, or that all shown operations be performed to obtain the desired results. In certain circumstances, multitasking and parallel processing may be advantageous.

[0091] The methods and apparatus of the present application can be implemented using standard programming techniques, utilizing rule-based logic or other logic to implement the various method steps. It should also be noted that the terms "means" and "module" as used herein and in the claims are intended to include implementations based on one or more lines of software code and / or hardware implementations and / or devices for receiving input.

[0092] Any steps, operations or procedures described herein may be performed or implemented based on one or more hardware or software modules, either alone or in combination with other devices. In one embodiment, the software module is implemented based on a computer program product comprising a computer-readable medium containing computer program code, which can be executed by a computer processor to perform any or all of the steps, operations or procedures described.

[0093] The foregoing description of the implementation of the present application has been provided for purposes of illustration and description. The foregoing description is not intended to be exhaustive or to limit the present application to the precise form disclosed, and various variations and modifications are possible in accordance with the above teachings or may result from the practice of the present application. These embodiments have been selected and described in order to illustrate the principles of the present application and its practical application, so as to enable those skilled in the art to utilize the present application in various embodiments and modifications as appropriate for the particular use contemplated.

[0094] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0095] It is further understood that, unless otherwise specified, “connection” includes a direct connection where there are no other components between the two elements, and also includes an indirect connection where there are other elements between the two elements.

[0096] It should be further understood that although operations are described in a particular order in the drawings in the embodiments of the present application, this should not be construed as requiring that these operations be performed in the particular order shown or in a serial order, or that all of the illustrated operations be performed to obtain the desired results. In certain circumstances, multitasking and parallel processing may be advantageous.

[0097] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. This application is intended to encompass any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the field of the present application that are not disclosed herein. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the scope of claims below.

[0098] It should be understood that the present application is not limited to the precise structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the scope of the appended claims.

[0099] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A case analysis method based on deep learning, characterized in that: include: Acquire a target data set comprising image data and text data; wherein the image data and the text data are respectively data obtained based on a material case; Extract features from image data in the target dataset based on pre-trained convolutional neural networks; The extracted feature vectors are input into a pre-trained natural language processing model to analyze the text data in the target dataset; Generate a new feature vector by linear interpolation fusion based on the image feature vector processed by the convolutional neural network and the text feature vector processed by the natural language processing model; Expanding the target data set based on the generated new feature vector, and training a deep learning model based on the expanded target data set to obtain a trained deep learning model; Inputting the case to be analyzed as input into the trained deep learning model to obtain the case analysis result output by the trained deep learning model; The convolutional neural network is a pre-trained ResNet50 model, which uses the following settings: The preprocessing block of the ResNet50 model is configured with a normalization module. The data input to the preprocessing block is processed by the normalization module and output as pixel values with normalized features. The value range of the pixel values conforms to 0 to 1. The ResNet50 model is configured with a rotation module, a scaling module, a flipping module, and a randomization module. The randomization module randomly and repeatedly extracts and arranges the rotation module, the scaling module, and the flipping module in parallel to obtain a module array. The module array is used to output image feature vectors of different generalization levels based on multiple different output heads that match the number of columns. The ResNet50 model is configured with an architecture fine-tuning module for increasing or decreasing the number of convolutional layers based on the characteristics of the medical image of the case to be analyzed; wherein the randomization module randomly and repeatedly extracts and arranges the convolutional layers in parallel based on the number of convolutional layers so that the number of convolutional layers is consistent with the number of rows of the module array; The natural language processing model is a pre-trained BERT model, which uses the following settings: The BERT model is pre-trained on general text data at the first level and on medical text data at the second level; wherein the medical text data is configured with a customized word segmenter based on a medical vocabulary; The BERT model is configured with a network layer dynamic connection module for, when the medical field to which the case to be analyzed belongs is determined based on text recognition, searching for matching fully connected layers and output layers according to the medical field, and setting the found fully connected layers and output layers as layer connections according to a preconfigured connection method; The BERT model is configured with a loop termination module, which is used to terminate the loop when the loop round of the BERT model reaches a round threshold, and instruct the dynamic connection module to reset the layer connection according to the upper-level medical field of the medical field to which the currently identified case to be analyzed belongs; The BERT model is configured with a startup module for starting a loop when the layer connections are set.

2. The method according to claim 1, characterized in that The linear interpolation fusion includes: Selecting the characteristic vectors of two different cases to be analyzed in the target data set as reference points; Calculating a linear interpolation coefficient according to the distance between the two eigenvectors; Based on the linear interpolation coefficients, a new feature vector is generated.

3. The method according to claim 2, characterized in that The expanding the target data set includes: The image data of the same dimension in the target data set are fused with feature maps by improved linear interpolation, and reconstructed back into image space through a decoder to generate a new medical image; According to the new medical image processed by the improved linear interpolation and combined with the pre-trained natural language processing model, the text data in the target data set is expanded.

4. The method according to claim 3, characterized in that The linear interpolation formula includes: F mixed =α·F A +β·F B ; Among them, F mixed Indicates the result after interpolation, F A 、F B They represent the feature maps of image data of the same dimension, α and β represent the basic interpolation parameters, and β is used to control the nonlinearity of the parameters.

5. The method according to claim 3, characterized in that The linear interpolation formula includes: F mixed =α′·F A +β′·F B ; Among them, F mixed Indicates the result after interpolation, F A 、F B They represent the feature maps of image data of the same dimension, α′ and β′ represent the optimized interpolation parameters, which are obtained using the following formula: ； ； Wherein, C represents the complexity factor of the case to be analyzed, T represents the type weight of the case to be analyzed, D represents the time weight, and α and β represent basic interpolation parameters respectively.

6. A case analysis system based on deep learning, characterized in that: include: A data acquisition module, configured to acquire a target data set comprising image data and text data; wherein the image data and the text data are respectively data obtained based on a material case; Image feature extraction module, used to extract features from image data in the target dataset based on a pre-trained convolutional neural network; The text feature analysis module is used to input the extracted feature vectors into the pre-trained natural language processing model to analyze the text data in the target dataset; A feature fusion module is used to generate a new feature vector by linear interpolation fusion based on the image feature vector processed by the convolutional neural network and the text feature vector processed by the natural language processing model; A data set expansion module is used to expand the target data set based on the generated new feature vectors, and train a deep learning model based on the expanded target data set to obtain a trained deep learning model; An input-output module, configured to input the case to be analyzed as input into the trained deep learning model, and obtain the case analysis results output by the trained deep learning model; The convolutional neural network is a pre-trained ResNet50 model, which uses the following settings: The preprocessing block of the ResNet50 model is configured with a normalization module. The data input to the preprocessing block is processed by the normalization module and output as pixel values with normalized features. The value range of the pixel values conforms to 0 to 1. The ResNet50 model is configured with a rotation module, a scaling module, a flipping module, and a randomization module. The randomization module randomly and repeatedly extracts and arranges the rotation module, the scaling module, and the flipping module in parallel to obtain a module array. The module array is used to output image feature vectors of different generalization levels based on multiple different output heads that match the number of columns. The ResNet50 model is configured with an architecture fine-tuning module for increasing or decreasing the number of convolutional layers based on the characteristics of the medical image of the case to be analyzed; wherein the randomization module randomly and repeatedly extracts and arranges the convolutional layers in parallel based on the number of convolutional layers so that the number of convolutional layers is consistent with the number of rows of the module array; The natural language processing model is a pre-trained BERT model, which uses the following settings: The BERT model is pre-trained on general text data at the first level and on medical text data at the second level; wherein the medical text data is configured with a customized word segmenter based on a medical vocabulary; The BERT model is configured with a network layer dynamic connection module for, when the medical field to which the case to be analyzed belongs is determined based on text recognition, searching for matching fully connected layers and output layers according to the medical field, and setting the found fully connected layers and output layers as layer connections according to a preconfigured connection method; The BERT model is configured with a loop termination module, which is used to terminate the loop when the loop round of the BERT model reaches a round threshold, and instruct the dynamic connection module to reset the layer connection according to the upper-level medical field of the medical field to which the currently identified case to be analyzed belongs; The BERT model is configured with a startup module for starting a loop when the layer connections are set.

Images