Medical data analysis method and system based on image data

Abstract

The invention particularly relates to a medical data analysis method and system based on image data. The method comprises the steps of obtaining a current to-be-recognized image of an analysis object and a description text corresponding to the to-be-recognized image; the to-be-recognized image comprises a drawn datura painting image based on a standard structural formula; the description text comprises a description text of the to-be-recognized image; extracting image feature data corresponding to the to-be-recognized image, describing text feature data corresponding to the text, and configuring the text feature data as input data; the image feature data is used for describing color features of the to-be-recognized image; and inputting the input data into a trained data analysis model, performing feature analysis on the image feature data by using the image analysis sub-model, performing feature analysis on the text feature data by using the text analysis sub-model, and outputting a schizophrenia classification probability result corresponding to an analysis object based on a feature analysis result. According to the scheme, the datura stramonium is used for drawing the image, so that the accuracy of schizophrenia analysis results is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a medical data analysis method and system based on image data, a storage medium, a computer program product, and an electronic device. Background Art

[0002] Schizophrenia is a serious mental illness. Auxiliary examination methods, as the basis for the diagnosis of schizophrenia, are helpful for the treatment of schizophrenia and improving the prognosis. In related technologies, when diagnosing schizophrenia clinically, the diagnostic criteria are generally based on symptomology, such as the International Classification of Diseases (ICD-10), the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5), etc. In addition, auxiliary diagnosis of schizophrenia can also be achieved based on technical equipment, such as neuropsychological tests (positive and negative symptom scales), eye tracking examinations, and electroencephalogram tests. However, there are certain problems in the clinical diagnosis of schizophrenia. For example, the diagnostic criteria based on symptomology are relatively subjective, lack objective laboratory evidence to support the diagnosis, and there are differences between different diagnostic criteria, resulting in consistency deviations in clinical applications. In addition, when using auxiliary means for diagnosis, there is a great deal of heterogeneity in the diagnostic results, which can easily lead to inaccurate diagnostic results.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0004] The present invention provides a medical data analysis method and device based on image data, a storage medium, a computer program product, and an electronic device, which can improve the accuracy of data analysis results for schizophrenia and effectively overcome the defects in the prior art.

[0005] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0006] According to a first aspect of the present disclosure, a medical data analysis method based on image data is provided, the method comprising: Acquire the current image to be identified of the analysis object, and the description text corresponding to the image to be identified; wherein the image to be identified includes a mandala painting image based on a standard structure formula that has been drawn; and the description text includes the description text of the image to be identified generated by the analysis object based on a preset text template; Extracting image feature data corresponding to the image to be identified and text feature data corresponding to the description text, and configuring them as input data; wherein the image feature data is used to describe the color features of the image to be identified; The input data is input into a trained data analysis model, the image feature data is feature parsed using an image analysis sub-model, the text feature data is feature parsed using a text analysis sub-model, and a schizophrenia classification probability result corresponding to the analysis object is output based on the feature parsing result.

[0007] In some exemplary embodiments, the image feature data includes: any one or a combination of any two of: color type features, color component features, color quantity features, color distribution features, color symmetry area features, and image features outside the standard structure.

[0008] In some exemplary embodiments, after obtaining the current image to be identified of the analysis object, the method further includes: The image to be identified is preprocessed to obtain a preprocessed image to be identified; wherein the preprocessing includes image size adjustment and image definition adjustment.

[0009] In some exemplary embodiments, the input data further includes: identity feature data and historical case data of the target subject; The method further comprises: Perform text recognition on the identity feature data and historical case data, and extract target keywords corresponding to preset feature fields; and configure the target keywords as input parameters of the text analysis sub-model.

[0010] In some exemplary embodiments, the image to be identified includes multiple frames of painted mandala painting images, each mandala painting image is based on a different standard structural formula; The method further comprises: Utilizing the data analysis model to analyze and process each frame of the drawn mandala painting image and the corresponding description text, so as to obtain a plurality of corresponding classification probability results; A probability curve corresponding to the analysis object is constructed based on the classification probability result, and a comprehensive classification probability result is determined based on the probability interval.

[0011] In some exemplary embodiments, the image analysis sub-model includes: a convolutional neural network, a first fully connected layer, a second fully connected layer, a normalization layer, and a third fully connected layer arranged in sequence; The text analysis sub-model includes: a recurrent neural network, a first fully connected layer, a second fully connected layer, a normalization layer, and a third fully connected layer, which are arranged in sequence.

[0012] In some exemplary embodiments, the method further comprises: pre-training the data analysis model, comprising: Collect user case data, build an initial sample data set based on the case data; and mark the initial sample data; wherein the initial sample data includes a drawn mandala painting image based on a standard structure and a corresponding description text; Extracting corresponding image feature data and text feature data from the sample data, and constructing a target sample data set; The target sample data set is used to perform forward feedback training on the initial data analysis model to obtain the trained data analysis model; wherein the data analysis model includes an image analysis sub-model for performing feature analysis on image feature data, and a text analysis sub-model for performing feature analysis on text feature data.

[0013] According to a second aspect of the present disclosure, a medical data analysis system based on image data is provided, the system comprising: An image acquisition module, used to acquire the current image to be identified of the analysis object, and the description text corresponding to the image to be identified; wherein the image to be identified includes a mandala painting image based on a standard structural formula that has been drawn; and the description text includes the description text of the image to be identified generated by the analysis object based on a preset text template; A feature extraction module, used to extract image feature data corresponding to the image to be identified and text feature data corresponding to the description text, and configure them as input data; wherein the image feature data is used to describe the color features of the image to be identified; The feature analysis module is used to input the input data into a trained data analysis model, perform feature analysis on the image feature data using an image analysis sub-model, perform feature analysis on the text feature data using a text analysis sub-model, and output a schizophrenia classification probability result corresponding to the analysis object based on the feature analysis result.

[0014] According to a third aspect of the present disclosure, a storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned medical data analysis method based on image data is implemented.

[0015] According to a fourth aspect of the present disclosure, there is provided an electronic device, including: Processor; and A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to implement the above-mentioned medical data analysis method based on image data by executing the executable instructions.

[0016] According to a fifth aspect of the present disclosure, a computer program product is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned medical data analysis method based on image data is implemented.

[0017] The medical data analysis method based on image data provided by the embodiments of the present disclosure uses a mandala painting with a standard structure, takes a mandala painting image based on a standard structure of a target object, and a corresponding description text as input data, and parses the image features in the mandala painting image in combination with the text features, thereby mapping the image and text to the same feature space, obtaining an accurate assessment of the patient's mental state, and thereby improving the recognition accuracy of schizophrenia patients.

[0018] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0020] Figure 1 A schematic diagram schematically illustrating an exemplary system architecture of the technical solution of an exemplary embodiment of the present disclosure; Figure 2 A schematic diagram schematically illustrates a medical data analysis method based on image data according to an exemplary embodiment of the present disclosure; Figure 3 A schematic diagram schematically illustrates a mandala painting template structure according to an exemplary embodiment of the present disclosure; Figure 4 A schematic diagram schematically illustrates another mandala painting template structure in an exemplary embodiment of the present disclosure; Figure 5 A schematic diagram schematically illustrates another mandala painting template structure in an exemplary embodiment of the present disclosure; Figure 6 A schematic diagram schematically illustrates an electrocardiogram signal classification device according to an exemplary embodiment of the present disclosure; Figure 7 The diagram schematically shows the components of an electronic device in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0021] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0022] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0023] In view of the shortcomings and deficiencies of the prior art, this example implementation provides an ECG signal classification method and an ECG signal classification model training method to solve the problems of large number of parameters and low degree of parallelization of traditional ECG classification models.

[0024] Figure 1 A schematic diagram showing an exemplary system architecture to which the technical solution of an embodiment of the present invention can be applied.

[0025] like Figure 1 As shown, the system architecture may include an intelligent terminal device 101, a server 103, and a data storage system 104. The intelligent terminal device 101 may be a computer, a mobile phone, a tablet computer, etc. The intelligent terminal device 101 may communicate and transmit data with the server 103 through a communication network 102. The server 103 is connected to the data storage system 104. The communication network 102 may include various connection types, such as a wired communication link, a wireless communication link, etc. Among them, the server 103 may be a server cluster composed of multiple servers. The data storage system 104 may be a database.

[0026] In one embodiment of the present invention, the user may initiate an analysis request for one or more patients on the smart terminal device 101. After receiving the analysis request from the user terminal device, the server may obtain the patient's corresponding data from the database and perform data analysis to generate corresponding schizophrenia classification probability results, and then feed the data analysis results back to the smart terminal device 101.

[0027] It should be noted that the method provided in the embodiment of the present invention can be executed by the server 103. However, in other embodiments of the present invention, the smart terminal device 101 and the server 103 can also cooperate to implement the medical data analysis method based on image data provided in the embodiment of the present invention.

[0028] Exemplary, reference Figure 2 As shown, the medical data analysis method based on image data may specifically include: Step S11, obtaining the current image to be identified of the analysis object and the description text corresponding to the image to be identified; wherein the image to be identified includes a mandala painting image based on a standard structure formula that has been drawn; and the description text includes the description text of the image to be identified generated by the analysis object based on a preset text template; Step S12, extracting image feature data corresponding to the image to be identified and text feature data corresponding to the description text, and configuring them as input data; wherein the image feature data is used to describe the color features of the image to be identified; Step S13, inputting the input data into the trained data analysis model, performing feature analysis on the image feature data using the image analysis sub-model, performing feature analysis on the text feature data using the text analysis sub-model, and outputting the schizophrenia classification probability result corresponding to the analysis object based on the feature analysis result.

[0029] The medical data analysis method based on image data provided by the embodiment of the present disclosure automatically recognizes and analyzes mandala images drawn by patients and the corresponding descriptive texts through a training model, thereby realizing cross-modal analysis of images and corresponding text descriptions, improving the recognition accuracy and analysis speed of schizophrenia, and thus helping clinical physicians to assist in the diagnosis of the disease.

[0030] Below, each step of the medical data analysis method based on image data in this example implementation will be described in more detail with reference to the accompanying drawings and embodiments.

[0031] In step S11, the current image to be identified of the analysis object and the description text corresponding to the image to be identified are obtained; wherein the image to be identified includes a mandala painting image that has been drawn based on a standard structure; and the description text includes the description text of the image to be identified generated by the analysis object based on a preset text template.

[0032] In this example implementation, the above-mentioned image to be identified is the corresponding electronic image data collected after the patient fills in the standard structural mandala painting template according to the drawing guidance information. Among them, the above-mentioned drawing guidance information may include the predefined number and type of drawing colors, as well as drawing guidance statements. For example, the predefined number and type of drawing colors may be 12-color standard colored pens (including: red, orange, light yellow, yellow, grass green, green, blue, purple, magenta, burnt ochre, white, black). The drawing guidance statement may include: coloring on paper, no requirements. For example, a piece of paper may be provided, a drawing guidance statement may be written on the top of the paper, and a mandala painting image template based on the standard structural formula may be arranged below the paper. After the patient completes coloring the painting template, a photo of the corresponding paper is taken to obtain the corresponding image to be identified. Among them, the standard structural mandala painting template may be used as follows: Figure 3-Figure 5 As shown in the style. Figure 3 As shown in the figure, this paradigm is one of the basic paradigms of structural mandala. It is characterized by embracing in form and protection in content. It can show the inner "self" function of the painter. In addition, this picture is representative in the type of protection structure. Other types of paradigms, such as differentiation structure, cohesion structure, transcendence structure, etc., can only be carried out on the basis of protection structure, which is not conducive to the painter's first drawing. Another reason for using this paradigm is that it is assumed that the "self" of schizophrenia patients is damaged during the acute stage of the disease. Through this paradigm, the patient's "damaged self" can be concretely displayed (in this scheme, the damaged self refers to the drawn graphics, images and explanations) to achieve the purpose of identification and auxiliary diagnosis.

[0033] In some exemplary embodiments, a plurality of templates with different patterns may be pre-configured, and the medical staff may randomly select one of the templates and assign it to the current patient for the patient to fill in.

[0034] In addition, a corresponding image description template can be provided for the patient. The image description template includes multiple questions. After the patient completes the painting of the standard structured mandala painting template, the image description template can be provided to the patient, and the patient can fill in the image description template, answer the corresponding questions, and configure the corresponding answers as description text; or, the patient's answers to the questions in the image description template can be recorded by recording and configured as description text. For example, the questions in the image description template can be: What is the shape of the painting? Why is it painted like this? What is the image of the painting content? What is the meaning of painting this color in the center or on the edge? Why paint it in this place? What does the whole / part look like, why is it painted like this, and give this painting a name, why give it this name? (The blue font can be used as an extension to continue asking questions, which belongs to a "why" or "arrow down" technique. In theory, the more the patient's thinking content is exposed, the easier it is for schizophrenia patients to expose their weird thinking problems. It just means that more text description analysis modules will be temporarily added to the current analysis system. Please consider whether to expand the text processing function of the system here) For example, in a quiet and comfortable ward, patients can be provided with 12-color standard crayons, standard structured mandala painting templates, and image description templates, and the patients can fill in the templates. After the patients have completed the filling, they are scanned and processed to obtain the corresponding electronic image data, thereby reducing unnecessary color differences and avoiding image information loss, and configuring it as an image to be recognized. Alternatively, it can also be image data and text data filled in by a group of people obtained in the mental health screening of students and employees.

[0035] In this example implementation, after obtaining the current image to be identified of the analysis object, the method further includes: preprocessing the image to be identified to obtain a preprocessed image to be identified; wherein the preprocessing includes image size adjustment and image clarity adjustment.

[0036] Specifically, for the currently acquired image to be identified, its size can be identified and adjusted, and the image can be modified to a pre-defined image size to complete the image size determination, so as to facilitate subsequent feature recognition and extraction of the image, as well as batch processing of the image. Figure 5 As shown; the image size can be configured to 224*224 and other sizes.

[0037] For example, for the image description template filled in by the patient, the corresponding text image can be collected, and then the text content in the image can be extracted using an optical character recognition algorithm to be configured as the description text corresponding to the image to be recognized. Alternatively, for the collected audio data, the corresponding text content can be extracted using an audio conversion tool to be configured as the description text.

[0038] In step S12, the image feature data corresponding to the image to be identified and the text feature data corresponding to the description text are extracted and configured as input data; wherein the image feature data is used to describe the color features of the image to be identified.

[0039] In this example implementation, the current image to be recognized and the corresponding description text may be input into a trained model to extract image features and extract text feature data.

[0040] Exemplarily, the image feature data includes: any one or a combination of any two of: color type features, color component features, color quantity features, color distribution features, color symmetry area features, and image features outside the standard structure.

[0041] Among them, the color type feature can be used to indicate the color type selected by the patient from the specified 12 colors when filling in the mandala image, such as the cool color type and the warm color type. The color component feature can be used to indicate the specific color used by the patient from the specified 12 colors when filling in the mandala image; for example, the patient currently uses red, black; or blue, green, white; etc. The color quantity feature can be used to indicate the specific number of colors used by the patient from the specified 12 colors when filling in the mandala image, and the color quantity belongs to a preset specific color quantity interval; for example, the configuration color quantity interval includes a first interval of 1-4, a second interval of 5-8, and a third interval of 9-12. The color symmetry area feature can be used to describe the number of color symmetry areas in the mandala image, the colors used, and the position and angle of the symmetry area in the image.

[0042] In some exemplary embodiments, the pattern types in the image template may also be classified and marked, such as patterns with sharp corners and rounded patterns. When recognizing image features, different types of images in the template image and corresponding color types may also be marked separately. For example, a pattern with sharp corners filled with dark or cool colors such as black, gray, and brown; or a circular pattern filled with warm colors, etc.

[0043] In addition, the image features outside the standard structure can be used to describe the image content outside the standard structure drawn by the patient on the paper when drawing; wherein the image content outside the standard structure can include images and / or text. For example, the image features outside the standard structure can be irregular images, regular symmetrical images, or animal and plant images, etc. The text can be the text content written by the patient on the image paper.

[0044] In this example implementation, the input data also includes: identity feature data and historical case data of the target object. The method also includes: performing text recognition on the identity feature data and historical case data, and extracting target keywords corresponding to preset feature fields; and configuring the target keywords as input parameters of the text analysis sub-model.

[0045] Specifically, the above-mentioned preset feature fields can be the patient's age, gender, work characteristics, historical medical examination results, etc. By using these historical data as text data for model analysis, the changing process of the patient's medical experience reflected in the historical data can be referred to, thereby improving the accuracy of the current analysis results.

[0046] In step S13, the input data is input into the trained data analysis model, the image feature data is feature parsed using the image analysis sub-model, the text feature data is feature parsed using the text analysis sub-model, and the schizophrenia classification probability result corresponding to the analysis object is output based on the feature parsing result.

[0047] In this example implementation, a data analysis model may be pre-trained to perform feature recognition and analysis on input data, wherein the data analysis model may include an image analysis sub-model and a text analysis sub-model.

[0048] Exemplarily, the image analysis submodel includes: a convolutional neural network layer, a first fully connected layer, a second fully connected layer, a normalization layer, and a third fully connected layer arranged in sequence.

[0049] Among them, the convolutional neural network layer may include an input layer, multiple convolutional layers, a pooling layer, and a fully connected layer. Among them, the input layer is used to receive raw image data, such as an image to be recognized. The convolutional layer is used to extract local features of the input data through convolution operations. Each convolution kernel in the convolutional layer can extract a specific feature, and multiple convolution kernels can work in parallel to extract different types of features. The pooling layer is used to downsample the output of the convolutional layer to reduce the number of parameters and improve computational efficiency. Common pooling operations include maximum pooling and average pooling. The fully connected layer is used to combine the features extracted by the previous layer for tasks such as classification or regression. Each neuron in the fully connected layer is connected to all neurons in the previous layer.

[0050] Alternatively, the convolutional neural network layer may use a deep neural network, and the convolutional neural network layer may be used to extract features of the image to be identified. The extracted image features may include any one of color type features, color component features, color quantity features, color distribution features, color symmetry area features, and image features outside the standard structure, or any combination of multiple of them.

[0051] Among them, a deep neural network may include an input layer, a hidden layer, and an output layer. Among them, the input layer is the first layer of the network and is responsible for receiving input data. In image recognition tasks, the input layer may receive an array of pixel values. There may be multiple hidden layers; a hidden layer includes a number of neurons. These neurons process the data received from the previous layer and pass the results to the next layer. The number of hidden layers and the number of neurons in each layer can be set according to the complexity of the specific problem.

[0052] Exemplary. In the above-mentioned image analysis sub-model, the first fully connected layer and the second fully connected layer can be fully connected layers configured with 1024 nodes. The feature vector output by the convolutional neural network layer is input into the first fully connected layer; the feature vector output by the first fully connected layer is input into the second fully connected layer. The normalization layer outputs the normalized feature vector. The third fully connected layer can be a fully connected layer including 2 nodes.

[0053] Exemplarily, the text analysis sub-model includes: a recurrent neural network, a first fully connected layer, a second fully connected layer, a normalization layer, and a third fully connected layer, which are arranged in sequence.

[0054] Among them, the recurrent neural network (RNN) is a neural network model that can process sequence data. RNN retains a hidden state when processing each input, and the hidden state will be passed to the next time step so that the model can remember the previous information. Among them, the recurrent neural network can adopt a directed loop network architecture. The first fully connected layer and the second fully connected layer can be a fully connected layer (Fully ConnectedLayer) configured with 1024 nodes. The normalization layer outputs the normalized feature vector. The third fully connected layer can be a fully connected layer including 2 nodes.

[0055] Exemplarily, in some exemplary embodiments, a weighted operation layer may be provided, and the third fully connected layer of the image analysis sub-model and the third fully connected layer of the text analysis sub-model may be fused into a third fully connected layer. Specifically, the first output data of the normalization layer of the image analysis sub-model and the second output data of the normalization layer of the text analysis sub-model may be subjected to weighted averaging processing using the weighted operation layer, and then input into the third fully connected layer to output the classification result.

[0056] In this exemplary embodiment, the method further includes: pre-training the data analysis model, including: Step S21, collecting user case data, constructing an initial sample data set based on the case data; and marking the initial sample data; wherein the initial sample data includes a drawn mandala painting image based on a standard structural formula and a corresponding description text; Step S22, extracting corresponding image feature data and text feature data from the initial sample data, and constructing a target sample data set; Step S23, using the target sample data set to perform forward feedback training on the initial data analysis model to obtain the trained data analysis model; wherein the data analysis model includes an image analysis sub-model for performing feature analysis on image feature data, and a text analysis sub-model for performing feature analysis on text feature data.

[0057] Specifically, user case data can be collected in advance to construct sample data required for model training. Patients who meet the diagnostic criteria for schizophrenia in the 10th revision of the International Classification of Diseases Ten Version (ICD-10), with an age range of 20-59 years old, regardless of gender; right-handed; PANSS hostility P7≤3 points, uncooperative G8≤4 points, total attack risk score≤13 points, or ordinary healthy people can be selected as patient samples. Among them, the patient's case data may include outpatient cases, inpatient cases, examination and testing data, etc. Among them, the examination and testing data may include painting data or corresponding description text. Among them, the case data is provided with data using keywords to construct initial sample data. Among them, the above-mentioned initial sample data may include: basic information of the patient, such as name, code, age, gender, etc. The painted mandala painting image based on the standard structure and the corresponding description text can be obtained from the examination and testing data. In addition, corresponding tags can be added to the initial sample data according to the case data; wherein the tag content may include the disease level, disease type, and whether the disease is present. For the initial sample data, the corresponding image feature data and text feature data may be extracted, and a target sample data set may be constructed based on the feature data.

[0058] Exemplarily, an initial data analysis model can be constructed, which includes an image analysis submodel and a text analysis submodel. Among them, the image analysis submodel is based on a convolutional neural network architecture; the text analysis submodel is based on a recurrent neural network architecture. Sample data is input into the initial data analysis model, image and text feature extraction is performed on the image and text, and the image molecular submodel and the text analysis submodel are used for recognition and analysis respectively, and the recognition result is output. The recognition result is compared with the corresponding labeled data, the error is calculated, and the model parameters are adjusted according to the error. The above process is repeated to train the model, optimize the model, and obtain a trained data analysis model. For example, the sample data is input into the model to perform error-based forward feedback training on the image molecular submodel, such as training using a gradient descent method. The text analysis submodel is trained based on error back propagation, such as training using a BP error back propagation method. The optimized and adjusted image recognition submodel and text recognition submodel are connected in a weighted average to form a multimodal model.

[0059] In some exemplary embodiments, the image to be identified includes multiple frames of painted mandala painting images, each of which is based on a different standard structural formula. The method further includes: Step S31, using the data analysis model to analyze and process each frame of the drawn mandala painting image and the corresponding description text to obtain a plurality of corresponding classification probability results; Step S32, constructing a probability curve corresponding to the analysis object based on the classification probability result, and determining a comprehensive classification probability result based on the probability interval.

[0060] Specifically, when diagnosing the patient, multiple mandala painting image templates can be provided to the patient, and multiple images filled in by the patient and corresponding description texts can be obtained and analyzed separately, so as to obtain multiple schizophrenia classification probability results for the current patient. Then, a probability curve is constructed in the coordinate system for the multiple classification probability results, and a linear analysis / average operation is performed on the probability curve to obtain a comprehensive classification probability result.

[0061] By constructing a probability area curve, it is possible to identify abnormal classification probability results and eliminate the data; then the remaining multiple sets of data are used for calculation and analysis to obtain more accurate analysis results.

[0062] Exemplarily, the mandala painting image templates can be divided into two categories. The first category includes images composed of circles, rectangles and other patterns, such as Figure 4 , Figure 5 As shown; the second category of the embracing image also includes a triangular pattern, or a pattern with a sharp angle less than 90 degrees, such as Figure 3When configuring multiple sets of mandala painting image templates for the current patient, a template image can be randomly selected from the two categories respectively.

[0063] In addition, in some exemplary embodiments, a head image of the patient may be collected and analyzed to obtain the head image analysis result of the current patient. The head image analysis result is used to compare and verify the above-mentioned schizophrenia classification probability result. If the comparison result is the same, the result can be directly output and the data can be saved in the database. Alternatively, if the comparison result is different, a prompt message is generated to indicate that the current test result may be wrong.

[0064] The medical data analysis method based on image data provided by the disclosed embodiment provides patients with a structural mandala painting and 12-color standard colored crayons as a reference; the original picture painted by the patient is scanned into an electronic image through standardization, the electronic image is preprocessed by size circling, and the patient's language description of the picture corresponding to the image and the picture is used as model input data, which is input into the trained data analysis model to analyze the color components of the electronic image, the color, area and symmetry characteristics of the color distribution; and the text features are integrated to achieve the integration and cross-modal analysis of image features and text features. This scheme uses the model designed by using mandala painting to map images and texts to the same feature space. The theoretical basis of mandala painting mainly comes from Jung's psychoanalytic theory. Carl Gustav Jung is a Swiss psychoanalyst who proposed the concept of collective unconsciousness and believed that mandala is a symbolic pattern that can help individuals connect to the collective unconsciousness, thereby achieving the purpose of self-discovery and healing. Mandala painting technology not only has its basis in theory, but also shows its evaluation and treatment functions in clinical psychotherapy. By analyzing and interpreting the patient's mandala works, it can help evaluate and improve the patient's self-self function and promote individual self-discovery and healing. In addition, mandala painting also involves image theory, color theory, functional theory and stage theory, which are all theoretical components of mandala painting psychotherapy. This method can provide data analysis results with higher accuracy by analyzing the patient's mandala painting images. It can also provide an objective, simple and fast tool to assist in the diagnosis of first-episode schizophrenia patients.

[0065] It should be noted that the above figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0066] For further reference, Figure 6As shown, in the embodiment of this example, a medical data analysis system 60 based on image data is also provided, including: The image acquisition module 601 is used to acquire the current image to be identified of the analysis object, and the description text corresponding to the image to be identified; wherein the image to be identified includes a mandala painting image based on a standard structure formula that has been drawn; and the description text includes the description text of the image to be identified generated by the analysis object based on a preset text template; The feature extraction module 602 is used to extract the image feature data corresponding to the image to be identified and the text feature data corresponding to the description text, and configure them as input data; wherein the image feature data is used to describe the color feature of the image to be identified; The feature analysis module 603 is used to input the input data into a trained data analysis model, perform feature analysis on the image feature data using an image analysis sub-model, perform feature analysis on the text feature data using a text analysis sub-model, and output a schizophrenia classification probability result corresponding to the analysis object based on the feature analysis result.

[0067] The specific implementation of each module in the above system has been explained in detail in the corresponding method embodiments.

[0068] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0069] Figure 7 A schematic diagram of an electronic device suitable for implementing an embodiment of the present invention is shown.

[0070] It should be noted that Figure 7 The electronic device 1000 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0071] like Figure 7As shown, the electronic device 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1002 or the program loaded from the storage part 1008 to the random access memory (RAM) 1003. Various programs and data required for system operation are also stored in the RAM 1003. The CPU 1001, the ROM 1002, and the RAM 1003 are connected to each other through a bus 1004. The input / output (I / O) interface 1005 is also connected to the bus 1004.

[0072] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed so that a computer program read therefrom is installed into the storage section 1008 as needed.

[0073] In particular, according to an embodiment of the present invention, the process described below with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present invention includes a computer program product, which includes a computer program carried on a storage medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 1009, and / or installed from a removable medium 1011. When the computer program is executed by the central processing unit (CPU) 1001, various functions defined in the system of the present application are executed.

[0074] Specifically, the electronic device may be a smart mobile electronic device such as a mobile phone, a tablet computer or a laptop computer, or may be a smart electronic device such as a desktop computer, or a smart wearable device.

[0075] It should be noted that the storage medium shown in the embodiment of the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any storage medium other than computer-readable storage media, which may send, propagate, or transmit programs for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0076] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0077] The units involved in the embodiments of the present invention may be implemented by software or hardware, and the units described may also be arranged in a processor. The names of these units do not, in some cases, limit the units themselves.

[0078] It should be noted that, as another aspect, the present application also provides a storage medium, which can be included in an electronic device; or it can exist independently without being installed in the electronic device. The above storage medium carries one or more programs, and when the above one or more programs are executed by an electronic device, the electronic device implements the method described in the following embodiments. For example, the electronic device can implement the following Figure 2 The individual steps of the method are shown.

[0079] In one embodiment, the present application provides a computer program product, including a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

[0080] In addition, the above-mentioned figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above-mentioned figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0081] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0082] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A medical data analysis method based on image data, characterized in that: The method comprises: Acquire the current image to be identified of the analysis object, and the description text corresponding to the image to be identified; wherein the image to be identified includes a mandala painting image based on a standard structure formula that has been drawn; and the description text includes the description text of the image to be identified generated by the analysis object based on a preset text template; Extracting image feature data corresponding to the image to be identified and text feature data corresponding to the description text, and configuring them as input data; wherein the image feature data is used to describe the color features of the image to be identified; The input data is input into a trained data analysis model, the image feature data is feature parsed using an image analysis sub-model, the text feature data is feature parsed using a text analysis sub-model, and a schizophrenia classification probability result corresponding to the analysis object is output based on the feature parsing result; wherein the image analysis sub-model includes: a convolutional neural network layer, a first fully connected layer, a second fully connected layer, a normalization layer, and a third fully connected layer arranged in sequence; the text analysis sub-model includes: a recurrent neural network layer, a first fully connected layer, a second fully connected layer, a normalization layer, and a third fully connected layer arranged in sequence.

2. The method according to claim 1, characterized in that: The image feature data includes: any one or a combination of any multiple of color type features, color component features, color quantity features, color distribution features, color symmetry area features, and image features outside the standard structure.

3. The method according to claim 1, characterized in that After obtaining the current image to be identified of the analysis object, the method further includes: The image to be identified is preprocessed to obtain a preprocessed image to be identified; wherein the preprocessing includes image size adjustment and image definition adjustment.

4. The method according to claim 1, characterized in that: The input data also includes: identity feature data and historical case data of the target object; The method further comprises: Perform text recognition on the identity feature data and historical case data, and extract target keywords corresponding to preset feature fields; and configure the target keywords as input parameters of the text analysis sub-model.

5. The method according to claim 1, characterized in that The image to be identified includes a plurality of frames of drawn mandala painting images, each of which is based on a different standard structural formula; The method further comprises: Utilizing the data analysis model to analyze and process each frame of the drawn mandala painting image and the corresponding description text, so as to obtain a plurality of corresponding classification probability results; A probability curve corresponding to the analysis object is constructed based on the classification probability result, and a comprehensive classification probability result is determined based on the probability interval.

6. The method according to claim 1, characterized in that The method further includes: pre-training the data analysis model, including: Collect user case data, build an initial sample data set based on the case data; and mark the initial sample data; wherein the initial sample data includes a drawn mandala painting image based on a standard structure and a corresponding description text; Extracting corresponding image feature data and text feature data from the initial sample data, and constructing a target sample data set; The target sample data set is used to perform forward feedback training on the initial data analysis model to obtain the trained data analysis model; wherein the data analysis model includes an image analysis sub-model for performing feature analysis on image feature data, and a text analysis sub-model for performing feature analysis on text feature data.

7. A medical data analysis system based on image data, characterized in that: The system comprises: An image acquisition module, used to acquire the current image to be identified of the analysis object, and the description text corresponding to the image to be identified; wherein the image to be identified includes a mandala painting image based on a standard structural formula that has been drawn; and the description text includes the description text of the image to be identified generated by the analysis object based on a preset text template; A feature extraction module, used to extract image feature data corresponding to the image to be identified and text feature data corresponding to the description text, and configure them as input data; wherein the image feature data is used to describe the color features of the image to be identified; A feature analysis module is used to input the input data into a trained data analysis model, perform feature analysis on the image feature data using an image analysis sub-model, perform feature analysis on the text feature data using a text analysis sub-model, and output a schizophrenia classification probability result corresponding to the analysis object based on the feature analysis result; wherein the image analysis sub-model includes: a convolutional neural network layer, a first fully connected layer, a second fully connected layer, a normalization layer, and a third fully connected layer arranged in sequence; and the text analysis sub-model includes: a recurrent neural network layer, a first fully connected layer, a second fully connected layer, a normalization layer, and a third fully connected layer arranged in sequence.

8. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the medical data analysis method based on image data according to any one of claims 1 to 6 is implemented.

9. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; Wherein, the processor is configured to perform the medical data analysis method based on image data according to any one of claims 1 to 6 by executing the executable instructions.

Images