Myopia progress risk prediction method and device based on multi-mode intelligent diagnosis model

Abstract

The invention discloses a myopia progress risk prediction method and device based on a multi-mode intelligent diagnosis model. The method comprises the following steps: acquiring user data of a target user; wherein the user data at least comprises multi-modal data of the target user; extracting target data from the user data, and generating a data analysis result based on the target data and a predetermined data analysis model; performing data matching operation on the data analysis result and a predetermined target data result to obtain a data matching result, and determining data feature information based on the data matching result; and generating a myopia progress risk prediction result according to the data feature information. Therefore, by implementing the method and the device, the lesion of the target user can be intelligently predicted based on the multi-modal intelligent diagnosis model, the accuracy and the reliability of the myopia progress risk prediction of the target user can be improved, and the intelligence and the efficiency of the myopia progress risk prediction of the target user can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence model and intelligent prediction, in particular to a myopia progression risk prediction method and device based on a multi-modal intelligent diagnosis model. BACKGROUND

[0002] In recent years, myopia has shown an outbreak trend similar to an epidemic disease in the world, and its incidence rate has been increasing year by year. According to the latest epidemiological survey, the global myopia prevalence rate of all age groups is about 30%, of which 163 million people (accounting for 2.7% of the total population) suffer from high myopia. It is estimated that by 2050, the global myopia population will reach 4.8 billion (accounting for 49.8% of the global population). In addition to harming eye health, myopia also has a significant impact on public health and social and economic well-being. At the same time, PM is highly prevalent in young adults and continues to progress, and its direct costs (expenditure on diagnosis, intervention, management, treatment) and productivity losses are enormous. It is of great importance to achieve comprehensive, efficient and accurate myopia screening in large populations, especially in adolescents, to detect PM and treat related myopic maculopathy in a timely manner, to warn of possible disease progression, and to prevent irreversible vision loss. Scientific research work carried out to meet this demand has great scientific significance and social value.

[0003] However, most of the current myopia progression risk prediction methods are based on patient medical record data and combined with the diagnosis experience of ophthalmologists to make diagnosis and prediction. However, this method completely depends on the experience of ophthalmologists and is prone to diagnosis errors. Therefore, it is particularly important to provide a new myopia progression risk prediction method to improve the prediction accuracy and reliability of the myopia progression risk of patients. SUMMARY

[0004] The present application provides a myopia progression risk prediction method and device based on a multi-modal intelligent diagnosis model, which can intelligently predict the myopia progression risk of a target user based on a multi-modal intelligent diagnosis model, thereby improving the accuracy and reliability of the myopia progression risk prediction of the target user and improving the intelligence and efficiency of the myopia progression risk prediction of the target user.

[0005] To solve the above technical problems, the present application discloses a myopia progression risk prediction method based on a multi-modal intelligent diagnosis model, which comprises:

[0006] obtaining user data of a target user; wherein the user data at least includes multi-modal data of the target user;

[0007] extract target data from the user data, generate a data analysis result based on the target data and a predetermined data analysis model;

[0008] perform a data matching operation on the data analysis result and a predetermined target data result to obtain a data matching result, and determine data feature information based on the data matching result;

[0009] generate a myopia progression risk prediction result according to the data feature information and a pre-constructed multi-modal intelligent diagnosis model.

[0010] As an optional implementation form, in the first aspect of the present application, before the myopia progression risk prediction result is generated according to the data feature information and the pre-constructed multi-modal intelligent diagnosis model, the method further comprises:

[0011] perform a data processing operation on the data feature information to obtain image data feature information and text data feature information, wherein each image data feature information has corresponding text data feature information;

[0012] for each image data feature information, perform a data conversion operation on the image data feature information to obtain a data conversion result of the image data feature information, and generate target data feature information of the image data feature information according to the data conversion result of the image data feature information and the text data feature information corresponding to the image data feature information;

[0013] wherein the myopia progression risk prediction result is generated according to the data feature information and the pre-constructed multi-modal intelligent diagnosis model, comprising:

[0014] generate a myopia progression risk prediction result according to all the target data feature information and the pre-constructed multi-modal intelligent diagnosis model.

[0015] As an optional implementation form, in the first aspect of the present application, after the myopia progression risk prediction result is generated according to the data feature information and the pre-constructed multi-modal intelligent diagnosis model, the method further comprises:

[0016] perform a verification operation on the myopia progression risk prediction result to obtain a prediction verification result of the myopia progression risk prediction result, and perform a model verification operation on the pre-determined data analysis model based on the prediction verification result to obtain a model verification result;

[0017] determine whether the model verification result meets a preset model running condition;

[0018] When it is judged that the model verification result does not satisfy the preset model running condition, a model optimization operation is performed on the data analysis model determined in advance based on the model verification result and the preset model running condition, so as to update the data analysis model.

[0019] As an optional implementation, in the first aspect of the present application, the data analysis result is generated based on the target data and the data analysis model determined in advance, which comprises:

[0020] The target data is input into the data analysis model determined in advance, so as to perform a data decoupling operation on the target data by the data analysis model determined in advance, so as to obtain a data decoupling result corresponding to the target data, wherein the data decoupling result at least comprises a data semantic feature corresponding to the target data.

[0021] The data analysis result corresponding to the target data is generated based on the data decoupling result corresponding to the target data.

[0022] As an optional implementation, in the first aspect of the present application, the data matching result is obtained by performing a data matching operation on the data analysis result and the target data result determined in advance, which comprises:

[0023] The first key information in the data analysis result is extracted, and the second key information in the target data result determined in advance is extracted, and an information matching operation is performed on the first key information and the second key information, so as to obtain an information matching result.

[0024] Based on the information matching result, the data distribution information corresponding to the data analysis result is generated, and the data matching result is generated according to the data distribution information.

[0025] As an optional implementation, in the first aspect of the present application, the myopia progression risk prediction result is generated according to all the target data feature information and the multi-modal intelligent diagnosis model constructed in advance, which comprises:

[0026] All the target data feature information is input into the multi-modal intelligent diagnosis model constructed in advance, so as to perform a feature extraction operation on all the target data feature information by a feature extraction layer in the multi-modal intelligent diagnosis model, so as to obtain key feature data, and perform a feature fusion operation on all the key feature data, so as to obtain a feature fusion result, and generate target key feature data based on the feature fusion result and a full connection layer in the multi-modal intelligent diagnosis model.

[0027] The myopia progression risk prediction result is generated based on the target key feature data.

[0028] As an optional implementation, in the first aspect of the present application, for each of the image data feature information, the target data feature information of the image data feature information is generated based on the data conversion result according to the image data feature information and the text data feature information corresponding to the image data feature information, and the target data feature information of the image data feature information comprises:

[0029] According to the data conversion result of the image data feature information, the data distribution parameter is determined, wherein the data distribution parameter comprises a prior distribution parameter and a conditional distribution parameter;

[0030] Based on the data distribution parameter, the data conversion result of the image data feature information and the text data feature information corresponding to the image data feature information, the target data feature information of the image data feature information is generated;

[0031] Wherein, the target data feature information of the image data feature information is generated based on the data distribution parameter, the data conversion result of the image data feature information and the text data feature information corresponding to the image data feature information, and the target data feature information of the image data feature information comprises:

[0032] According to the prior distribution parameter, the distribution feature information of the text data feature information corresponding to the image data feature information is determined, and according to the conditional distribution parameter, the text data feature information corresponding to the image data feature information and the hidden feature information of the image data feature information are determined;

[0033] According to the distribution feature information and the hidden feature information, the comprehensive data feature information of the image data feature information is determined, and the target data feature information of the image data feature information is generated based on the comprehensive data feature information of the image data feature information.

[0034] The second aspect of the present application discloses a myopia progression risk prediction device based on a multi-modal intelligent diagnosis model, and the device comprises:

[0035] The acquisition module is used for acquiring user data of a target user; wherein the user data at least comprises multi-modal data of the target user;

[0036] The extraction module is used for extracting target data from the user data;

[0037] The generation module is used for generating a data analysis result based on the target data and a pre-determined data analysis model;

[0038] The matching module is used for performing a data matching operation on the data analysis result and a pre-determined target data result to obtain a data matching result;

[0039] The determination module is used for determining data feature information based on the data matching result.

[0040] The generation module is further configured to generate a myopia progression risk prediction result according to the data feature information and a pre-constructed multi-modal intelligent diagnosis model.

[0041] As an optional implementation, in the second aspect, the apparatus further comprises:

[0042] The processing module is configured to perform a data processing operation on the data feature information to obtain image data feature information and text data feature information before the generation module generates the myopia progression risk prediction result according to the data feature information and the pre-constructed multi-modal intelligent diagnosis model, wherein each image data feature information has corresponding text data feature information.

[0043] The conversion module is configured to perform a data conversion operation on each image data feature information to obtain a data conversion result of the image data feature information.

[0044] The generation module is further configured to generate target data feature information of the image data feature information according to the data conversion result of the image data feature information and the text data feature information corresponding to the image data feature information.

[0045] The generation module generates the myopia progression risk prediction result according to the data feature information and the pre-constructed multi-modal intelligent diagnosis model in the following manner:

[0046] The generation module generates the myopia progression risk prediction result according to all the target data feature information and the pre-constructed multi-modal intelligent diagnosis model.

[0047] As an optional implementation, in the second aspect, the apparatus further comprises:

[0048] The verification module is configured to perform a verification operation on the myopia progression risk prediction result after the generation module generates the myopia progression risk prediction result according to the data feature information and the pre-constructed multi-modal intelligent diagnosis model, to obtain a prediction verification result of the myopia progression risk prediction result, and perform a model verification operation on the pre-determined data analysis model based on the prediction verification result, to obtain a model verification result.

[0049] The judgment module is configured to judge whether the model verification result meets a preset model running condition.

[0050] An updating module is configured to, when the judging module judges that the model verification result does not satisfy the preset model running condition, perform a model optimization operation on the data analysis model determined in advance based on the model verification result and the preset model running condition, so as to update the data analysis model.

[0051] As an optional implementation, in the second aspect, the specific manner in which the generating module generates the data analysis result based on the target data and the data analysis model determined in advance includes:

[0052] inputting the target data into the data analysis model determined in advance, so as to perform a data decoupling operation on the target data by using the data analysis model determined in advance, and obtain a data decoupling result corresponding to the target data, wherein the data decoupling result at least includes a data semantic feature corresponding to the target data;

[0053] generating the data analysis result corresponding to the target data based on the data decoupling result corresponding to the target data.

[0054] As an optional implementation, in the second aspect, the specific manner in which the matching module performs the data matching operation on the data analysis result and the target data result determined in advance to obtain the data matching result includes:

[0055] extracting first key information in the data analysis result and second key information in the target data result determined in advance, performing an information matching operation on the first key information and the second key information to obtain an information matching result;

[0056] generating data distribution information corresponding to the data analysis result based on the information matching result, and generating the data matching result according to the data distribution information.

[0057] As an optional implementation, in the second aspect, the specific manner in which the generating module generates the myopia progression risk prediction result according to all the target data feature information and the multi-modal intelligent diagnosis model constructed in advance includes:

[0058] inputting all the target data feature information into the multi-modal intelligent diagnosis model constructed in advance, so as to perform a feature extraction operation on all the target data feature information by using a feature extraction layer in the multi-modal intelligent diagnosis model, obtain key feature data, perform a feature fusion operation on all the key feature data to obtain a feature fusion result, and generate target key feature data based on the feature fusion result and a full connection layer in the multi-modal intelligent diagnosis model;

[0059] Generate a myopia progression risk prediction result based on the target key feature data.

[0060] As an optional implementation, in the second aspect of the present application, for each image data feature information, the generation module generates the target data feature information of the image data feature information according to the data conversion result of the image data feature information and the corresponding text data feature information of the image data feature information. The specific way includes:

[0061] According to the data conversion result of the image data feature information, determine the data distribution parameter, wherein the data distribution parameter includes the prior distribution parameter and the conditional distribution parameter;

[0062] Based on the data distribution parameter, the data conversion result of the image data feature information and the corresponding text data feature information of the image data feature information, generate the target data feature information of the image data feature information;

[0063] The specific way of the generation module generating the target data feature information of the image data feature information based on the data distribution parameter, the data conversion result of the image data feature information and the corresponding text data feature information of the image data feature information includes:

[0064] According to the prior distribution parameter, determine the distribution feature information of the corresponding text data feature information of the image data feature information, and according to the conditional distribution parameter, determine the corresponding text data feature information of the image data feature information and the hidden feature information of the image data feature information;

[0065] According to the distribution feature information and the hidden feature information, determine the comprehensive data feature information of the image data feature information, and generate the target data feature information of the image data feature information based on the comprehensive data feature information of the image data feature information.

[0066] The third aspect of the present application discloses another myopia progression risk prediction device based on a multi-modal intelligent diagnosis model, which comprises:

[0067] A memory storing executable program codes;

[0068] A processor coupled with the memory;

[0069] The processor calls the executable program codes stored in the memory to execute the myopia progression risk prediction method based on the multi-modal intelligent diagnosis model disclosed in the first aspect of the present application.

[0070] The fourth aspect of the present application discloses a computer storage medium, the computer storage medium stores computer instructions, when the computer instructions are invoked, the computer instructions are used to execute the myopia progression risk prediction method based on the multi-modal intelligent diagnosis model disclosed in the first aspect of the present application.

[0071] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0072] In the embodiments of the present application, user data of a target user is acquired, wherein the user data at least includes multi-modal data of the target user; target data is extracted from the user data, and a data analysis result is generated based on the target data and a pre-determined data analysis model; a data matching operation is performed on the data analysis result and a pre-determined target data result to obtain a data matching result, and data feature information is determined based on the data matching result; and a myopia progression risk prediction result is generated according to the data feature information. It can be seen that, by implementing the present application, the myopia progression risk of a target user can be intelligently predicted based on a multi-modal intelligent diagnosis model, which is beneficial to improving the accuracy and reliability of the myopia progression risk prediction of the target user, and is beneficial to improving the intelligence and efficiency of the myopia progression risk prediction of the target user. BRIEF DESCRIPTION OF DRAWINGS

[0073] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0074] Figure 1 is a flowchart of a myopia progression risk prediction method based on a multi-modal intelligent diagnosis model disclosed by the embodiments of the present application;

[0075] Figure 2 is a flowchart of another myopia progression risk prediction method based on a multi-modal intelligent diagnosis model disclosed by the embodiments of the present application;

[0076] Figure 3 is a structural diagram of a myopia progression risk prediction device based on a multi-modal intelligent diagnosis model disclosed by the embodiments of the present application;

[0077] Figure 4 is a structural diagram of another myopia progression risk prediction device based on a multi-modal intelligent diagnosis model disclosed by the embodiments of the present application;

[0078] Figure 5 is a structural diagram of still another myopia progression risk prediction device based on a multi-modal intelligent diagnosis model disclosed by the embodiments of the present application. DETAILED DESCRIPTION

[0079] In order to better understand the technical scheme of the present application, the technical scheme in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0080] The terms "first", "second", etc. in the specification and claims of the present application and the above drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product or end including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or end.

[0081] Reference herein to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearance of the phrase in various places in the specification does not necessarily all refer to the same embodiment, nor is it necessarily mutually exclusive of other embodiments. It is explicitly and implicitly understood that the embodiments described herein can be combined with other embodiments.

[0082] The present application discloses a myopia progression risk prediction method and device based on a multi-modal intelligent diagnosis model, which can intelligently predict the myopia progression risk of a target user based on a multi-modal intelligent diagnosis model, thereby improving the accuracy and reliability of myopia progression risk prediction for the target user, and improving the intelligence and efficiency of myopia progression risk prediction for the target user. The following will be described in detail.

[0083] Embodiment one

[0084] Please refer to Figure 1 , Figure 1 is a flowchart of a myopia progression risk prediction method based on a multi-modal intelligent diagnosis model disclosed by the embodiments of the present application. Among them, Figure 1 The described myopia progression risk prediction method based on a multi-modal intelligent diagnosis model can be applied to a myopia progression risk prediction device based on a multi-modal intelligent diagnosis model, wherein the myopia progression risk prediction device based on a multi-modal intelligent diagnosis model can be integrated in a cloud server or a local server, and the embodiments of the present application are not limited. For example Figure 1As shown, the myopia progression risk prediction method based on the multi-modal intelligent diagnosis model can include the following operations:

[0085] 101. Obtain user data of a target user.

[0086] In the embodiment of the application, the user data at least includes multi-modal data of the target user.

[0087] In the embodiment of the application, optionally, the user data of the target user can be obtained in real time, or can be obtained at a preset time interval, or can be obtained when the target user needs to be predicted for myopia progression risk, and the embodiment of the application does not make specific limitation. Further optionally, the target user can be a myopia patient or a normal patient, and the embodiment of the application does not make specific limitation.

[0088] In the embodiment of the application, optionally, the multi-modal data of the target user includes one or more of medical record text data, fundus image data, fundus video data, OCT data, OCTA data, and optometry examination data of the target user. The OCT data refers to data obtained by optical coherence tomography (OCT) technology. OCT is a high-resolution biological imaging technology widely used in medical, biological and material science fields. OCT is based on optical interference principle, uses the characteristics of laser beam and sample mutual interference to realize high-resolution imaging of sample internal structure. The OCT system uses a low-coherence laser light source, splits the laser beam into a reference light path and a sample light path through an optical beam splitter, and the reflected light in the sample and the reference light overlap on the light detector to form interference. By measuring the phase difference and amplitude difference between the reflected light signal and the reference light signal, the internal structure information of the sample is obtained. The OCTA data refers to data obtained by optical coherence tomography angiography (OCTA). It mainly covers high-resolution three-dimensional imaging information of retinal vascular structure. OCTA is a non-invasive imaging technology that constructs high-resolution vascular images by measuring light reflection and scattering in the retina tissue without using contrast agents. OCTA uses optical principles to detect blood flow in retinal blood vessels to provide detailed vascular structure and blood flow information.

[0089] 102. Extract target data from the user data, and generate a data analysis result based on the target data and a pre-determined data analysis model.

[0090] In the embodiment of the present application, optionally, the target data extracted from the user data can include data after cleaning and desensitization of the user data. For example, the desensitized data can include data after deleting the personal information such as the name and ID card of the target user; the cleaned data can include data after deleting the obviously missing disease information of the target user and the data with poor image quality, or data after deleting the data with image size and image quality not meeting the preset requirements.

[0091] In the embodiment of the present application, optionally, the data analysis model can include a Multi-Stage CogView2 model, wherein the Multi-Stage CogView2 model is a model further developed and optimized on the basis of CogView2, aiming to improve the quality and efficiency of generated images. The Multi-Stage CogView2 model is a cross-modal generation model that converts text descriptions into high-quality images. The model adopts a multi-stage generation strategy to gradually generate more detailed and complex images through multiple steps. The model mainly consists of two core parts: a text encoder (Text Encoder) and an image generator (Image Generator). The text encoder is responsible for converting the input text description into a vector representation that the model can understand. The image generator gradually generates images based on the vector representation output by the text encoder.

[0092] 103. Perform a data matching operation on the data analysis result and the pre-determined target data result to obtain a data matching result, and determine the data feature information based on the data matching result.

[0093] In the embodiment of the present application, optionally, the pre-determined target data result can include a keyword or a keyword. Further optionally, the pre-determined target data result can also include a word or a synonym corresponding to the data analysis result.

[0094] In the embodiment of the present application, optionally, the data matching result can include the matching degree between the data analysis result and the pre-determined target data result.

[0095] In the embodiment of the present application, optionally, the determination of the data feature information based on the data matching result can include:

[0096] Based on the data matching result, the matching degree between the data analysis result and the pre-determined target data result is determined, and based on the matching degree, the feature information matching the matching degree is determined in the pre-determined feature database, and the data feature information is determined based on all the feature information.

[0097] 104、generate the myopia progression risk prediction result according to the data feature information and the multi-modal intelligent diagnosis model constructed in advance.

[0098] Optionally, the myopia progression risk prediction result includes a myopia progression risk prediction result corresponding to the target user.

[0099] It can be seen that the implementation Figure 1 The described myopia progression risk prediction method based on a multi-modal intelligent diagnosis model can obtain user data of a target user and extract target data, generate data analysis results based on the target data and a data analysis model, perform data matching operations on the data analysis results and a target data result determined in advance to obtain a data matching result and determine data feature information, and generate a myopia progression risk prediction result according to the data feature information. The method can more comprehensively understand the health status of the target user through the obtained multi-modal data of the target user, thereby improving the accuracy and reliability of subsequent myopia progression risk prediction of the target user. The target data is processed through the pre-determined data analysis model, which can efficiently extract key information related to myopia progression risk prediction, and is also beneficial to improving the accuracy and reliability of the obtained myopia progression risk prediction result. Matching the data analysis results with the pre-determined target data result can help identify data features related to the target lesion. Through the data matching and feature extraction process, the myopia progression risk prediction result can be based on data features closely related to the lesion, thereby improving the pertinence and accuracy of the prediction result. Through multi-modal data and a data analysis model, a personalized myopia progression risk prediction result for the target user can be generated, which is beneficial to improving the accuracy and reliability of the obtained myopia progression risk prediction result of the target user, and is also beneficial to improving the intelligence and efficiency of the obtained myopia progression risk prediction result of the target user.

[0100] Embodiment Two

[0101] Please refer to Figure 2 , Figure 2 is another flowchart of a myopia progression risk prediction method based on a multi-modal intelligent diagnosis model disclosed in the embodiments of the present application. Wherein, Figure 2 The described myopia progression risk prediction method based on a multi-modal intelligent diagnosis model can be applied to a myopia progression risk prediction device based on a multi-modal intelligent diagnosis model. The myopia progression risk prediction device based on a multi-modal intelligent diagnosis model can be integrated in a cloud server or a local server, and the embodiments of the present application do not limit it. As Figure 2 The myopia progression risk prediction method based on a multi-modal intelligent diagnosis model can include the following operations:

[0102] 201、obtain user data of a target user.

[0103] 202. Extract target data from the user data, and generate a data analysis result based on the target data and a predetermined data analysis model.

[0104] 203. Perform a data matching operation on the data analysis result and the predetermined target data result to obtain a data matching result, and determine data feature information based on the data matching result.

[0105] In the embodiment of the present invention, for the detailed description of steps 201 to 203 , please refer to the other descriptions of steps 101 to 103 in the first embodiment, and the embodiment of the present invention will not be repeated.

[0106] 204. Perform data processing operations on the data feature information to obtain image data feature information and text data feature information.

[0107] In the embodiment of the present invention, each piece of image data feature information contains text data feature information corresponding to the image data feature information.

[0108] In the embodiment of the present invention, optionally, the image data feature information may include all image information included in the data feature information; and the text data feature information may include all text information included in the data feature information.

[0109] 205. For each image data feature information, perform a data conversion operation on the image data feature information to obtain a data conversion result of the image data feature information, and generate target data feature information of the image data feature information based on the data conversion result of the image data feature information and the text data feature information corresponding to the image data feature information.

[0110] In an embodiment of the present invention, optionally, the data conversion operation may include one or more of a data format conversion operation, a data size conversion operation, a data color conversion operation, and a data texture conversion operation, which is not specifically limited in the embodiment of the present invention.

[0111] In the embodiment of the present invention, optionally, each image data feature information has text data matching it. Further, optionally, the text data corresponding to different image data feature information can be different or the same, which is not specifically limited in the embodiment of the present invention.

[0112] 206. Generate myopia progression risk prediction results based on all target data feature information and the pre-built multimodal intelligent diagnostic model.

[0113] In the embodiment of the present invention, optionally, the myopia progression risk prediction result includes the myopia progression risk prediction result corresponding to the target user.

[0114] It can be seen that the implementation Figure 2 The myopia progression risk prediction method based on the multi-modal intelligent diagnosis model can perform data processing operations on the image data feature information and the text data feature information obtained from the data feature information, perform data conversion operations on the image data feature information to obtain data conversion results of the image data feature information, generate target data feature information based on the data conversion results of the image data feature information and the text data feature information corresponding to the image data feature information, and then generate a myopia progression risk prediction result according to the target data feature information. By performing data processing operations on the data feature information, the original data can be clearly divided into image data feature information and text data feature information, and different types of data can be processed more accurately, thereby improving the accuracy of myopia progression risk prediction. By performing data conversion operations on the image data feature information, a data format more suitable for machine learning or deep learning model processing can be obtained, the complexity of the model can be reduced, the prediction efficiency can be improved, and the prediction accuracy can be further improved. Matching the corresponding text data feature information for each image data feature information provides more rich context information for myopia progression risk prediction, and joint analysis of images and text can capture more potential disease patterns or risk factors, thereby improving the comprehensiveness and accuracy of myopia progression risk prediction. Combining the data conversion results of the image data feature information and the corresponding text data feature information to generate target data feature information is beneficial to the accuracy and reliability of the generated target data feature information, and is beneficial to improving the intelligence and efficiency of the myopia progression risk prediction result of the target user.

[0115] In an optional embodiment, after generating the myopia progression risk prediction result according to the data feature information and the pre-constructed multi-modal intelligent diagnosis model, the method further includes:

[0116] performing a verification operation on the myopia progression risk prediction result to obtain a prediction verification result of the myopia progression risk prediction result, and performing a model verification operation on the pre-determined data analysis model based on the prediction verification result to obtain a model verification result;

[0117] determining whether the model verification result meets a preset model running condition;

[0118] When it is determined that the model verification result does not meet the preset model running condition, performing a model optimization operation on the pre-determined data analysis model based on the model verification result and the preset model running condition to update the data analysis model.

[0119] In the optional embodiment, optionally, the performing a verification operation on the myopia progression risk prediction result to obtain a prediction verification result of the myopia progression risk prediction result can include: obtaining a target prediction result corresponding to the target user, calculating a result similarity between the target prediction result and the myopia progression risk prediction result, and generating the prediction verification result of the myopia progression risk prediction result based on the result similarity. The target prediction result corresponding to the target user includes a result obtained by an ophthalmologist performing artificial prediction, and the higher the result similarity, the higher the accuracy of the verification result, and the lower the result similarity, the lower the accuracy of the verification result.

[0120] In the optional embodiment, optionally, the performing a verification operation on the myopia progression risk prediction result to obtain a prediction verification result of the myopia progression risk prediction result can include: obtaining a target prediction result corresponding to the target user, calculating a result similarity between the target prediction result and the myopia progression risk prediction result, and generating the prediction verification result of the myopia progression risk prediction result based on the result similarity. The target prediction result corresponding to the target user includes a result obtained by an ophthalmologist performing artificial prediction, and the higher the result similarity, the higher the accuracy of the verification result, and the lower the result similarity, the lower the accuracy of the verification result.

[0121] Based on the model verification result and the preset model running condition, at least one to-be-adjusted parameter is determined in the predetermined data analysis model, and an optimization parameter corresponding to each to-be-adjusted parameter is determined. The parameter optimization operation matched with the optimization parameter corresponding to each to-be-adjusted parameter is performed on each to-be-adjusted parameter to update the to-be-adjusted parameter, and thus the data analysis model is updated.

[0122] In the optional embodiment, optionally, when it is judged that the model verification result meets the preset model running condition, the process can be ended.

[0123] It can be seen that by implementing the optional embodiment, the check operation can be performed on the myopia progression risk prediction result to obtain a prediction check result, and the check operation can be performed on the data analysis model based on the prediction check result to obtain a model check result, and it is determined whether the model check result meets the model running condition. If it does not meet, the model optimization operation is performed on the data analysis model based on the model check result and the model running condition to update the data analysis model. By performing the check operation on the myopia progression risk prediction result, the prediction check result can be obtained, thereby ensuring the accuracy and reliability of the prediction result. Based on the prediction check result, the model check operation is performed on the data analysis model, which can evaluate the performance of the model in actual application, thereby avoiding prediction errors caused by model running failure or running errors. By determining whether the model check result meets the preset model running condition, the running condition of the model can be dynamically adjusted according to the actual situation, thereby making the model better adapt to the actual application environment, improving the prediction accuracy and efficiency. Based on the model check result and the preset model running condition, the model optimization operation is performed on the data analysis model, which can include adjusting model parameters, improving model structure, adding new features, etc., which is beneficial to improve the prediction ability and robustness of the model. By continuously checking and optimizing the model, the adaptability of the model can be enhanced. By ensuring the accuracy and reliability of the prediction result and continuously optimizing the model to improve the prediction ability, the trust of the user on the prediction result can be enhanced, the user experience can be improved, thereby being beneficial to improve the accuracy and reliability of the myopia progression risk prediction of the target user, and being beneficial to improve the intelligence and efficiency of the myopia progression risk prediction of the target user.

[0124] In another optional embodiment, based on the target data and the predetermined data analysis model, a data analysis result is generated, including:

[0125] The target data is input into the predetermined data analysis model to perform a data decoupling operation on the target data through the predetermined data analysis model to obtain a data decoupling result corresponding to the target data, wherein the data decoupling result at least includes data semantic features corresponding to the target data;

[0126] Based on the data decoupling result corresponding to the target data, a data analysis result corresponding to the target data is generated.

[0127] In this optional embodiment, optionally, the predetermined data analysis model can be a Multi-StageCogView2 model.

[0128] In this optional embodiment, optionally, the above inputting the target data into the predetermined data analysis model to perform a data decoupling operation on the target data through the predetermined data analysis model to obtain a data decoupling result corresponding to the target data can include:

[0129] inputting the target data into the pre-determined data parsing model, performing data decoupling operation on the target data through the data parsing model and the GNDP algorithm, and obtaining a data decoupling result corresponding to the target data.

[0130] In this optional embodiment, optionally, the GNDP algorithm can include a neural algorithm inference based on a graph neural network (GNN), which benefits from the introduction of an algorithmic alignment concept. Algorithmic alignment refers to a good alignment between each component of the neural network and the target algorithm, which helps the neural network to better learn to perform the inference task. GNN is considered to be consistent with dynamic programming (DP), which is a general problem solving strategy for expressing polynomial time algorithms.

[0131] In this optional embodiment, optionally, the data semantic features corresponding to the target data can include one or more of shape features, color features, and morphological features.

[0132] In this optional embodiment, optionally, the above generating a data parsing result corresponding to the target data based on the data decoupling result corresponding to the target data can include determining the data decoupling result corresponding to the target data as the data parsing result corresponding to the target data.

[0133] As can be seen, implementing this optional embodiment can input the target data into the pre-determined data parsing model and perform data decoupling operation on the target data to obtain a data decoupling result, generate a data parsing result corresponding to the target data based on the data decoupling result corresponding to the target data, which can separate different information components in the target data through data decoupling operation, which can help the data parsing model to more accurately capture the core information and features of the target data. The pre-determined data parsing model is trained and optimized, which can quickly and accurately process the input target data to generate high-quality parsing results. Data decoupling operation can reduce the influence of noise and other interference factors on the parsing result, improve the robustness of data parsing, and provide a basis for deeper data analysis and application through data semantic features obtained through data decoupling operation. Through continuous optimization and improvement of data parsing model and decoupling technology, the accuracy and efficiency of data parsing can be further improved, which is conducive to improving the accuracy and reliability of myopia progression risk prediction of the target user, and is conducive to improving the intelligence and efficiency of myopia progression risk prediction of the target user.

[0134] In yet another optional embodiment, performing data matching operation on the data parsing result and the pre-determined target data result to obtain a data matching result, includes:

[0135] The first key information in the data analysis result is extracted, and the second key information in the predetermined target data result is extracted. An information matching operation is performed on the first key information and the second key information to obtain an information matching result.

[0136] Based on the information matching result, data distribution information corresponding to the data analysis result is generated, and a data matching result is generated according to the data distribution information.

[0137] In this optional embodiment, optionally, the first key information includes data semantic features corresponding to the target data in the data analysis result, and the second key information includes literal information corresponding to the target data or synonym information corresponding to the target data.

[0138] In this optional embodiment, optionally, the information matching result at least includes an information matching degree between the first key information and the second key information. Further optionally, the data distribution information corresponding to the data analysis result includes feature distribution information of the target data.

[0139] In this optional embodiment, optionally, for example, the shape, color, and morphology of the target are extracted from the pixel level of the medical image, and various semantic features such as the shape, color, and morphology of the target are extracted. These features can include the geometric properties, texture, and color distribution of the target. The semantic features extracted from the medical image are matched with the corresponding literal description or synonym to further understand the meaning of the medical image. The model needs to learn complex data distribution in order to generate different features and targets with complex overlapping and combination relationships. Through the processes of decoupling semantic features, matching literal descriptions, and learning complex distribution, the model can achieve advanced cognitive ability of the medical image beyond basic visual function, including deep understanding and semantic reasoning of the target in the image.

[0140] It can be seen that by extracting the first key information in the data analysis result and the second key information in the target data result, and performing information matching operation on the first key information and the second key information to obtain information matching result, and generating data distribution information corresponding to the data analysis result based on the information matching result to further generate data matching result, the accuracy of the data matching result can be improved by extracting and comparing the first key information in the data analysis result and the second key information in the target data result, verifying whether the data analysis process accurately captures the core features of the target data, and ensuring the consistency of the two data in the key information through the information matching operation. The generated data distribution information can reveal the distribution of the data analysis result in different dimensions, and based on the data distribution information, the data analysis model can be adjusted and optimized to further improve the accuracy and efficiency of data analysis. The data matching operation is an important part of the data governance process, which helps to improve the consistency and accuracy of data, enhance the monitorability of data quality, support data analysis, and enhance the trust of users in the prediction result, improve user experience, and further improve the accuracy and reliability of the prediction of the progress risk of the target user, and improve the intelligence and efficiency of the prediction of the progress risk of the target user.

[0141] In another optional embodiment, the myopia progression risk prediction result is generated according to all target data feature information and a pre-constructed multi-modal intelligent diagnosis model, including:

[0142] All target data feature information is input into the pre-constructed multi-modal intelligent diagnosis model to perform feature extraction operation on all target data feature information through the feature extraction layer in the multi-modal intelligent diagnosis model to obtain key feature data, perform feature fusion operation on all key feature data to obtain feature fusion result, and generate target key feature data based on the feature fusion result and the full connection layer in the multi-modal intelligent diagnosis model.

[0143] The myopia progression risk prediction result is generated based on the target key feature data.

[0144] In this optional embodiment, optionally, the key feature data can include one or more of image feature information, text feature information, texture feature information, and color feature information in the target data feature information.

[0145] In this optional embodiment, optionally, the feature extraction layer in the multi-modal intelligent diagnosis model can include a plurality of ST-GCN units, wherein the ST-GCN unit is composed of three layers.

[0146] In this optional embodiment, further optionally, for example, the ST-GCN units are composed of three layers. The first layer performs a regular two-dimensional (2D) convolution operation, expanding the dimension of the input node features. Then, the features of the expanded nodes are propagated along the graph edges using graph convolution. After that, a feature map containing the aggregated information of the nodes and their neighbors can be generated. The last layer is similar to the first layer but has a different kernel size. A two-dimensional convolution operation is performed on the time axis to extract the temporal information from the previous layer feature map. For this, a higher-level patient feature map will be generated. Each ST-GCN unit is followed by a channel attention layer to help the model focus on the channels with more meaningful functions. The first two, the middle two, and the last two ST-GCN units have 64, 128, and 256 output channels, respectively, while each channel is followed by a global average pooling layer. The outputs of these pooling layers are concatenated together to achieve feature fusion and generate the final patient feature map. Note that the outputs of the first two pooling operations are not passed to the following units. At the end of the model framework, a fully connected layer and a sigmoid activation function are applied to generate the final output of the diagnosis prediction. Further, the fully connected layer receives all the nodes of the previous layer as input, multiplies them with weights and adds biases, and then passes them to the activation function. The sigmoid function is commonly used in the output layer because it can interpret the output as a probability, representing the probability of the positive class. At the output layer of the model, through the fully connected layer and the sigmoid activation function, the final output of the diagnosis prediction is generated. This means that the output of the model is a value between 0 and 1, representing the probability of predicting the positive class (e.g., the presence of a certain disease).

[0147] It can be seen that implementing the optional embodiment can input all target data features to the multi-modal intelligent diagnosis model to perform feature extraction operations on the target data feature information through the feature extraction layer to obtain key feature data, and perform feature fusion operations on all key feature data to obtain a feature fusion result, and generate target key feature data based on the feature fusion result and the fully connected layer of the multi-modal intelligent diagnosis model to further generate the myopia progression risk prediction result. The feature extraction layer of the multi-modal intelligent diagnosis model can efficiently and accurately extract features for different modal target data feature information, ensuring the integrity and accuracy of the key information. Compared with single-modal data, multi-modal data feature extraction can provide more dimensions and richer information, thereby more comprehensively reflecting the lesion condition. The feature fusion operation can effectively integrate key feature data from different modalities to form a more comprehensive and accurate feature representation. Based on the feature fusion result and the fully connected layer, the multi-modal intelligent diagnosis model can generate target key feature data. The fully connected layer combines and converts each feature in the feature fusion result through full connection, further extracts feature information highly related to the lesion, and can be beneficial to capture more subtle and complex lesion characteristics, improve the accuracy and sensitivity of myopia progression risk prediction. The myopia progression risk prediction result generated based on the target key feature data can more accurately reflect the condition and degree of the lesion. The multi-modal intelligent diagnosis model can more comprehensively understand multiple aspects of the lesion by integrating data information from different modalities, thereby making more accurate and reliable predictions. By combining the feature extraction layer, feature fusion operation, and fully connected layer of the multi-modal intelligent diagnosis model, the accuracy and reliability of myopia progression risk prediction can be significantly improved, which can enhance user trust in the prediction result, improve user experience, and further improve the accuracy and reliability of myopia progression risk prediction for target users, and improve the intelligence and efficiency of myopia progression risk prediction for target users.

[0148] In yet another optional embodiment, for each image data feature information, target data feature information of the image data feature information is generated based on data conversion results of the image data feature information and corresponding textual data feature information of the image data feature information, including:

[0149] According to the data conversion results of the image data feature information, determine the data distribution parameters, wherein the data distribution parameters include the prior distribution parameters and the conditional distribution parameters;

[0150] Based on the data distribution parameters, the data conversion results of the image data feature information, and the corresponding textual data feature information of the image data feature information, the target data feature information of the image data feature information is generated;

[0151] The target data feature information of the image data feature information is generated based on the data distribution parameter, the data conversion result of the image data feature information, and the text data feature information corresponding to the image data feature information, and includes:

[0152] According to the prior distribution parameter, the distribution feature information of the text data feature information corresponding to the image data feature information is determined, and according to the conditional distribution parameter, the hidden feature information of the image data feature information and the text data feature information corresponding to the image data feature information are determined;

[0153] According to the distribution feature information and the hidden feature information, the comprehensive data feature information of the image data feature information is determined, and the target data feature information of the image data feature information is generated based on the comprehensive data feature information of the image data feature information.

[0154] In the optional embodiment, optionally, the comprehensive data feature information at least includes the comprehensive data feature information corresponding to the image data feature information, wherein the comprehensive data feature information at least includes the distribution feature information and the hidden feature information.

[0155] In the optional embodiment, optionally, the prior distribution parameter is a core concept in Bayesian statistics, which represents the distribution parameter obtained according to the knowledge of other related parameters before conducting statistical experiments or observations to obtain samples. These parameters reflect people's guesses or beliefs about the possible values of unknown parameters. The following is an explanation of the prior distribution parameter, which is a distribution parameter obtained according to the knowledge of other related parameters before conducting statistical experiments; the conditional distribution parameter is a parameter that describes the probability distribution of one random variable under the condition that one or more random variables take a value. These parameters help us better understand the dependence between variables and play an important role in probability modeling and data analysis.

[0156] In the optional embodiment, optionally, for example, Multi-Stage CogView2 is mainly based on VAE to make in-depth optimization and improvement, and the goal is to optimize the ELBO (evidence lower bound) of the image and text joint likelihood. In CogView, the dataset is composed of images x and corresponding description texts t. Assuming that the image x can be described by a random process containing hidden variables z, the image generation text process can be converted into the following three steps. First, the text t i can be generated by the prior p(t, θ); then the hidden variable z i can be generated by the conditional distribution p(z|t=t i , θ); finally, the image x i is generated by p(x|z=z i, ψ). Wherein the text t is generated by the prior distribution p(t, θ). This process represents that you assume a prior probability distribution about the text t and the parameter θ. Given the parameter θ, you can sample the text t from this distribution. Wherein θ is the parameter of the prior distribution p(t, θ). This prior distribution is used to generate the text t. In the generation process, it determines the generation rule of the text, such as the distribution characteristics, tendency, etc. of the text; ψ is the parameter of the conditional distribution p(x|z=zi, ψ). This conditional distribution is used to generate the image x, given the hidden variable z. ψ determines the generation rule of the image, including color, shape, texture and other characteristics; z is the hidden variable, which is generated by the conditional distribution p(z|t=ti, θ). Given the text t and the parameter θ, z represents the hidden features related to it. These hidden features may be factors that are not directly observed in the data, but have an impact on the generation of data; the image x is generated by the conditional distribution p(x|z=zi, ψ). Given the hidden variable z and the parameter ψ, there is a conditional probability distribution that describes the generation process of the image x. Sample the image x from this conditional distribution. Further, the Transformer in Multi-Stage CogView2 is responsible for modeling the generation of text images. The single-direction GPT in CogView2 is used for joint modeling of images and text, which contains a 48-layer self-attention architecture, 2560 hidden units in each layer, 40 attention heads, and a total of 4 billion parameters. In addition to containing corresponding text and image tokens, the input also adds the following four separators, and adjusts the input length to 1088, where the four separator tokens can include ROI1, BASE, BOI1, and EOI1.

[0157] It can be seen that implementing the optional embodiment can determine the data distribution parameter according to the data conversion result of the image data feature information, generate the target data feature information based on the data distribution parameter, the data conversion result of the image data feature information and the corresponding text data feature information, determine the distribution feature information of the corresponding text data feature information according to the prior distribution parameter, and determine the text data feature information corresponding to the image data feature information and the hidden feature information according to the conditional distribution parameter, determine the comprehensive data feature information of the image data feature information according to the distribution feature information and the hidden feature information to generate the comprehensive data feature information and then generate the target data feature information. By combining the data conversion result of the image data feature information and the text data feature information, and using the prior distribution parameter and the conditional distribution parameter to determine the data distribution feature, the association and difference between the image and the text can be more accurately captured, thereby improving the accuracy of data fusion. The introduction of the data distribution parameter can help to discover the hidden feature information in the image data feature information, and the fusion of these hidden feature information into the target data feature information can enrich the expression of data and make it more fully reflect the characteristics of the original data. The use of the prior distribution parameter can introduce the information of the domain knowledge or historical data, so that the model can utilize these prior knowledge to make more reasonable inferences when processing new data, which is beneficial to improve the generalization ability of the model. Through the explicit data distribution parameter and the feature fusion process, data processing and feature extraction can be more efficiently performed, which is beneficial to improve the efficiency, speed and intelligence of data processing. Through the data conversion and feature fusion process, data information from different modalities can be effectively integrated to provide more abundant data support for subsequent model training and data analysis, thereby improving the accuracy and reliability of the generated target data feature information, and improving the richness of the generated target data feature information and the generalization ability of the model. The accuracy and reliability of myopia progression risk prediction can be significantly improved, the trust of the user on the prediction result can be enhanced, the user experience can be improved, and the accuracy and reliability of myopia progression risk prediction of the target user can be improved, and the intelligence and efficiency of myopia progression risk prediction of the target user can be improved.

[0158] Embodiment three

[0159] Please refer to Figure 3 , Figure 3 is a structural schematic diagram of a myopia progression risk prediction device based on a multi-modal intelligent diagnosis model disclosed by the embodiment of the application. As Figure 3 shown, the myopia progression risk prediction device based on a multi-modal intelligent diagnosis model can include:

[0160] The acquisition module 301 is configured to acquire user data of a target user, wherein the user data at least includes multi-modal data of the target user.

[0161] The extraction module 302 is configured to extract the target data from the user data;

[0162] The generation module 303 is configured to generate a data analysis result based on the target data and a pre-determined data analysis model;

[0163] The matching module 304 is configured to perform a data matching operation on the data analysis result and a pre-determined target data result to obtain a data matching result;

[0164] The determination module 305 is configured to determine data feature information based on the data matching result;

[0165] The generation module 303 is further configured to generate a myopia progression risk prediction result according to the data feature information and a pre-constructed multi-modal intelligent diagnosis model.

[0166] It can be seen that the apparatus Figure 3 The described apparatus can obtain user data of a target user and extract target data, generate a data analysis result based on the target data and a data analysis model, perform a data matching operation on the data analysis result and a pre-determined target data result to obtain a data matching result and then determine data feature information, and generate a myopia progression risk prediction result according to the data feature information, which can more comprehensively understand the health status of the target user through the obtained multi-modal data of the target user, thereby improving the accuracy and reliability of subsequent myopia progression risk prediction of the target user, and through the pre-determined data analysis model, the target data can be processed to efficiently extract key information related to myopia progression risk prediction, which is also conducive to improving the accuracy and reliability of the obtained myopia progression risk prediction result, matching the data analysis result with the pre-determined target data result can help identify data features related to the target lesion, and through the data matching and feature extraction process, the myopia progression risk prediction result can be based on data features closely related to the lesion, thereby improving the pertinence and accuracy of the prediction result, through multi-modal data and a data analysis model, a personalized myopia progression risk prediction result for the target user can be generated, which is conducive to improving the accuracy and reliability of the obtained myopia progression risk prediction result of the target user, and is conducive to improving the intelligence and efficiency of the obtained myopia progression risk prediction result of the target user, thereby being conducive to improving the accuracy and reliability of the myopia progression risk prediction of the target user, and being conducive to improving the intelligence and efficiency of the myopia progression risk prediction of the target user.

[0167] In an optional embodiment, as Figure 4 shown, the apparatus further comprises:

[0168] The processing module 306 is configured to perform a data processing operation on the data feature information to obtain image data feature information and text data feature information before the generating module 303 generates the myopia progression risk prediction result according to the data feature information and the pre-constructed multi-modal intelligent diagnosis model, and each image data feature information has corresponding text data feature information.

[0169] The conversion module 307 is configured to perform a data conversion operation on each image data feature information to obtain a data conversion result of the image data feature information.

[0170] The generating module 303 is further configured to generate target data feature information of the image data feature information according to the data conversion result of the image data feature information and the text data feature information corresponding to the image data feature information.

[0171] The specific manner in which the generating module 303 generates the myopia progression risk prediction result according to the data feature information and the pre-constructed multi-modal intelligent diagnosis model includes:

[0172] The myopia progression risk prediction result is generated according to all the target data feature information and the pre-constructed multi-modal intelligent diagnosis model.

[0173] It can be seen that the implementation Figure 4The described device can obtain image data feature information and text data feature information by performing data processing operations on data feature information, perform data conversion operations on the image data feature information to obtain data conversion results of the image data feature information, generate target data feature information based on the data conversion results of the image data feature information and the corresponding text data feature information of the image data feature information, and then generate myopia progression risk prediction results according to the target data feature information. By performing data processing operations on data feature information, the original data can be clearly divided into image data feature information and text data feature information, which can more accurately process different types of data, thereby improving the accuracy of myopia progression risk prediction. Performing data conversion operations on image data feature information can obtain a data format more suitable for machine learning or deep learning model processing, which can reduce the complexity of the model, improve the prediction efficiency, and possibly further improve the prediction accuracy. Matching the corresponding text data feature information for each image data feature information provides more rich context information for myopia progression risk prediction, and joint analysis of images and text can capture more potential disease patterns or risk factors, thereby improving the comprehensiveness and accuracy of myopia progression risk prediction. Combining the data conversion results of the image data feature information and the corresponding text data feature information to generate target data feature information is beneficial to the accuracy and reliability of generating target data feature information, and is beneficial to improving the intelligence and efficiency of obtaining myopia progression risk prediction results for target users, thereby being beneficial to improving the accuracy and reliability of myopia progression risk prediction for target users, and being beneficial to improving the intelligence and efficiency of myopia progression risk prediction for target users.

[0174] In another optional embodiment, as shown in Figure 4 The device further comprises:

[0175] The verification module 308 is configured to perform a verification operation on the myopia progression risk prediction result after the generation module 303 generates the myopia progression risk prediction result based on the data feature information, to obtain a prediction verification result of the myopia progression risk prediction result, and perform a model verification operation on the pre-determined data analysis model based on the prediction verification result, to obtain a model verification result.

[0176] The judgment module 309 is configured to judge whether the model verification result meets a preset model running condition.

[0177] The update module 310 is configured to perform a model optimization operation on the pre-determined data analysis model based on the model verification result and the preset model running condition to update the data analysis model when the judgment module 309 judges that the model verification result does not meet the preset model running condition.

[0178] It can be seen that the implementation Figure 4The described device can perform a verification operation on the myopia progression risk prediction result to obtain a prediction verification result, perform a verification operation on the data analysis model based on the prediction verification result to obtain a model verification result, and determine whether the model verification result meets a model running condition. If not, perform a model optimization operation on the data analysis model based on the model verification result and the model running condition to update the data analysis model. By performing a verification operation on the myopia progression risk prediction result, a prediction verification result can be obtained, thereby ensuring the accuracy and reliability of the prediction result. Based on the prediction verification result, a model verification operation is performed on the data analysis model, which can evaluate the performance of the model in actual application and avoid prediction errors caused by model running failure or running errors. By determining whether the model verification result meets the preset model running condition, the running condition of the model can be dynamically adjusted according to the actual situation, which can make the model better adapt to the actual application environment and improve the prediction accuracy and efficiency. Based on the model verification result and the preset model running condition, a model optimization operation is performed on the data analysis model, which can include adjusting model parameters, improving model structure, adding new features, etc., which is conducive to improving the prediction ability and robustness of the model. By continuously verifying and optimizing the model, the adaptability of the model can be enhanced. By ensuring the accuracy and reliability of the prediction result and continuously optimizing the model to improve the prediction ability, the trust of users in the prediction result can be enhanced, the user experience can be improved, and the accuracy and reliability of the myopia progression risk prediction of the target user can be improved, and the intelligence and efficiency of the myopia progression risk prediction of the target user can be improved.

[0179] In yet another optional embodiment, as shown in Figure 4 The generation module 303 generates the data analysis result based on the target data and the predetermined data analysis model in the following specific manner:

[0180] The target data is input into the predetermined data analysis model to perform a data decoupling operation on the target data by the predetermined data analysis model to obtain a data decoupling result corresponding to the target data, wherein the data decoupling result at least includes data semantic features corresponding to the target data.

[0181] Based on the data decoupling result corresponding to the target data, a data analysis result corresponding to the target data is generated.

[0182] As can be seen, the implementation Figure 4The described apparatus can input target data into a predetermined data parsing model and perform a data decoupling operation on the target data to obtain a data decoupling result, generate a data parsing result corresponding to the target data based on the data decoupling result corresponding to the target data, and separate different information components in the target data through the data decoupling operation, which can help the data parsing model more accurately capture the core information and features of the target data. The predetermined data parsing model is trained and optimized, can quickly and accurately process the input target data, and generate high-quality parsing results. The data decoupling operation can reduce the influence of noise and other interference factors on the parsing result, improve the robustness of data parsing, and provide a basis for deeper data analysis and application through the data semantic features obtained through the data decoupling operation. Through continuous optimization and improvement of the data parsing model and decoupling technology, the accuracy and efficiency of data parsing can be further improved, which is conducive to improving the accuracy and reliability of myopia progression risk prediction of the target user, and is conducive to improving the intelligence and efficiency of myopia progression risk prediction of the target user.

[0183] In yet another optional embodiment, as shown in Figure 4 The matching module 304 performs a data matching operation on the data parsing result and the predetermined target data result to obtain a data matching result. The specific manner includes:

[0184] extracting first key information in the data parsing result and second key information in the predetermined target data result, performing an information matching operation on the first key information and the second key information to obtain an information matching result;

[0185] Based on the information matching result, generate data distribution information corresponding to the data parsing result, and generate a data matching result according to the data distribution information.

[0186] As can be seen, the implementation Figure 4The described apparatus can extract first key information in the data analysis result and second key information in the target data result, perform an information matching operation on the first key information and the second key information to obtain an information matching result, generate data distribution information corresponding to the data analysis result based on the information matching result, and further generate a data matching result. By extracting and comparing the first key information in the data analysis result and the second key information in the target data result, it can be verified whether the data analysis process accurately captures the core features of the target data, and the information matching operation ensures the consistency of the two data in the key information, thereby improving the accuracy of the data matching result. The generated data distribution information can reveal the distribution of the data analysis result in different dimensions. Based on the data distribution information, the data analysis model can be adjusted and optimized to further improve the accuracy and efficiency of data analysis. The data matching operation is an important part of the data governance process. By ensuring the consistency and accuracy of the data, it is beneficial to improve the consistency and accuracy of the data, enhance the monitorability of the data quality, support data analysis, and ensure the accuracy and reliability of the prediction result. By continuously optimizing the model to improve the prediction ability, it can enhance the trust of users on the prediction result, improve the user experience, and further improve the accuracy and reliability of the myopia progression risk prediction of the target user, and improve the intelligence and efficiency of the myopia progression risk prediction of the target user.

[0187] In yet another optional embodiment, as shown in Figure 4 The generation module 303 generates a myopia progression risk prediction result according to all target data feature information and a pre-constructed multi-modal intelligent diagnosis model in the following specific manner:

[0188] Input all target data feature information into the pre-constructed multi-modal intelligent diagnosis model, perform feature extraction on all target data feature information through the feature extraction layer in the multi-modal intelligent diagnosis model to obtain key feature data, perform feature fusion on all key feature data to obtain a feature fusion result, and generate target key feature data based on the feature fusion result and the full connection layer in the multi-modal intelligent diagnosis model.

[0189] Generate a myopia progression risk prediction result based on the target key feature data.

[0190] As can be seen, the implementation Figure 4The described device can input all target data features into the multimodal intelligent diagnosis model to perform feature extraction operations on the target data feature information through the feature extraction layer to obtain key feature data, and perform feature fusion operations on all key feature data to obtain feature fusion results. Based on the feature fusion results and the fully connected layer of the multimodal intelligent diagnosis model, target key feature data is generated and then a myopia progression risk prediction result is generated. The feature extraction layer of the multimodal intelligent diagnosis model can perform efficient and accurate feature extraction on target data feature information of different modalities to ensure the integrity and accuracy of key information. Compared with single-modality data, feature extraction of multimodal data can provide more dimensions and richer information, thereby more comprehensively reflecting the lesion condition. The feature fusion operation can effectively integrate key feature data from different modalities to form a more comprehensive and accurate feature representation. Based on the feature fusion results and the fully connected layer, the multimodal intelligent diagnosis model can generate target key feature data. According to the fully connected layer, the various features in the feature fusion results are combined and transformed in a fully connected manner, and the feature information that is highly relevant to the lesion is further extracted, which can help capture more subtle and complex lesion features and improve the accuracy and sensitivity of myopia progression risk prediction. The myopia progression risk prediction results generated based on the target key feature data can more accurately reflect the situation and degree of the lesion. The multimodal intelligent diagnosis model can more comprehensively understand multiple aspects of the lesion by integrating data information of different modalities, thereby making more accurate and reliable predictions. By combining the feature extraction layer, feature fusion operation and fully connected layer of the multimodal intelligent diagnosis model, the accuracy and reliability of myopia progression risk prediction can be significantly improved, the user's trust in the prediction results can be enhanced, and the user experience can be improved, which is conducive to improving the accuracy and reliability of myopia progression risk prediction for target users, as well as improving the intelligence and efficiency of myopia progression risk prediction for target users.

[0191] In another optional embodiment, Figure 4 As shown, for each image data feature information, the generation module 303 generates the target data feature information of the image data feature information according to the data conversion result of the image data feature information and the text data feature information corresponding to the image data feature information. The specific method includes:

[0192] Determining data distribution parameters according to a data conversion result of the image data feature information, wherein the data distribution parameters include prior distribution parameters and conditional distribution parameters;

[0193] generating target data feature information of the image data feature information based on the data distribution parameter, the data conversion result of the image data feature information, and the text data feature information corresponding to the image data feature information;

[0194] The generating module 303 generates the target data feature information of the image data feature information based on the data distribution parameter, the data conversion result of the image data feature information, and the text data feature information corresponding to the image data feature information. The specific manner includes:

[0195] According to the prior distribution parameter, the distribution feature information of the text data feature information corresponding to the image data feature information is determined, and according to the conditional distribution parameter, the hidden feature information of the image data feature information and the text data feature information corresponding to the image data feature information are determined.

[0196] According to the distribution feature information and the hidden feature information, the comprehensive data feature information of the image data feature information is determined, and the target data feature information of the image data feature information is generated based on the comprehensive data feature information of the image data feature information.

[0197] It can be seen that the implementation Figure 5The described device can determine data distribution parameters according to the data conversion result of the image data feature information, generate target data feature information based on the data distribution parameters, the data conversion result of the image data feature information and the corresponding text data feature information, and determine the distribution feature information of the corresponding text data feature information according to the prior distribution parameter, and determine the text data feature information corresponding to the image data feature information and the hidden feature information according to the conditional distribution parameter, determine the comprehensive data feature information of the image data feature information according to the distribution feature information and the hidden feature information to generate the comprehensive data feature information and then generate the target data feature information. By combining the data conversion result of the image data feature information and the text data feature information, and using the prior distribution parameter and the conditional distribution parameter to determine the data distribution feature, the association and difference between the image and the text can be more accurately captured, thereby improving the accuracy of data fusion. The introduction of the data distribution parameter can help to discover the hidden feature information in the image data feature information, and the fusion of these hidden feature information into the target data feature information can enrich the expression of data and make it more fully reflect the characteristics of the original data. The use of the prior distribution parameter can introduce the information of the domain knowledge or historical data, so that the model can use these prior knowledge to make more reasonable inferences when processing new data, which is beneficial to improve the generalization ability of the model. Through the explicit data distribution parameter and the feature fusion process, data processing and feature extraction can be more efficiently performed, which is beneficial to improve the efficiency, speed and intelligence of data processing. Through the data conversion and feature fusion process, data information from different modalities can be effectively integrated to provide more abundant data support for subsequent model training and data analysis, thereby improving the accuracy and reliability of the generated target data feature information, and improving the richness of the generated target data feature information and the generalization ability of the model. The accuracy and reliability of the myopia progression risk prediction can be significantly improved, the trust of the user on the prediction result can be enhanced, the user experience can be improved, and the accuracy and reliability of the myopia progression risk prediction of the target user can be improved, and the intelligence and efficiency of the myopia progression risk prediction of the target user can be improved.

[0198] Embodiment four

[0199] Please refer to Figure 5 , Figure 5 is another structure schematic view of the myopia progression risk prediction device based on the multi-modal intelligent diagnosis model according to an embodiment of the present application. As ​ shown, the myopia progression risk prediction device based on the multi-modal intelligent diagnosis model can include:

[0200] a memory 401 storing executable program codes;

[0201] a processor 402 coupled with the memory 401;

[0202] The processor 402 invokes the executable program code stored in the memory 401 to perform the steps in the myopia progression risk prediction method based on the multi-modal intelligent diagnosis model described in the embodiment one or the embodiment two.

[0203] Embodiment five

[0204] The embodiment of the present application discloses a computer storage medium, which stores computer instructions, and when the computer instructions are invoked, the steps in the myopia progression risk prediction method based on the multi-modal intelligent diagnosis model described in the embodiment one or the embodiment two are executed.

[0205] Embodiment six

[0206] The embodiment of the present application discloses a computer program product, which includes a non-transitory computer readable storage medium storing a computer program, and the computer program is operable to cause a computer to perform the steps in the myopia progression risk prediction method based on the multi-modal intelligent diagnosis model described in the embodiment one or the embodiment two.

[0207] The device embodiments described above are only schematic, wherein the modules described as separate components can or can not be physically separated, and the components displayed as modules can or can not be physical modules, that is, they can be located in one place, or distributed on multiple network modules. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement it without creative labor.

[0208] Those skilled in the art can clearly understand the implementation of the various embodiments by means of software and the necessary general hardware platform through the specific description of the above embodiments, and of course, the various embodiments can also be implemented by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, and the computer software product can be stored in a computer readable storage medium, including a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disk storage, a magnetic disk storage, a magnetic tape storage, or any other computer readable medium that can be used to carry or store data.

[0209] Finally, it should be noted that: the myopia progression risk prediction method and device based on the multi-modal intelligent diagnosis model disclosed in the embodiments of the present application are only the preferred embodiments of the present application, and are used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that; the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for predicting myopia progression risk based on a multimodal intelligent diagnostic model, characterized in that: The method comprises: Acquire user data of a target user; wherein the user data includes at least multimodal data of the target user; extracting target data from the user data, and generating a data analysis result based on the target data and a predetermined data analysis model; Performing a data matching operation on the data analysis result and a predetermined target data result to obtain a data matching result, and determining data feature information based on the data matching result; Based on the data feature information and the pre-built multimodal intelligent diagnostic model, a myopia progression risk prediction result is generated.

2. The method for predicting myopia progression risk based on a multimodal intelligent diagnostic model according to claim 1, wherein: Before generating a myopia progression risk prediction result based on the data feature information and the pre-built multimodal intelligent diagnostic model, the method further includes: Performing data processing operations on the data feature information to obtain image data feature information and text data feature information, wherein each image data feature information has text data feature information corresponding to the image data feature information; For each piece of image data feature information, performing a data conversion operation on the image data feature information to obtain a data conversion result of the image data feature information, and generating target data feature information of the image data feature information based on the data conversion result of the image data feature information and the text data feature information corresponding to the image data feature information; The method of generating a myopia progression risk prediction result based on the data feature information and a pre-built multimodal intelligent diagnostic model includes: Based on all the target data feature information and the pre-built multimodal intelligent diagnostic model, a myopia progression risk prediction result is generated.

3. The method for predicting myopia progression risk based on a multimodal intelligent diagnostic model according to claim 1, wherein: After generating a myopia progression risk prediction result based on the data feature information and a pre-built multimodal intelligent diagnostic model, the method further includes: performing a verification operation on the myopia progression risk prediction result to obtain a prediction verification result of the myopia progression risk prediction result, and performing a model verification operation on the predetermined data analysis model based on the prediction verification result to obtain a model verification result; Determining whether the model verification result meets the preset model operation conditions; When it is determined that the model verification result does not meet the preset model operation conditions, a model optimization operation is performed on the predetermined data analysis model based on the model verification result and the preset model operation conditions to update the data analysis model.

4. The method for predicting myopia progression risk based on a multimodal intelligent diagnostic model according to claim 1, wherein: Generating a data analysis result based on the target data and a predetermined data analysis model includes: Inputting the target data into a predetermined data parsing model to perform a data decoupling operation on the target data using the predetermined data parsing model to obtain a data decoupling result corresponding to the target data, wherein the data decoupling result at least includes a data semantic feature corresponding to the target data; Based on the data decoupling result corresponding to the target data, a data parsing result corresponding to the target data is generated.

5. The method for predicting myopia progression risk based on a multimodal intelligent diagnostic model according to claim 4, characterized in that: The performing of a data matching operation on the data analysis result and the predetermined target data result to obtain a data matching result includes: Extracting first key information from the data analysis result and extracting second key information from the predetermined target data result, performing an information matching operation on the first key information and the second key information to obtain an information matching result; Based on the information matching result, data distribution information corresponding to the data analysis result is generated, and according to the data distribution information, a data matching result is generated.

6. The method for predicting myopia progression risk based on a multimodal intelligent diagnostic model according to claim 2, wherein: The method of generating a myopia progression risk prediction result based on all the target data feature information and a pre-built multimodal intelligent diagnostic model includes: Inputting all of the target data feature information into a pre-built multimodal intelligent diagnosis model, performing a feature extraction operation on all of the target data feature information through a feature extraction layer in the multimodal intelligent diagnosis model to obtain key feature data, performing a feature fusion operation on all of the key feature data to obtain a feature fusion result, and generating target key feature data based on the feature fusion result and a fully connected layer in the multimodal intelligent diagnosis model; Based on the target key feature data, a myopia progression risk prediction result is generated.

7. The method for predicting myopia progression risk based on a multimodal intelligent diagnostic model according to claim 2, wherein: For each piece of image data feature information, generating target data feature information of the image data feature information according to the data conversion result of the image data feature information and the text data feature information corresponding to the image data feature information includes: Determining data distribution parameters according to a data conversion result of the image data feature information, wherein the data distribution parameters include prior distribution parameters and conditional distribution parameters; generating target data feature information of the image data feature information based on the data distribution parameter, the data conversion result of the image data feature information, and the text data feature information corresponding to the image data feature information; The step of generating target data feature information of the image data feature information based on the data distribution parameter, the data conversion result of the image data feature information, and the text data feature information corresponding to the image data feature information includes: Determining distribution characteristic information of the text data characteristic information corresponding to the image data characteristic information based on the prior distribution parameter, and determining the text data characteristic information corresponding to the image data characteristic information and hidden characteristic information of the image data characteristic information based on the conditional distribution parameter; According to the distribution feature information and the hidden feature information, comprehensive data feature information of the image data feature information is determined, and based on the comprehensive data feature information of the image data feature information, target data feature information of the image data feature information is generated.

8. A device for predicting myopia progression risk based on a multimodal intelligent diagnostic model, characterized in that: The device comprises: An acquisition module, configured to acquire user data of a target user; wherein the user data at least includes multimodal data of the target user; An extraction module, configured to extract target data from the user data; A generating module, configured to generate a data analysis result based on the target data and a predetermined data analysis model; A matching module is used to perform a data matching operation on the data analysis result and a predetermined target data result to obtain a data matching result; A determination module, configured to determine data feature information based on the data matching result; The generation module is further used to generate a myopia progression risk prediction result based on the data feature information and a pre-built multimodal intelligent diagnosis model.

9. A device for predicting myopia progression risk based on a multimodal intelligent diagnostic model, characterized in that: The device comprises: a memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the myopia progression risk prediction method based on the multimodal intelligent diagnostic model as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that The computer storage medium stores computer instructions, which, when called, are used to execute the myopia progression risk prediction method based on a multimodal intelligent diagnostic model as described in any one of claims 1 to 7.

Images