Ophthalmic disease visual video generation method and device, terminal equipment and storage medium

Abstract

The invention discloses a visual video generation method and device for ophthalmic diseases, terminal equipment and a storage medium, and the method comprises the steps: inputting an initial visual description text for describing the vision of a patient into a preset large language model, so as to enable the large language model to be combined with a preset visual symptom knowledge graph of the ophthalmic diseases, performing semantic analysis on the initial visual description text, optimizing the initial visual description text according to a semantic analysis result, and generating a visual description text recording a dynamic visual scene of the patient; and inputting the visual description text into a preset text video model, so that the text video model outputs a subjective visual video meeting the view of the patient according to the visual description text. Therefore, the method and the device can output the subjective visual video meeting the view of the patient, thereby effectively helping a doctor to know the visual symptom of the patient, and improving the diagnosis efficiency of the doctor.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, terminal device, and storage medium for generating visual videos of ophthalmic diseases. Background Art

[0002] The characteristic of ophthalmic diseases is "what you see is what you get", and diagnosis relies heavily on symptoms, signs, and image diagnosis. Visual symptoms are one of the most basic manifestations of ophthalmic diseases, and most patients have visual symptoms, such as blurred vision, distorted vision, etc. In textbooks and medical records, in the past, only words such as "distorted vision" and "double vision" were used to briefly classify symptoms of various visual images, but this is actually a dimensionality reduction of information and cannot replace the real visual experience. For example, some patients mentioned "seeing a ball of yarn in the light but not in the dark" and "seeing the road extending to the sky when driving at night", which cannot be directly classified into the existing symptom classification. Secondly, due to the different cultural levels and language expression abilities of patients, the language expressions of symptoms vary greatly and are highly subjective, which may cause doctors to understand the differences.

[0003] Therefore, for ophthalmology, a discipline that relies heavily on visual symptoms for diagnosis and evaluation of therapeutic effects, describing medical records using only text and language will reduce the efficiency of information transmission and affect doctors' understanding and judgment of patients' visual symptoms. Summary of the Invention

[0004] The embodiments of the present invention provide a method, apparatus, terminal device, and storage medium for generating visual videos of ophthalmic diseases, which can effectively help doctors understand patients' visual symptoms and improve doctors' diagnostic efficiency.

[0005] An embodiment of the present invention provides a method for generating visual videos of ophthalmic diseases, comprising:

[0006] Obtaining the patient's initial visual description text describing his or her own vision;

[0007] Inputting the initial visual description text into a preset language macromodel, so that the language macromodel combines the preset eye disease visual symptom knowledge graph to perform semantic analysis on the initial visual description text, and optimizing the initial visual description text based on the semantic analysis result to generate a visual description text recording the patient's dynamic visual scene;

[0008] The visual description text is input into a preset Vincent video model so that the Vincent video model outputs a subjective visual video that is consistent with what the patient sees based on the visual description text.

[0009] Further, the language large model combines a preset knowledge graph of visual symptoms of eye diseases to perform semantic analysis on the initial visual description text, and optimizes the initial visual description text according to the results of the semantic analysis to generate a visual description text, including:

[0010] Perform semantic analysis and dependency grammar analysis on the initial visual description text, and extract symptom descriptions and dynamic scene descriptions in the initial visual description text;

[0011] According to the symptom descriptions, determine a number of symptom medical terms, and for each symptom medical term, match a target symptom node from the knowledge graph of visual symptoms of eye diseases, and determine the typical dynamic features and associated scene features associated with the target symptom node; wherein, the knowledge graph of visual symptoms of eye diseases is a ternary relationship network of symptom nodes, associated scene features, and typical dynamic features;

[0012] According to the symptom medical terms, the dynamic scene description, the typical dynamic features, and the associated scene features, perform text reconstruction to generate a visual description text recording the patient's dynamic visual scene.

[0013] Further, the text reconstruction according to the symptom medical terms, the dynamic scene description, the typical dynamic features, and the associated scene features to generate a visual description text recording the patient's dynamic visual scene includes:

[0014] Perform dynamic scene analysis on the dynamic scene description to generate an analysis result;

[0015] When the analysis result indicates that the dynamic scene description lacks specific scene elements, use the associated scene features to complement the dynamic scene description, and generate a visual description text according to the complemented dynamic scene description and the symptom medical terms;

[0016] When the analysis result indicates that the dynamic scene description lacks dynamic time sequence elements, use the typical dynamic features to complement the dynamic scene description, and generate a visual description text according to the complemented dynamic scene description and the symptom medical terms;

[0017] When the analysis result indicates that the dynamic scene description lacks specific scene elements and dynamic time sequence elements, use the associated scene features and the typical dynamic features to complement the dynamic scene description, and generate a visual description text according to the complemented dynamic scene description and the symptom medical terms.

[0018] Further, the text-to-video model outputs a subjective visual video that conforms to what the patient saw according to the visual description text, including:

[0019] Generate a dynamic scene video according to the dynamic scene description in the visual description text;

[0020] Extract corresponding symptom materials from a preset symptom video material library according to the symptom medical terms in the visual description text;

[0021] Integrate the symptom materials with the dynamic scene video to generate the subjective visual video.

[0022] Another embodiment of the present invention provides a visual video generation device for ophthalmic diseases, including:

[0023] A text acquisition module for acquiring an initial visual description text in which a patient describes their own vision;

[0024] A text reconstruction module for inputting the initial visual description text into a preset language large model, so that the language large model combines a preset ophthalmic disease visual symptom knowledge graph, performs semantic analysis on the initial visual description text, and optimizes the initial visual description text according to the semantic analysis result to generate a visual description text recording the patient's dynamic visual scene;

[0025] A video editing module for inputting the visual description text into a preset text-to-video model, so that the text-to-video model outputs a subjective visual video that conforms to what the patient sees according to the visual description text.

[0026] Further, the language large model combines a preset ophthalmic disease visual symptom knowledge graph, performs semantic analysis on the initial visual description text, and optimizes the initial visual description text according to the semantic analysis result to generate a visual description text, including:

[0027] Perform semantic analysis and dependency grammar analysis on the initial visual description text, and extract symptom descriptions and dynamic scene descriptions in the initial visual description text;

[0028] Determine a number of symptom medical terms according to the symptom descriptions, match target symptom nodes from the ophthalmic disease visual symptom knowledge graph for each symptom medical term, and determine typical dynamic features and associated scene features associated with the target symptom nodes; wherein, the ophthalmic disease visual symptom knowledge graph is a ternary relationship network of symptom nodes, associated scene features, and typical dynamic features;

[0029] Perform text reconstruction according to the symptom medical terms, the dynamic scene description, the typical dynamic features, and the associated scene features to generate a visual description text recording the patient's dynamic visual scene.

[0030] Further, based on the symptom medical terms, the dynamic scene description, the typical dynamic features, and the associated scene features, text reconstruction is performed to generate a visual description text recording the patient's dynamic visual scene, including:

[0031] Perform dynamic scene analysis on the dynamic scene description to generate an analysis result;

[0032] When the analysis result indicates that the dynamic scene description lacks specific scene elements, use the associated scene features to complement the dynamic scene description, and generate a visual description text based on the complemented dynamic scene description and the symptom medical terms;

[0033] When the analysis result indicates that the dynamic scene description lacks dynamic time sequence elements, use the typical dynamic features to complement the dynamic scene description, and generate a visual description text based on the complemented dynamic scene description and the symptom medical terms;

[0034] When the analysis result indicates that the dynamic scene description lacks both specific scene elements and dynamic time sequence elements, use the associated scene features and the typical dynamic features to complement the dynamic scene description, and generate a visual description text based on the complemented dynamic scene description and the symptom medical terms.

[0035] Further, the text-to-video model outputs a subjective visual video that conforms to what the patient sees based on the visual description text, including:

[0036] Generate a dynamic scene video based on the dynamic scene description in the visual description text;

[0037] Extract corresponding symptom materials from a preset symptom video material library according to the symptom medical terms in the visual description text;

[0038] Integrate the symptom materials with the dynamic scene video to generate the subjective visual video.

[0039] Another embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a method for generating a visual video of an ophthalmic disease as described in any one of the above embodiments.

[0040] Another embodiment of the present invention provides a storage medium, which includes a stored computer program. When the computer program runs, it controls the device where the storage medium is located to execute a method for generating a visual video of an ophthalmic disease as described in any one of the above embodiments.

[0041] The implementation of the present invention has the following beneficial effects:

[0042] The present invention discloses a method, device, terminal device, and storage medium for generating visual videos of ophthalmic diseases. By inputting the initial visual description text in which a patient describes their own vision into a preset large language model, the large language model combines a preset knowledge graph of ophthalmic disease visual symptoms to perform semantic analysis on the initial visual description text, and optimizes the initial visual description text according to the semantic analysis result to generate a visual description text recording the patient's dynamic visual scene; the visual description text is input into a preset text-to-video model, so that the text-to-video model outputs a subjective visual video conforming to what the patient sees. Therefore, the present invention can output a subjective visual video conforming to what the patient sees, thereby effectively helping doctors understand the patient's visual symptoms and improving the doctor's diagnosis efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a schematic flowchart of a method for generating a visual video of an ophthalmic disease provided by an embodiment of the present invention.

[0044] Figure 2 is a schematic structural diagram of a device for generating a visual video of an ophthalmic disease provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] To make the objectives, technical solutions, and advantages of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" and any variations thereof in the specification and claims of this application and the above drawings are intended to cover non-exclusive inclusion.

[0047] In the description of the embodiments of the present application, technical terms such as "first" and "second" are only used to distinguish different objects and cannot be understood as indicating or implying relative importance or implicitly indicating the quantity, specific order, or primary-secondary relationship of the indicated technical features. In the description of the embodiments of the present application, "a plurality" means two or more unless otherwise specifically defined.

[0048] As used herein, the term "embodiment" means that the specific features, structures, or characteristics described in connection with the embodiment may be included in at least one embodiment of the present application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein may be combined with other embodiments.

[0049] In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after.

[0050] In the description of the embodiments of the present application, the term "plurality" refers to two or more (including two). Similarly, "multiple groups" refers to two or more groups (including two groups), and "multiple pieces" refers to two or more pieces (including two pieces).

[0051] In the description of the embodiments of the present application, unless otherwise clearly specified and limited, technical terms such as "installation", "connection", "connection", and "fixation" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or integrated; it can also be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two components or the interaction relationship between two components. For those of ordinary skill in the art, the specific meanings of the above terms in the embodiments of the present application can be understood according to specific situations.

[0052] See Figure 1 , which is a schematic flowchart of a method for generating a visual video of an ophthalmic disease provided by an embodiment of the present invention, including:

[0053] S1. Obtain an initial visual description text in which the patient describes his own vision;

[0054] In a preferred embodiment of the present invention, the visual description text may be the speech-to-text dictated by the patient after watching a video, or the speech-to-text dictated by the patient about his own vision, or a questionnaire filled out by the patient. It should be noted that to avoid the limited expression ability of the patient, the visual description text may also be the text of the Q&A recording generated by the Q&A dialogue between the doctor and the patient.

[0055] S2. Input the initial visual description text into a preset large language model, so that the large language model combines a preset knowledge graph of visual symptoms of eye diseases, performs semantic analysis on the initial visual description text, and optimizes the initial visual description text according to the semantic analysis result to generate a visual description text recording the patient's dynamic visual scene;

[0056] Preferably, the large language model combines a preset knowledge graph of visual symptoms of eye diseases, performs semantic analysis on the initial visual description text, and optimizes the initial visual description text according to the semantic analysis result to generate a visual description text, including:

[0057] S21. Perform semantic analysis and dependency grammar analysis on the initial visual description text, and extract the symptom description and dynamic scene description in the initial visual description text;

[0058] S22. Determine a number of symptom medical terms according to the symptom description, match target symptom nodes from the knowledge graph of visual symptoms of eye diseases for each symptom medical term, and determine the typical dynamic features and associated scene features associated with the target symptom nodes; wherein, the knowledge graph of visual symptoms of eye diseases is a ternary relationship network of symptom nodes, associated scene features, and typical dynamic features;

[0059] S23. Perform text reconstruction according to the symptom medical terms, the dynamic scene description, the typical dynamic features, and the associated scene features to generate a visual description text recording the patient's dynamic visual scene.

[0060] Preferably, the performing text reconstruction according to the symptom medical terms, the dynamic scene description, the typical dynamic features, and the associated scene features to generate a visual description text recording the patient's dynamic visual scene includes:

[0061] S231. Perform dynamic scene analysis on the dynamic scene description to generate an analysis result;

[0062] S232. When the analysis result indicates that the dynamic scene description lacks specific scene elements, use the associated scene features to complement the dynamic scene description, and generate a visual description text according to the complemented dynamic scene description and the symptom medical terms;

[0063] S233. When the analysis result indicates that the dynamic scene description lacks dynamic time sequence elements, use the typical dynamic features to complement the dynamic scene description, and generate a visual description text according to the complemented dynamic scene description and the symptom medical terms;

[0064] S234. When the analysis result indicates that the dynamic scene description lacks specific scene elements and dynamic time-series elements, use the associated scene features and the typical dynamic features to complete the dynamic scene description, and generate a visual description text based on the completed dynamic scene description and the symptom medical terms.

[0065] In a preferred embodiment of the present invention, the large language model (LLM) is an advanced model in the field of artificial intelligence. Its core lies in mastering language rules and generating natural and fluent text through training with a large amount of text data. The structure of the knowledge graph of eye disease visual symptoms is a symptom-scene-dynamic ternary relationship network structure. For example, "floaters" is associated with the scene of "more obvious on a white background", and the dynamic attribute "drift with the rotation of the eyeball" is defined.

[0066] It should be further noted that the NER technology is used to extract symptom keywords (such as "black shadow", "distorted vision") in the patient's text and label them with medical standard terms. Analyze the time series (such as "appears in the morning and worsens in the afternoon"), spatial relationship (such as "the outer side of the left eye"), and triggering conditions (such as "suddenly occurs when turning the head") in the description as the dynamic scene description. Further, align the identified symptom entities with the nodes of the knowledge graph and retrieve their associated scenes and typical dynamic features. In this embodiment, potential missing information is also discovered through graph traversal. For example, the patient describes "ripples in front of the eyes" but does not mention the duration, and the system automatically associates the possible progressive deterioration feature of "retinal detachment". Further, when the analysis result indicates that the dynamic scene description lacks specific scene elements, insert the scene prompt words recommended by the knowledge graph into the original patient description, such as optimizing "seeing things bent" to "finding that straight lines are distorted when reading a book". When the analysis result indicates that the dynamic scene description lacks dynamic time-series elements, convert the static description into a dynamic process according to the typical dynamic features of the symptoms, such as adding "when looking to the left, the vision gradually blurs", etc. Finally, this embodiment also excludes contradictory descriptions through the taboo relationship in the graph (such as the conflict between "continuous flash sensation" and "transient migraine visual aura").

[0067] Further, the construction process of the knowledge graph of eye disease visual symptoms is as follows:

[0068] First, obtain detailed descriptions of various eye diseases and their symptoms from experts, organize the standardized descriptions in authoritative ophthalmology guidelines, collect subjective descriptions of their own symptoms from actual patients, clean and standardize the collected text data, and remove redundant information. Tag the text data. The detailed processes of tagging and feature extraction are as follows: According to the final eye disease diagnosis of the patients in the existing data, classify the subjective visual effect descriptions, such as: visual effects of glaucoma, visual effects of fundus diseases, visual effects of strabismus, etc.; refer to the guidelines and textbooks of the visual effects of each tagged disease, and extract the standard subjective description features of each tagged disease, such as: visual effects of glaucoma - rainbow vision, visual field defect; visual effects of fundus diseases - distorted vision, fixed dark shadow in front of the eyes; visual effects of strabismus - double vision, etc.; finally, organize the typical features, associated scenarios (such as in strong light, during exercise) and dynamic change patterns (such as over time, triggered by head movements) of visual symptoms (such as visual field defect, flash sensation).

[0069] S3. Input the visual description text into a pre-set text-to-video model, so that the text-to-video model outputs a subjective visual video that conforms to what the patient sees according to the visual description text.

[0070] Preferably, the text-to-video model outputs a subjective visual video that conforms to what the patient sees according to the visual description text, including:

[0071] S31. Generate a dynamic scene video according to the dynamic scene description in the visual description text.

[0072] S32. Extract the corresponding symptom materials from a pre-set symptom video material library according to the symptom medical terms in the visual description text.

[0073] S33. Integrate the symptom materials with the dynamic scene video to generate the subjective visual video.

[0074] In a preferred embodiment of the present invention, the text-to-video model first analyzes the input dynamic scene description through natural language processing techniques (such as word sense analysis, semantic understanding), extracts scene elements, combines the associated data sets of text, images, and videos, maps the key elements to visual elements, and then extracts dynamic descriptions to construct a video script, and then constructs a dynamic scene video based on the visual elements. Furthermore, based on the extracted symptom materials, the dynamic scene video is processed and edited.

[0075] Furthermore, the training steps of the video editing model are as follows:

[0076] Fine-tune the model using the CogVideo-based training framework. With the standardized text description of subjective vision and the corresponding video input, adjust the video output. Reduce the difference between the generated video and the video selected by the patient, making it easier for the model to generate videos that meet the expected effect and improving the efficiency of the patient's video selection. Compare the generated video with the feedback provided by the patient to evaluate the generation effect of the model and the matching degree with real patients. Based on the patient's feedback, preferably select "text description - video" data pairs to construct a high-quality secondary training dataset. By introducing new preferred data pairs, continue to fine-tune the model to improve the matching accuracy between the generated video and the patient's description. Continuously evaluate until the results generated by the model reach a high level of satisfaction in clinical applications. Further, the present invention can also be applied to the learning scenario of ophthalmology medical students to help students understand ophthalmic diseases.

[0077] This embodiment provides a method for generating a visual video of an ophthalmic disease. By obtaining the visual description text after the patient watches the initial video; inputting the initial visual description text into a preset large language model so that the large language model extracts and outputs several visual keywords from the visual description text; inputting the visual keywords and the initial video into a preset video editing model so that the video editing model edits the initial video according to the visual keywords and outputs a subjective visual video that conforms to what the patient sees. Therefore, the present invention can edit the initial video to output a subjective visual video that conforms to what the patient sees, thereby effectively helping doctors understand the patient's visual symptoms and improving the doctor's diagnosis efficiency.

[0078] See Figure 2 , which is a schematic structural diagram of a device for generating a visual video of an ophthalmic disease provided by an embodiment of the present invention, includes:

[0079] A text acquisition module for acquiring the initial visual description text in which the patient describes their own vision;

[0080] A text reconstruction module for inputting the initial visual description text into a preset large language model so that the large language model combines a preset knowledge graph of ophthalmic disease visual symptoms to perform semantic analysis on the initial visual description text, and optimizes the initial visual description text according to the semantic analysis result to generate a visual description text recording the patient's dynamic visual scene;

[0081] A video editing module for inputting the visual description text into a preset text-to-video model so that the text-to-video model outputs a subjective visual video that conforms to what the patient sees according to the visual description text.

[0082] Preferably, the language large model combines a preset knowledge graph of visual symptoms of eye diseases, performs semantic analysis on the initial visual description text, and optimizes the initial visual description text according to the semantic analysis result to generate a visual description text, including:

[0083] Performing semantic analysis and dependency grammar analysis on the initial visual description text, and extracting symptom descriptions and dynamic scene descriptions in the initial visual description text;

[0084] According to the symptom descriptions, determining a number of symptom medical terms, and for each symptom medical term, matching a target symptom node from the knowledge graph of visual symptoms of eye diseases, and determining typical dynamic features and associated scene features associated with the target symptom node; wherein, the knowledge graph of visual symptoms of eye diseases is a ternary relationship network of symptom nodes, associated scene features, and typical dynamic features;

[0085] Performing text reconstruction according to the symptom medical terms, the dynamic scene description, the typical dynamic features, and the associated scene features to generate a visual description text recording the patient's dynamic visual scene.

[0086] Preferably, the step of performing text reconstruction according to the symptom medical terms, the dynamic scene description, the typical dynamic features, and the associated scene features to generate a visual description text recording the patient's dynamic visual scene includes:

[0087] Performing dynamic scene analysis on the dynamic scene description to generate an analysis result;

[0088] When the analysis result indicates that the dynamic scene description lacks specific scene elements, using the associated scene features to complement the dynamic scene description, and generating a visual description text according to the complemented dynamic scene description and the symptom medical terms;

[0089] When the analysis result indicates that the dynamic scene description lacks dynamic time sequence elements, using the typical dynamic features to complement the dynamic scene description, and generating a visual description text according to the complemented dynamic scene description and the symptom medical terms;

[0090] When the analysis result indicates that the dynamic scene description lacks specific scene elements and dynamic time sequence elements, using the associated scene features and the typical dynamic features to complement the dynamic scene description, and generating a visual description text according to the complemented dynamic scene description and the symptom medical terms.

[0091] Preferably, the text-to-video model outputs a subjective visual video conforming to what the patient sees according to the visual description text, including:

[0092] Generate a dynamic scene video according to the dynamic scene description in the visual description text;

[0093] Extract the corresponding symptom materials from a preset symptom video material library according to the symptom medical terms in the visual description text;

[0094] Integrate the symptom materials with the dynamic scene video to generate the subjective visual video.

[0095] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that they have a communication connection, which can be specifically implemented as one or more communication buses or signal lines. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0096] Those skilled in the art can clearly understand that for the convenience and simplicity, the specific working process of the device described above can refer to the corresponding process in the foregoing method embodiment, and will not be repeated here.

[0097] Another preferred embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a method for generating a visual video of an ophthalmic disease as described in any one of the above embodiments.

[0098] The terminal device can be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server. The terminal device may include, but is not limited to, a processor and a memory.

[0099] The so-called processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The processor is the control center of the terminal device and connects various parts of the entire terminal device through various interfaces and lines.

[0100] The memory can be used to store the computer program. The processor realizes various functions of the terminal device by running or executing the computer program stored in the memory and calling the data stored in the memory. The memory may mainly include a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function, etc.; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory and may also include non-volatile memory, such as a hard disk, memory, plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, at least one magnetic disk storage device, flash device, or other volatile solid-state storage devices.

[0101] Another preferred embodiment of the present invention provides a storage medium, which is a computer-readable storage medium. The computer program is stored in the computer-readable storage medium. When the computer program is executed by the processor, the steps of the above various method embodiments can be realized. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc.

[0102] The above are the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements are also regarded as the protection scope of the present invention.

Claims

1. A method for generating visual videos of an ophthalmic disease, characterized in that, Including: Obtain the initial visual description text in which the patient describes their own vision; Input the initial visual description text into a pre-set large language model, so that the large language model combines a pre-set knowledge graph of visual symptoms of eye diseases to perform semantic analysis on the initial visual description text, and optimize the initial visual description text according to the semantic analysis results to generate a visual description text recording the patient's dynamic visual scene; Input the visual description text into a pre-set text-to-video model, so that the text-to-video model outputs a subjective visual video that conforms to what the patient sees according to the visual description text.

2. The visual video generation method for an ophthalmic disease according to claim 1, characterized in that, The large language model combines a pre-set knowledge graph of visual symptoms of eye diseases to perform semantic analysis on the initial visual description text, and optimize the initial visual description text according to the semantic analysis results to generate a visual description text, including: Perform semantic analysis and dependency grammar analysis on the initial visual description text to extract symptom descriptions and dynamic scene descriptions in the initial visual description text; Determine a number of symptom medical terms according to the symptom descriptions, match target symptom nodes from the knowledge graph of visual symptoms of eye diseases for each symptom medical term, and determine the typical dynamic features and associated scene features associated with the target symptom nodes; wherein, the knowledge graph of visual symptoms of eye diseases is a ternary relationship network of symptom nodes, associated scene features, and typical dynamic features; Perform text reconstruction according to the symptom medical terms, the dynamic scene description, the typical dynamic features, and the associated scene features to generate a visual description text recording the patient's dynamic visual scene.

3. The visual video generation method for an ophthalmic disease according to claim 2, wherein The performing text reconstruction according to the symptom medical terms, the dynamic scene description, the typical dynamic features, and the associated scene features to generate a visual description text recording the patient's dynamic visual scene includes: Perform dynamic scene analysis on the dynamic scene description to generate an analysis result; When the analysis result indicates that the dynamic scene description lacks specific scene elements, use the associated scene features to complement the dynamic scene description, and generate a visual description text according to the complemented dynamic scene description and the symptom medical terms; When the analysis result indicates that the dynamic scene description lacks dynamic time sequence elements, use the typical dynamic features to complement the dynamic scene description, and generate a visual description text according to the complemented dynamic scene description and the symptom medical terms; When the analysis result indicates that the dynamic scene description lacks specific scene elements and dynamic time sequence elements, use the associated scene features and the typical dynamic features to complement the dynamic scene description, and generate a visual description text according to the complemented dynamic scene description and the symptom medical terms.

4. The visual video generation method for an ophthalmic disease according to claim 3, characterized in that, The text-to-video model outputs a subjective visual video that conforms to what the patient sees according to the visual description text, including: Generate a dynamic scene video according to the dynamic scene description in the visual description text; Extract corresponding symptom materials from a pre-set symptom video material library according to the symptom medical terms in the visual description text; Integrate the symptom material with the dynamic scene video to generate the subjective visual video.

5. A visual video generation device for an ophthalmic disease, characterized in that, It includes: A text acquisition module for acquiring the initial visual description text in which the patient describes their own vision. A text reconstruction module for inputting the initial visual description text into a preset large language model, so that the large language model combines a preset knowledge graph of visual symptoms of eye diseases to perform semantic analysis on the initial visual description text, and optimizes the initial visual description text according to the semantic analysis result to generate a visual description text recording the patient's dynamic visual scene. A video editing module for inputting the visual description text into a preset text-to-video model, so that the text-to-video model outputs a subjective visual video that conforms to what the patient sees according to the visual description text.

6. The visual video generation device for an ophthalmic disease according to claim 5, wherein, The large language model combines a preset knowledge graph of visual symptoms of eye diseases to perform semantic analysis on the initial visual description text, and optimizes the initial visual description text according to the semantic analysis result to generate a visual description text, including: Perform semantic analysis and dependency grammar analysis on the initial visual description text, and extract the symptom description and dynamic scene description in the initial visual description text. According to the symptom description, determine several symptom medical terms, and for each symptom medical term, match a target symptom node from the knowledge graph of visual symptoms of eye diseases, and determine the typical dynamic features and associated scene features associated with the target symptom node; wherein, the knowledge graph of visual symptoms of eye diseases is a ternary relationship network of symptom nodes, associated scene features, and typical dynamic features. Perform text reconstruction according to the symptom medical terms, the dynamic scene description, the typical dynamic features, and the associated scene features to generate a visual description text recording the patient's dynamic visual scene.

7. The visual video generation device for an ophthalmic disease according to claim 6, characterized in that, The performing text reconstruction according to the symptom medical terms, the dynamic scene description, the typical dynamic features, and the associated scene features to generate a visual description text recording the patient's dynamic visual scene includes: Perform dynamic scene analysis on the dynamic scene description to generate an analysis result. When the analysis result indicates that the dynamic scene description lacks specific scene elements, use the associated scene features to complement the dynamic scene description, and generate a visual description text according to the complemented dynamic scene description and the symptom medical terms. When the analysis result indicates that the dynamic scene description lacks dynamic time sequence elements, use the typical dynamic features to complement the dynamic scene description, and generate a visual description text according to the complemented dynamic scene description and the symptom medical terms. When the analysis result indicates that the dynamic scene description lacks specific scene elements and dynamic time sequence elements, use the associated scene features and the typical dynamic features to complement the dynamic scene description, and generate a visual description text according to the complemented dynamic scene description and the symptom medical terms.

8. The visual video generation device for an ophthalmic disease according to claim 7, wherein The text-to-video model outputs a subjective visual video that conforms to what the patient sees according to the visual description text, including: Generate a dynamic scene video according to the dynamic scene description in the visual description text; Extract corresponding symptom materials from a preset symptom video material library according to the symptom medical terms in the visual description text; Integrate the symptom materials with the dynamic scene video to generate the subjective visual video.

9. A terminal device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements a method for generating a visual video of an ophthalmic disease as described in any one of claims 1 to 4.

10. A storage medium, characterized in that, The storage medium includes a stored computer program, wherein when the computer program runs, it controls the device where the storage medium is located to execute a method for generating a visual video of an ophthalmic disease as described in any one of claims 1 to 4.

Images