Visual information generation method and system based on document semantic retrieval

Abstract

The invention is suitable for the field of visual information processing, and provides a visual information generation method and system based on document semantic retrieval. According to the system, a document is obtained through a receiving module, a text ambiguity analysis module identifies a visual subject and description keywords thereof, semantic vectors are generated after ambiguity is detected and classified, a main semantic instruction set and adjustable intermediate state data are constructed through weight fusion and selection, and a first version of visual result is generated through a main vision generation module. When a user modifies a specific subject through the editing interface module, the interactive visual optimization module quickly generates and presents a local visual alternative scheme based on the adjustable intermediate state data, and a final result is exported by the output module after the user confirms the local visual alternative scheme. According to the method, semantic conflicts and fuzziness in the document are actively analyzed and managed before generation, and unadopted semantic paths are structured and archived, so that the consistency of a visual result and the intention of the document and the control force of a user on the generation process are effectively improved.

Description

Technical Field

[0001] This invention applies to the field of visual information processing and provides a method and system for generating visual information based on document semantic retrieval. Background Technology

[0002] Visual information generation based on document semantic retrieval refers to the use of natural language processing technology to extract and understand the core semantics of a given document, thereby driving a visual generation model to automatically create visual representations that match the document's semantics. This technology aims to achieve cross-modal conversion from text descriptions to visual content and can be widely applied in fields such as knowledge presentation, auxiliary explanations, process visualization, and interactive content generation.

[0003] Currently, the typical implementation process of visual information generation based on document semantic retrieval is as follows: First, the input document is parsed and semantically encoded, and the semantic vector representation or key information set of the document is obtained through a pre-trained language model; then, the semantic information is used as a condition input into the visual generation model to guide the model to generate images or graphics that conform to the text description.

[0004] Existing methods suffer from a fundamental flaw in the document semantic understanding stage: the lack of a mechanism to refine useful information from the original document text. Specifically, existing technologies input the entire document into an encoder and then encode it as a whole into a semantic vector and a set of key information. However, in real-world documents, descriptive text often contains redundant information, semantically ambiguous words, or logically conflicting limiting descriptions of the same visual subject. For example, a document may contain contradictory attribute descriptions of the same subject, or a complex limiting sentence may imply multiple possible stresses and semantic emphases. Existing technologies indiscriminately encode this inherently conflicting, ambiguous, or redundant textual information into uniform semantic conditions and directly input them into the visual generation model. This results in the model receiving "neutralized" data information, leading to inaccurate retrieved visual reference materials and inevitably biased final visual content. Therefore, how to effectively purify and semantically consistent the document text before semantic encoding, in order to eliminate noise, resolve conflicts, and clarify key points, and provide clean, clear, and consistent semantic guidance for subsequent retrieval and generation, is a key technical problem that has not been fully solved in the existing technology and seriously affects the quality of generation. Summary of the Invention

[0005] To address the aforementioned shortcomings, the present invention aims to provide a method and system for generating visual information based on document semantic retrieval, in order to solve the problems mentioned in the background art. The system includes:

[0006] The receiving module serves as the system's text data input port.

[0007] The text ambiguity analysis module is used to obtain all subjects and keywords describing the subjects; then, after ambiguity identification, classification, and vector transformation of keyword information, the vectors in each category after vectorization are weighted and fused to finally obtain the classification vectors, which are then selected to construct the main semantic instruction set and adjustable intermediate state data.

[0008] An alternative semantic database is used to store adjustable intermediate state data;

[0009] The main visual generation module is used to receive the main semantic instruction set, synthesize the first version of the visual result using the visual generation model, and extract adjustable intermediate data from the candidate semantic database and synchronously input it into the subsequent interactive visual optimization module.

[0010] The interactive visual optimization module responds to user commands to modify specific subjects in an image and quickly generates visual alternatives based on the alternative semantics of the original document.

[0011] The editing interface module provides users with a graphical user interface for interaction.

[0012] The output module is used to present the first version of the visual results and integrates a user editing panel.

[0013] Furthermore, the text ambiguity analysis module specifically includes:

[0014] Text structured units utilize natural language processing technology to identify all visual subjects in a document and multiple keywords describing each subject;

[0015] The ambiguity detection unit receives the output of the text structuring unit and detects whether there are conflicting semantics and / or ambiguous keywords among the descriptive keywords under the same visual subject;

[0016] The keyword vectorization unit is responsible for generating a corresponding semantic vector for each keyword; then, the semantic vector corresponding to each keyword is assigned to each classification group.

[0017] The semantic importance assessment unit is responsible for performing weight analysis on each semantic vector generated within each classification group;

[0018] The semantic vector fusion unit is responsible for fusing several semantic vectors within each classification group to generate a single classification semantic vector.

[0019] The classification vector selection unit is responsible for performing the final selection based on the importance scores of each classification semantic vector, forming the main semantic instruction set and adjustable intermediate state data.

[0020] Furthermore, the interactive visual optimization module specifically includes:

[0021] The request parsing unit receives user instructions from the editing interface module, which are the visual subjects to be modified.

[0022] The alternative data retrieval unit retrieves all adjustable intermediate data related to the subject from the alternative semantic database and presents it to the user editing panel of the editing interface module. The user can then select and invoke the data.

[0023] The component generation unit uses the visual generation model with adjustable intermediate state data as new generation conditions, and combines the potential space of the subject in the main visual generation module during the first generation to perform targeted local recalculation.

[0024] The preview synthesis unit seamlessly integrates the local recalculation results with the first version of the visual results through image synthesis technology, generating a replaced overall image result for users to preview in real time.

[0025] Furthermore, the ambiguity detection unit utilizes a pre-trained large language model; the ambiguity detection unit identifies multiple types of ambiguity based on preset rules, classifies keywords with synonymous substitution relationships into the same classification group, and keywords from multiple different classification groups describing the same subject are considered "ambiguous semantics" to each other, while keywords in the same classification group are considered "definite descriptions" to each other.

[0026] Furthermore, the semantic importance assessment unit assigns a semantic importance score to each vector by analyzing the location, frequency, and whether the keywords are modified by emphasis words in the document, as well as their semantic consistency with other "definite descriptions" of the subject.

[0027] Furthermore, the semantic vector fusion unit adopts a weighted fusion algorithm, which uses the "semantic importance" score obtained by each semantic vector in the semantic importance evaluation unit as its fusion weight, and performs a weighted average on the semantic vectors to calculate the final "classification semantic vector" of the classification group.

[0028] Furthermore, the classification vector selection unit selects the optimal classification semantic vector for each subject as the main semantic instruction, and transmits and stores the main semantic instructions of several subjects into a main semantic instruction set for subsequent generation of visual image data.

[0029] The classification vector selection unit packages and sends the classification semantic vector data corresponding to each classification group (excluding the main semantic instruction) to the candidate semantic database for archiving. The classification semantic vector data corresponding to each unselected classification group of each subject is used as "adjustable intermediate state data" to retain complete data information for subsequent interactive optimization.

[0030] A method and system for generating visual information based on document semantic retrieval, specifically including the following steps:

[0031] S1. The system receives the raw text document input by the user;

[0032] S2. The system parses the original text document to obtain all visual subjects and multiple keywords corresponding to each visual subject; through ambiguity detection based on preset rules, keywords with synonym substitution relationships are classified into the same category group; different categories are marked as mutually exclusive "ambiguous semantic" relationships.

[0033] S3. The system generates the main visual based on the main semantic instruction set; specifically, it includes the following steps:

[0034] S3.1 The main visual generation module receives the main semantic instruction set, uses it as a conditional input to the visual generation model, and synthesizes the complete first version of the visual result;

[0035] S3.2 During or after the generation process, the system establishes the association between each visual subject in the first version of the visual result and its corresponding adjustable intermediate data in the candidate semantic database, providing an entry point for subsequent optimization.

[0036] S4. Interactive visual optimization process; specifically including the following steps:

[0037] S4.1 The editing interface module displays the first version of the visual result to the user. When alternative data for the subject is detected, a visual prompt is provided through the user editing panel.

[0038] S4.2 When a user initiates an optimization request for a specific subject through the editing interface module, the interactive visual optimization module is activated, and the alternative data retrieval unit retrieves all associated adjustable intermediate state data from the alternative semantic database based on the subject identifier specified by the user.

[0039] S4.4 The component generation unit uses a lightweight editing model, taking the user-selected adjustable intermediate data as a condition, and combining the potential space of the subject in the main visual generation module during the first generation, to perform targeted local recalculation to obtain visual candidate results. Then, the visual candidate results are merged with the original image to generate a replacement preview effect.

[0040] S4.5 The user compares multiple alternative previews through the editing interface module and confirms the final selection. The system then composites the user-confirmed visual components into the final image.

[0041] Furthermore, step S2 includes the following sub-steps:

[0042] S2.1 Utilize text structured units to identify all visual subjects in the document through natural language processing technology, and extract multiple keywords describing each subject;

[0043] S2.2 The ambiguity detection unit scans and retrieves multiple keywords under the same visual subject, and classifies keywords with synonymous substitution relationships into the same classification group according to preset rules;

[0044] In this step, different keywords within the same category group are defined as having a "definite description" relationship;

[0045] S2.3 The keyword vectorization unit generates a semantic vector for each keyword and assigns it according to the classification group determined in step S2.2.

[0046] S2.4 The semantic importance assessment unit performs weight analysis on each semantic vector within each classification group;

[0047] S2.5 The semantic vector fusion unit takes each classification group as a unit, and uses a weighted fusion algorithm to fuse multiple vectors in the group into a single classification semantic vector based on the "semantic importance" score of each semantic vector in the group.

[0048] S2.6 The classification vector selection unit selects the one with the highest importance score from all its classification semantic vectors for each visual subject as the main semantic instruction; the main semantic instructions of all visual subjects are collected into the main semantic instruction set.

[0049] Furthermore, in step S2.6, the selection process for classification semantic vectors will perform the following processing on the unselected classification semantic vectors:

[0050] S2.6.1 The classification vector selection unit stores the classification semantic vector data of each subject that has not been selected as adjustable intermediate state data in the candidate semantic database.

[0051] Therefore, the beneficial effects of the present invention are as follows:

[0052] 1. Effectively solves the problem of generation deviation caused by semantic conflicts and ambiguity within the document.

[0053] Existing technologies encode documents containing contradictory, redundant, and ambiguous information as a whole, leading to inaccurate instructions received by the generative model. This invention proactively identifies and manages "ambiguous semantics" through a text ambiguity analysis module and uses a semantic importance assessment unit to quantitatively arbitrate conflicting descriptions. This ensures that the input to the generative model is a purified and conflict-free set of main semantic instructions, fundamentally avoiding visual generation biases caused by inherent contradictions in the instructions.

[0054] 2. Existing technologies output a single generated result. This invention introduces the fusion and selection of classification semantic vectors and stores all adjustable intermediate data in a candidate semantic database. This process not only ensures the quality of the first version of the visual result generated, but also makes the decision-making logic of the entire generation process traceable and modifiable.

[0055] 3. By collaborating with the alternative semantic database and the interactive visual optimization module, user modification requests are strictly limited to the alternative semantics already mentioned in the document. The component generation unit performs local recalculation, achieving precise replacement without requiring a full image redraw. This ensures that modifications do not deviate from the original meaning of the document, while greatly improving the optimization experience and efficiency through efficient local updates.

[0056] 4. Users can actively choose among multiple visual versions derived from the document through the editing interface module. This approach of transforming document ambiguity into a controllable interactive resource allows the final output to better match the user's detailed understanding of complex and ambiguous text, generating visual results that are more in line with their comprehension.

[0057] 5. The rules of the ambiguity detection unit and the model of the semantic importance evaluation unit of the present invention can be adapted and enhanced for different fields. They are not only applicable to general scenarios, but can also be easily embedded into more professional fields, and have wide applicability and scalability. Detailed Implementation

[0058] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0059] Existing visual information generation technologies typically rely solely on the overall document embedding vector and simple keyword matching. This invention differs from existing technologies by integrating semantic understanding and utilizing existing artificial intelligence techniques for vectorized classification, fusion, and selective application of semantic conflicts. This generates visual content with high semantic consistency and controllable detail, technically achieving the mapping from noisy text to multi-output visual instructions.

[0060] The purpose of this invention is to provide a visual information generation method and system based on document semantic retrieval. In general, this invention is based on text documents, performs in-depth analysis of the "conflicts" and "ambiguities" in the inherent semantics of the documents, and then performs processes such as classifying, selecting and fusing, selecting and generating, and pre-storing alternative data for conflicting items, and finally outputs an interactive visual generated data.

[0061] It should be understood that this invention needs to quantify and associate multiple descriptions of the same visual subject based on the two core concepts of "ambiguous emphasis words" and "semantic emphasis". In this way, in the final generation stage, through the adaptive generation process of main task generation, alternative path archiving and interactive optimization, the visual content of the document description is accurately and controllably visualized.

[0062] It's important to explain that the semantic conflicts and ambiguities upon which "ambiguous emphasis words" are based include, but are not limited to, conflicts in descriptions of single subject attributes, polysemy, unclear referents, and ambiguous modification relationships—various forms of textual uncertainty. Furthermore, ambiguity not only represents different interpretations based on word meaning, but also includes conflicting attribute descriptions such as color, size, and style under the same subject, as well as situations where attribution is ambiguous due to unclear referents; it also includes suggestive hints of optionality in descriptions, such as alternative descriptions guided by words like "or," "maybe," and "perhaps." The system thus drives the ambiguity management process, thereby constructing a classification of alternative semantics.

[0063] Based on the concept of "ambiguous emphasis words" mentioned above, it is important to explain that since this system generates several vectors based on different semantic meanings under keywords as input for the final result, this invention utilizes a "semantic emphasis" quantification model to select and model the fused semantic vectors after classifying each keyword. In other words, in the initial document, after ambiguity classification, the system introduces an evaluation emphasis on the semantic vectors of each category for each keyword describing the subject. Thus, based on the original document data, it can select the semantic interpretation that should be prioritized from several ambiguous and conflicting descriptions and input it into the visual generation model.

[0064] More specifically, the introduction of the "semantic importance" quantification model is based on comprehensive evaluation rules, such as statistical features like word frequency, position, and total number, syntactic features like emphasis words, contextual consistency features, and effectiveness features. The semantic importance quantification model captures the relative importance of current candidate semantics within the document context. By learning the document's own descriptive logic, it processes and generates importance data for keywords after each category. The comprehensive evaluation rules upon which the semantic importance quantification model of this invention relies are a method for quantifying the relative importance of semantic units (such as words and phrases) in a text within a specific context through multi-dimensional statistical and linguistic feature analysis, which is existing technology. The system of this invention uses the "semantic importance" quantification model to create vector labels associated with multiple possible descriptions in the semantic dimension for each key visual subject in the document. These labels not only contain basic semantic vectors based on a pre-trained model but also assign weights based on other statistical data of the description in the document, thus deriving weighted semantic vectors.

[0065] For the final generation of a visual subject with multiple possible descriptions, the above weighted semantic vectors are used to arbitrate before generation, selecting the vector with the highest weight as the main path basis, while all candidate vectors are structured and archived.

[0066] In other words, the initial generation of the visual image is based on the arbitrated main semantic path. In addition to the preview data of each subject image, it also includes alternative data for each subject image. The system will then support local corrections to the main path generation result during the interactive phase based on the archived alternative semantic vectors.

[0067] During the correction phase, the system receives the user's modification instructions for a specific subject in the generated image, and uses the image editing engine to retrieve all candidate semantic vectors corresponding to that subject from the candidate semantic database.

[0068] During the interactive optimization phase, users can manipulate the image editing engine to locally regenerate a specific subject or directly replace the subject without altering other parts of the image.

[0069] Therefore, to ensure that the final visual output aligns with the user's detailed understanding of the document, the system uses alternative semantic vectors from the original document as the basis for modification, ensuring that all visual changes remain within the scope of the document.

[0070] Finally, after the main path is generated and optional interactive optimizations are performed, the system outputs visual content that is highly consistent with the semantics of the document and has been confirmed by the user. The format can be adapted to various application scenarios such as report illustrations, concept design diagrams, and educational visualization materials.

[0071] In conclusion, this invention, based on "ambiguous emphasis words" and "semantic emphasis," ensures the controllability and explorability of the visual generation process. It governs the generation of the main visual and potential interactive optimizations under the same set of semantic rules derived from the original document, which have been evaluated and categorized. This guarantees the consistency of the output content in its core semantics while preserving the ability to explore the diversity of details.

[0072] Therefore, at the system level, this invention provides a method and system for generating visual information based on document semantic retrieval, the system comprising:

[0073] The system comprises a receiving module, a text ambiguity analysis module, a main visual generation module, a candidate semantic database, an interactive visual optimization module, an output module, and an editing interface module. Through the collaborative work of these modules, a complete process from inputting noisy documents to generating and optimizing accurate and controllable visual content is achieved.

[0074] The receiving module serves as the system's text data input port, responsible for receiving raw text documents so that they can be transmitted to the subsequent ambiguity management module for semantic information extraction, ambiguity recognition, and other processes.

[0075] The text ambiguity analysis module first performs a text structure analysis process on the received text data to obtain all subjects and keyword information describing the subjects; then it performs ambiguity identification on several keywords of each subject, then performs vector transformation on the keyword information, then performs weight fusion on the vectorized vectors of each category, and finally selects the vectors of each category.

[0076] Specifically, to implement the above process of the text ambiguity analysis module, the text ambiguity analysis module includes a text structuring unit, an ambiguity detection unit, a keyword vectorization unit, a semantic importance assessment unit, and a classification vector selection unit.

[0077] The text structuring unit uses natural language processing technology to parse the input document, identify all visual subjects in the document such as objects, people, and scenes, and identify multiple keywords describing each subject.

[0078] The ambiguity detection unit receives the output of the text structuring unit and detects whether there are conflicting semantics and / or ambiguous keywords among descriptive keywords under the same visual subject. The ambiguity detection unit utilizes a pre-trained large language model to capture the specific meaning of words in the document context. Based on preset rules, the ambiguity detection unit identifies various types of ambiguity, such as attribute conflicts, polysemy, and multiple optional descriptions represented by words like "or" and "possibly." Thus, the ambiguity detection unit categorizes keywords with synonymous substitution relationships into the same classification group; keywords from multiple different classification groups describing the same subject are considered "ambiguous semantics," while keywords within the same classification group are considered "definite descriptions."

[0079] The keyword vectorization unit is responsible for generating a corresponding semantic vector for each keyword. Then, the semantic vector for each keyword is assigned to each category group, and the vectors are transmitted to the semantic importance assessment unit for importance assessment on a group-by-group basis.

[0080] The semantic importance assessment unit performs authoritative quantitative evaluation and analysis, i.e., weight analysis, on each semantic vector generated within each classification group. The unit assigns a semantic importance score to each vector by analyzing the keyword's position and frequency in the document, whether it is modified by an emphasis word, and its semantic consistency with other "definite descriptions" of the subject. This score characterizes the relative importance and authenticity of the semantic interpretation within the document context. Subsequently, the semantic importance assessment unit sends the weighted semantic vectors from each classification group to the semantic vector fusion unit.

[0081] The semantic vector fusion unit is responsible for processing the vectors of all classification groups output from the semantic importance assessment unit. In other words, it fuses several semantic vectors within each classification group to generate a single, representative "classification semantic vector".

[0082] Specifically, for each classification group, the semantic vector fusion unit uses a weighted fusion algorithm. It uses the "semantic importance" score obtained by each semantic vector in the semantic importance assessment unit as its fusion weight, and performs a weighted average of the semantic vectors to calculate the final "classification semantic vector" for that classification group.

[0083] This results in several ambiguous "classification semantic vectors" under the same subject, simplifying and synthesizing all possible weighted semantic information in each ambiguous direction of the keyword. This process is not simply extracting a vector from multiple ambiguities that is closer to the average of all semantics, as is done in existing technologies, nor is it a vector obtained by weighting and averaging all semantic vectors. This makes the pre-input information closer to the true meaning expressed in the original document. It is worth noting that the above vector weighting fusion algorithm is a common method existing in this field and will not be elaborated upon here.

[0084] The classification vector selection unit performs the final selection based on the importance score of each classification semantic vector, that is, it selects the optimal classification semantic vector for each subject as the main semantic instruction. The main semantic instructions for several subjects are transmitted and stored as a main semantic instruction set for subsequent generation of visual image data. Simultaneously, the classification vector selection unit packages the classification semantic vector data corresponding to each classification group (excluding the main semantic instruction) and sends it to the candidate semantic database for archiving. The classification semantic vector data corresponding to each unselected classification group for each subject is treated as "adjustable intermediate data," preserving complete data information for subsequent interactive optimization.

[0085] The main visual generation module receives the conflict-free main semantic instruction set output by the text ambiguity analysis module, and uses the visual generation model to synthesize a complete first version of the visual result that conforms to all descriptions. After the initial generation, the main visual generation module extracts "adjustable intermediate state data" corresponding to each visual subject in the final image from the candidate semantic database, and synchronously inputs it into the subsequent interactive visual optimization module.

[0086] Specifically, the main visual generation module uses a visual generation model to generate the final visual result data for each subject. Then, each subject is represented on the image, and the generated result data for each subject is bound to a specific area within the image. This provides the technical basis for replacing only specific subjects without redrawing the entire image. The above steps are based on a condition-guided visual generation model and its potential spatial operations, which are existing technologies and will not be elaborated upon here.

[0087] The alternative semantic database is the system's storage database, used to structurally store all classification semantic vector data that were not selected in the classification vector selection unit of the ambiguity management module. Each record is uniquely associated with a visual subject and an alternative semantic description. The alternative semantic database ensures that every modification suggestion proposed in the subsequent interaction optimization phase originates from semantics already expressed in the original document but not adopted by the main path, thereby preventing subsequent modifications from arbitrarily generating content outside the document.

[0088] The interactive visual optimization module is the core module of the system for enabling user-oriented, fine-tuned visual adjustments. After the first version of the visual result is generated, the interactive visual optimization module is in a standby state, waiting for user commands and responding to user commands to modify specific subjects in the image, quickly generating visual alternatives based on the alternative semantics of the original document.

[0089] In conjunction with the interactive visual optimization module, the editing interface module is the graphical user interface through which the system interacts with the user.

[0090] The editing interface module is responsible for presenting the first version of the visual result and integrates a user editing panel. The core function of the user editing panel is to visualize the visual result and provide an interactive interface for the user. When the system has several visual subjects with corresponding adjustable intermediate state data in the candidate semantic database, the user editing panel can provide clear visual cues or other prompts next to that subject, such as listing a separate list of candidate tasks in the editing interface. Users can trigger optimization requests for specified subjects through the user editing panel, browse the candidate visual previews generated in real time by the interactive visual optimization module, and perform operations such as replacement, regeneration, and semantic adjustment. During the editing process, all user operations are recorded as a traceable editing history. Simultaneously, the system allows users to freely switch and select between various visual versions generated based on different classification semantic vectors until the user selects the final image that best matches their understanding.

[0091] Thus, the collaboration between the editing interface module and the interactive visual optimization module transforms complex semantic ambiguity management into an intuitive and controllable user interaction experience.

[0092] Specifically, the interactive visual optimization module includes a request parsing unit, a candidate data retrieval unit, a component generation unit, and a preview compositing unit. The specific workflow of these units is as follows:

[0093] The request parsing unit receives user instructions from the editing interface module, which typically specify the visual subject to be modified.

[0094] Based on the subject identifier sent by the request parsing unit, the alternative data retrieval unit accurately retrieves all adjustable intermediate data associated with the subject from the alternative semantic database, that is, retrieves the classification semantic vector data corresponding to each unselected classification group, presents the link to the user editing panel of the editing interface module, and calls it after the user selects it, and sends it to the subsequent component generation unit.

[0095] The component generation unit utilizes a visual generation model with a specific adjustable intermediate state data retrieved by the candidate data retrieval unit as a new generation condition. This data, combined with the potential space of the subject retained in the main visual generation module during the initial generation, is used for targeted local recalculation. This process reconstructs only the target subject, efficiently generating visual candidate results for that subject while leaving the rest of the image unchanged.

[0096] The preview synthesis unit receives the visual alternatives output by the component generation unit and seamlessly integrates them with the first version of the visual results through image synthesis technology to generate a "replaced overall image result" for users to preview in real time.

[0097] The output module receives the first version of the visual result from the main visual generation module, or the final optimized version confirmed by the user from the interactive visual optimization module. It encodes and encapsulates the visual data stream processed internally by the system into image or video files that conform to industry standards or user-specified formats, thus completing the final export.

[0098] Therefore, based on the collaboration of all the above modules, this system transforms the inherent uncertainty in document semantic understanding from a burden on the generative model into an interactive optimization resource that is manageable by the system and can be participated in by the user. Ultimately, while ensuring the availability of the first generation, it significantly improves the final accuracy and user controllability of complex document visualization.

[0099] Based on the above system, this invention proposes a visual information generation method based on document semantic retrieval, the method comprising the following steps:

[0100] S1. Receive the original document; the system receives the original text document input by the user.

[0101] S2. The system parses the original text document to obtain all visual subjects and multiple keywords corresponding to each visual subject; through ambiguity detection based on preset rules, keywords with synonymous substitution relationships are classified into the same category group; different multiple category groups are marked as mutually exclusive "ambiguous semantic" relationships. This step aims to generate the main visual generation instruction and alternative files based on the noisy text.

[0102] Specifically, this step corresponds to the text ambiguity analysis module in the system activating its text structuring unit to parse the original text document, identify all visual subjects in the document, extract multiple keywords describing each visual subject, and then classify the keywords. This includes the following sub-steps:

[0103] S2.1 Text structuring process: Using text structuring units, natural language processing technology is used to identify all visual subjects in the document and extract multiple keywords describing each subject.

[0104] S2.2 Classification process based on ambiguity detection: The ambiguity detection unit scans and retrieves multiple keywords under the same visual subject, detects semantic conflicts and fuzzy semantic queries according to preset rules, and classifies keywords with synonym substitution relationships into the same classification group;

[0105] In this step, different keywords within the same category group are defined as having a "definite description" relationship; multiple category groups with different visual semantics are defined as having a mutually exclusive "ambiguous semantic" relationship.

[0106] S2.3 Keyword Vectorization Process: The keyword vectorization unit generates a corresponding semantic vector for each keyword and assigns it according to the classification group affiliation in S2.2. The assignment process is as follows:

[0107] The keyword vectorization unit calculates the semantic vector for each keyword, and then assigns the output to the category group to which the keyword belongs. After the semantic vectors of all keywords in all category groups are calculated, they are transmitted to the next unit for processing in groups.

[0108] S2.4 Semantic Importance Assessment Process: Within each classification group, the semantic importance assessment unit performs weight analysis on each semantic vector. In this step, by analyzing the keyword's occurrence position, frequency, emphasis, and consistency with other "definite descriptions" in the document, a "semantic importance" score is assigned to the semantic vector of each keyword.

[0109] S2.5, Generating Classification Semantic Vectors: The semantic vector fusion unit takes each classification group as a unit, and uses a weighted fusion algorithm based on the "semantic importance" score of each semantic vector in the group to fuse multiple vectors in the group into a single classification semantic vector that represents the overall semantics of the group.

[0110] This operation differs from the simple global averaging or random selection of all ambiguous vectors in existing technologies. Instead, it preserves a weighted, independent semantic representation based on the ambiguous direction.

[0111] S2.6 Selection process of classification semantic vectors: For each visual subject, the classification vector selection unit selects the one with the highest importance score from all its classification semantic vectors as the main semantic instruction. The main semantic instructions of all visual subjects are collected into the main semantic instruction set.

[0112] Meanwhile, the selection of classification semantic vectors will process the unselected classification semantic vectors as follows:

[0113] S2.6.1 The classification vector selection unit stores the classification semantic vector data of each subject that has not been selected as adjustable intermediate state data in the candidate semantic database.

[0114] S3. The system generates the main visual based on the main semantic instruction set. This includes the following steps:

[0115] S3.1 The main visual generation module receives the main semantic instruction set, uses it as a condition input to the visual generation model, and synthesizes the complete first version of the visual result.

[0116] S3.2 During or after the generation process, the system establishes the association between each visual subject in the first version of the visual result and its corresponding adjustable intermediate data in the candidate semantic database, providing an entry point for subsequent optimization.

[0117] S4. Interactive visual optimization process. This includes the following steps:

[0118] S4.1 The editing interface module displays the first version of the visual results to the user. When alternative data for a subject is detected, a visual prompt is provided through the user editing panel.

[0119] S4.2 When a user initiates an optimization request for a specific subject through the editing interface module, the interactive visual optimization module is activated, and the alternative data retrieval unit retrieves all associated adjustable intermediate state data from the alternative semantic database based on the subject identifier specified by the user.

[0120] S4.4 The component generation unit uses a lightweight editing model, taking a user-selected adjustable intermediate state data as a condition, and combines the potential space of the subject in the main visual generation module during the first generation to perform targeted local recalculation to obtain visual candidate results, while the rest of the image remains unchanged. Subsequently, the preview synthesis unit merges the visual candidate results with the original image to generate a replacement preview effect.

[0121] Therefore, this step enables efficient local visualization exploration based on document alternative semantics.

[0122] S4.5 Confirmation Phase: The user compares multiple alternative previews through the editing interface module and confirms the final selection. The system then composites the user-confirmed visual components into the final image.

[0123] S5. Output the final visual content. The output module will encode and encapsulate the final visual result confirmed by the user, or directly use the first version of the visual result, into a standard format image or video file and output it.

[0124] In summary, in practical application, this system first acquires the original text document through the receiving module; then, the text ambiguity analysis module performs structured parsing, ambiguity detection, and classification on the document, generating and fusing classification semantic vectors for each classification group through the keyword vectorization unit, semantic importance evaluation unit, and semantic vector fusion unit; subsequently, the classification vector selection unit selects the optimal classification semantic vector based on semantic importance to form the main semantic instruction set, and stores the unselected vectors as adjustable intermediate state data in the candidate semantic database; the main visual generation module generates the first version of the visual result based on the main semantic instruction set; users can trigger optimization requests for specific subjects through the editing interface module, and the interactive visual optimization module generates candidate visual components based on the adjustable intermediate state data for users to preview and select; finally, the output module exports the visual content confirmed by the user.

[0125] Therefore, this method proactively identifies, quantifies, and arbitrates document conflicts and ambiguous descriptions before generation by introducing ambiguity detection and semantic importance assessment mechanisms. It also uses a candidate semantic database to structurally archive unadopted semantic paths. Furthermore, by combining an interactive visual optimization module, users can make localized and precise visual adjustments based on the candidate semantics of the original document after generation. This solution effectively overcomes the semantic bias problem caused by directly encoding noisy text in existing technologies. While ensuring the usability of the first round of generation, it significantly improves the consistency between the visual results and the original meaning of the document, as well as the user's control over the generation process.

[0126] Of course, the present invention may have other various embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and modifications according to the present invention, but these corresponding changes and modifications should all fall within the protection scope of the appended claims.

Claims

1. A visual information generation system based on document semantic retrieval, characterized in that, include: The receiving module serves as the system's text data input port. The text ambiguity analysis module is used to derive all subjects and the keywords that describe those subjects; Subsequently, after ambiguity identification, classification, and vectorization of keyword information, the vectors in each category after vectorization are weighted and fused to obtain the final classification vectors. The resulting vectors are then selected to construct the main semantic instruction set and adjustable intermediate state data. An alternative semantic database is used to store adjustable intermediate state data; The main visual generation module is used to receive the main semantic instruction set, synthesize the first version of the visual result using the visual generation model, and extract adjustable intermediate data from the candidate semantic database and synchronously input it into the subsequent interactive visual optimization module. The interactive visual optimization module responds to user commands to modify specific subjects in an image and quickly generates visual alternatives based on the alternative semantics of the original document. The editing interface module provides users with a graphical user interface for interaction. The output module is used to present the first version of the visual results and integrates a user editing panel.

2. The visual information generation system based on document semantic retrieval according to claim 1, characterized in that, The text ambiguity analysis module specifically includes: Text structured units utilize natural language processing technology to identify all visual subjects in a document and multiple keywords describing each subject; The ambiguity detection unit receives the output of the text structuring unit and detects whether there are conflicting semantics and / or ambiguous keywords among the descriptive keywords under the same visual subject; The keyword vectorization unit is responsible for generating a corresponding semantic vector for each keyword; then, the semantic vector corresponding to each keyword is assigned to each classification group. The semantic importance assessment unit is responsible for performing weight analysis on each semantic vector generated within each classification group; The semantic vector fusion unit is responsible for fusing several semantic vectors within each classification group to generate a single classification semantic vector. The classification vector selection unit is responsible for performing the final selection based on the importance scores of each classification semantic vector, forming the main semantic instruction set and adjustable intermediate state data.

3. The visual information generation system based on document semantic retrieval according to claim 1, characterized in that, The interactive visual optimization module specifically includes: The request parsing unit receives user instructions from the editing interface module, which are the visual subjects to be modified. The alternative data retrieval unit retrieves all adjustable intermediate data related to the subject from the alternative semantic database and presents it to the user editing panel of the editing interface module. The user can then select and invoke the data. The component generation unit uses the visual generation model with adjustable intermediate state data as new generation conditions, and combines the potential space of the subject in the main visual generation module during the first generation to perform targeted local recalculation. The preview synthesis unit seamlessly integrates the local recalculation results with the first version of the visual results through image synthesis technology, generating a replaced overall image result for users to preview in real time.

4. The visual information generation system based on document semantic retrieval according to claim 2, characterized in that, The ambiguity detection unit utilizes a pre-trained large language model. Based on preset rules, the ambiguity detection unit identifies various types of ambiguity, categorizes keywords with synonymous substitution relationships into the same classification group, identifies keywords from multiple different classification groups that describe the same subject as "ambiguous semantics," and identifies keywords from the same classification group as "definite descriptions." 5. The visual information generation system based on document semantic retrieval according to claim 2, characterized in that, The semantic importance assessment unit assigns a semantic importance score to each vector by analyzing the location and frequency of keywords in the document, whether they are modified by emphasis words, and their semantic consistency with other "definite descriptions" of the subject.

6. The visual information generation system based on document semantic retrieval according to claim 2, characterized in that, The semantic vector fusion unit uses a weighted fusion algorithm, which takes the "semantic importance" score obtained by each semantic vector in the semantic importance assessment unit as its fusion weight, and calculates the final "classification semantic vector" of the classification group by weighted averaging of the semantic vectors.

7. The visual information generation system based on document semantic retrieval according to claim 2, characterized in that, The classification vector selection unit selects the optimal classification semantic vector for each subject as the main semantic instruction, and transmits and stores the main semantic instructions of several subjects into a main semantic instruction set for subsequent generation of visual image data. The classification vector selection unit packages and sends the classification semantic vector data corresponding to each classification group (excluding the main semantic instruction) to the candidate semantic database for archiving. The classification semantic vector data corresponding to each unselected classification group of each subject is used as "adjustable intermediate state data" to retain complete data information for subsequent interactive optimization.

8. A method for generating visual information based on document semantic retrieval, characterized in that, The visual information generation system based on document semantic retrieval as described in claim 1 specifically includes the following steps: S1. The system receives the raw text document input by the user; S2. The system parses the original text document to obtain all visual subjects and multiple keywords corresponding to each visual subject; through ambiguity detection based on preset rules, keywords with synonym substitution relationships are classified into the same category group; different categories are marked as mutually exclusive "ambiguous semantic" relationships. S3. The system generates the main visual based on the main semantic instruction set; specifically, it includes the following steps: S3.1 The main visual generation module receives the main semantic instruction set, uses it as a conditional input to the visual generation model, and synthesizes the complete first version of the visual result; S3.2 During or after the generation process, the system establishes the association between each visual subject in the first version of the visual result and its corresponding adjustable intermediate data in the candidate semantic database, providing an entry point for subsequent optimization. S4. Interactive visual optimization process; specifically including the following steps: S4.1 The editing interface module displays the first version of the visual result to the user. When alternative data for the subject is detected, a visual prompt is provided through the user editing panel. S4.2 When a user initiates an optimization request for a specific subject through the editing interface module, the interactive visual optimization module is activated, and the alternative data retrieval unit retrieves all associated adjustable intermediate state data from the alternative semantic database based on the subject identifier specified by the user. S4.4 The component generation unit uses a lightweight editing model, taking the user-selected adjustable intermediate data as a condition, and combining the potential space of the subject in the main visual generation module during the first generation, to perform targeted local recalculation to obtain visual candidate results. Then, the visual candidate results are merged with the original image to generate a replacement preview effect. S4.5 The user compares multiple alternative previews through the editing interface module and confirms the final selection. The system then composites the user-confirmed visual components into the final image.

9. The visual information generation method based on document semantic retrieval according to claim 8, characterized in that, Step S2 includes the following sub-steps: S2.1 Utilize text structured units to identify all visual subjects in the document through natural language processing technology, and extract multiple keywords describing each subject; S2.2 The ambiguity detection unit scans and retrieves multiple keywords under the same visual subject, and classifies keywords with synonymous substitution relationships into the same classification group according to preset rules; In this step, different keywords within the same category group are defined as having a "definite description" relationship; S2.3 The keyword vectorization unit generates a semantic vector for each keyword and assigns it according to the classification group determined in step S2.

2. S2.4 The semantic importance assessment unit performs weight analysis on each semantic vector within each classification group; S2.5 The semantic vector fusion unit takes each classification group as a unit, and uses a weighted fusion algorithm to fuse multiple vectors in the group into a single classification semantic vector based on the "semantic importance" score of each semantic vector in the group. S2.6 The classification vector selection unit selects the one with the highest importance score from all its classification semantic vectors for each visual subject as the main semantic instruction; the main semantic instructions of all visual subjects are collected into the main semantic instruction set.

10. The visual information generation method based on document semantic retrieval according to claim 9, characterized in that, In step S2.6, the selection process for classification semantic vectors will perform the following processing on the unselected classification semantic vectors: S2.6.1 The classification vector selection unit stores the classification semantic vector data of each subject that has not been selected as adjustable intermediate state data in the candidate semantic database.