Multimodal document association updating method, electronic device, and program product

Abstract

The present disclosure provides a multi-modal document association updating method, an electronic device and a program product. The method of the present disclosure comprises: obtaining a plurality of product research and development documents of different modalities; extracting semantic concepts of each component element in the product research and development documents, and assigning semantic identifiers to the component elements; establishing a cross-modality semantic binding relationship using the semantic concepts and the semantic identifiers; constructing a cross-modality dependency graph based on the semantic binding relationship and the dependency relationship between the component elements; in response to a change request for a first component element of a first product research and development document in the plurality of different modalities, determining at least one second component element affected in the cross-modality dependency graph; performing an update operation on a second product research and development document to which the second component element belongs.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more particularly to a multimodal document association and update method, electronic device and program product. Background Technology

[0002] In the development of modern internet products, product requirements documents (PRDs), business process diagrams, interface prototypes, and other product development documents are important carriers for comprehensively expressing the product's business logic. They clarify the product's functions and constraints from different dimensions such as text description, flow logic, and interactive visuals, and serve as the benchmark for cross-role collaboration, development execution, and acceptance testing during the development phase.

[0003] In traditional multi-product development document collaborative maintenance scenarios, product managers or designers often rely on independent writing and manual synchronization of deliverables from different modalities. When facing complex business systems or frequent requirement changes, this manual approach is prone to various problems. For example, the lack of underlying semantic connections between product development documents leads to a massive and extremely tedious workload for synchronization modifications; manual verification is prone to omissions; and discrepancies arise in the descriptions of the same function across different modalities, resulting in logical gaps or semantic inconsistencies. Therefore, a simple and efficient solution is needed to achieve comprehensive updates of multi-product development documents. Summary of the Invention

[0004] This disclosure provides a method for updating multimodal document association, as well as electronic devices and program products.

[0005] According to a first aspect of this disclosure, a multimodal document association and update method is provided. The method specifically includes: acquiring multiple product development documents of different modalities; extracting semantic concepts of each component element in the product development documents and assigning semantic identifiers to the component elements; establishing cross-modal semantic binding relationships using the semantic concepts and semantic identifiers; constructing a cross-modal dependency graph based on the semantic binding relationships and the dependencies between component elements; in response to a change request for a first component element of a first product development document in multiple different modalities, determining at least one affected second component element in the cross-modal dependency graph; and performing an update operation on the second product development document to which the second component element belongs.

[0006] In at least one embodiment of this disclosure, the semantic concepts of each component element in the product development document are extracted, and semantic identifiers are assigned to the component elements, including: identifying the component elements in different product development documents; extracting the semantics of the component elements through a semantic analysis model to obtain semantic concepts that characterize the corresponding functional attributes of the component elements; and assigning a globally unique semantic identifier to each component element.

[0007] In at least one embodiment of this disclosure, establishing cross-modal semantic binding relationships using semantic concepts and semantic identifiers includes: constructing a semantic binding table; determining globally unique semantic identifiers corresponding to multiple different constituent elements having the same semantic concept in multiple different modalities; and establishing semantic binding relationships between semantic identifiers of multiple different modalities and the same semantic concept in the semantic binding table.

[0008] In at least one embodiment of this disclosure, a cross-modal dependency graph is constructed based on semantic binding relationships and dependencies between constituent elements, including: identifying dependencies between constituent elements, wherein the dependencies include: semantic dependencies, process dependencies, interaction dependencies, and constraint dependencies; and constructing a cross-modal dependency graph with semantic identifiers as nodes and dependencies as edges.

[0009] In at least one embodiment of this disclosure, in response to a change request for a first component element of a first product development document in multiple different modalities, determining at least one affected second component element in a cross-modal dependency graph includes: obtaining a first semantic identifier corresponding to the first component element; traversing the cross-modal dependency graph starting from the first semantic identifier to determine at least one second semantic identifier that is reachable from the first semantic identifier through a directed path; and determining the component element corresponding to the at least one second semantic identifier as a second component element.

[0010] In at least one embodiment of this disclosure, traversing a cross-modal dependency graph starting from a first semantic identifier to determine at least one second semantic identifier that is reachable from the first semantic identifier via a directed path includes: obtaining the edge type and weight representing the dependency strength of each edge in the cross-modal dependency graph; filtering candidate connection edges during the traversal process based on the weight and edge type; and determining the node corresponding to the filtered connection edge as at least one second semantic identifier.

[0011] In at least one embodiment of this disclosure, performing an update operation on a second product development document to which the second component element belongs includes: obtaining the changed content of the first component element; locating a target editing position in the second product development document based on the semantic identifier of the second component element; generating update data adapted to the second product development document according to the type of the changed content and the semantic binding relationship; and applying the update data at the target editing position.

[0012] In at least one embodiment of this disclosure, applying updated data to the target editing location includes: determining an update strategy, wherein the update strategy includes at least one of conflict marking, synchronous modification suggestion, and automatic patching; when the update strategy is conflict marking, highlighting the target editing location in the second product development document; and / or, when the update strategy is synchronous modification suggestion, generating modification prompt text associated with the target editing location; and / or, when the update strategy is automatic patching, directly applying the updated data to the target editing location.

[0013] In at least one embodiment of this disclosure, after applying updated data to the target editing location, the method further includes: performing consistency checks on multiple updated product development documents of different modalities after applying the updated data; verifying whether the updated constituent elements match the semantic concepts; verifying whether the business process connections are complete and whether there are input / output conflicts; verifying whether the triggering relationship between the interface controls and the backend logic corresponds; and verifying whether the permission control, state transition, or exception prompts are consistent with the new functions.

[0014] According to a second aspect of this disclosure, an electronic device is provided, comprising: a memory storing execution instructions; and a processor executing the execution instructions stored in the memory, such that the processor performs a first aspect of any embodiment of this disclosure.

[0015] According to a third aspect of this disclosure, a readable storage medium is provided, wherein executable instructions are stored therein, which, when executed by a processor, are used to implement a first aspect of any embodiment of this disclosure.

[0016] According to a fourth aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements a first aspect of any embodiment of this disclosure. Attached Figure Description

[0017] The accompanying drawings illustrate exemplary embodiments of the present disclosure and, together with the description thereof, serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0018] Figure 1 This is a flowchart illustrating the multimodal document association and update method provided in this embodiment of the disclosure.

[0019] Figure 2 This is a flowchart illustrating the semantic identifier generation method provided in an embodiment of this disclosure.

[0020] Figure 3 A flowchart illustrating the semantic binding relationship establishment method provided in this embodiment of the disclosure.

[0021] Figure 4 This is a flowchart illustrating the cross-modal dependency graph construction method provided in this embodiment of the disclosure.

[0022] Figure 5 This is a flowchart illustrating the cross-modal element modification method provided in this embodiment of the disclosure.

[0023] Figure 6 This is a flowchart illustrating the cross-modal traversal method provided in an embodiment of this disclosure.

[0024] Figure 7 This is a schematic diagram illustrating the process of changing constituent elements provided in an embodiment of this disclosure.

[0025] Figure 8 This is a flowchart illustrating the data update method provided in an embodiment of the present disclosure.

[0026] Figure 9 This is a flowchart illustrating the consistency detection method after updating data provided in an embodiment of this disclosure.

[0027] Figure 10 This is a schematic block diagram of a multimodal document association update apparatus according to one embodiment of the present disclosure.

[0028] Figure 11 This is a schematic block diagram of an electronic device according to one embodiment of the present disclosure. Detailed Implementation

[0029] The present disclosure will now be described in further detail with reference to the accompanying drawings and examples. It should be understood that the specific examples described herein are for illustrative purposes only and are not intended to limit the scope of the disclosure. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present disclosure are shown in the accompanying drawings.

[0030] It should be noted that, where there is no conflict, the embodiments and features described in this disclosure can be combined with each other. The technical solutions of this disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0031] Figure 1 This is a flowchart illustrating the multimodal document association and update method provided in an embodiment of this disclosure. Figure 1 The method shown includes steps S101 to S106. This method can be applied to the server side (e.g., local server, cloud server, etc.).

[0032] Specifically, Figure 1 The method shown includes step S101: obtaining product development documents of multiple different modalities.

[0033] It should be noted that the product development documents mentioned here refer to a collection of documents in different forms generated during the development of Internet products to describe the same business requirements. These include at least a Product Requirement Document (PRD), a business process diagram, and an interface prototype. The PRD is used to describe function definitions, field constraints, business rules, etc.; the business process diagram is used to describe processing nodes, decision branches, and logical flow; and the interface prototype is used to describe control layout, interactive feedback, and visual style.

[0034] In practical applications, the first step is to acquire multiple product development documents in different modalities that describe the same business requirement. These product development documents include at least a product requirement document, a business process diagram, and an interface prototype. In typical internet product development scenarios, these documents are usually generated in parallel by an AI generation model based on a natural language requirement input by the user (e.g., "support batch export of reports for customers"). It is important to emphasize that the multimodal product development documents processed in this disclosed solution are not limited to AI generation models; they can also be multiple documents describing the same business requirement, either manually written, imported from other systems, or generated in any other way. As long as these documents can be parsed into structured data, this solution can perform subsequent semantic binding and association update operations. After acquiring these product development documents, they are not treated as isolated final products, but rather as raw data objects for subsequent structured parsing and semantic binding. Through the document parsing interface, the product requirement document is parsed into a traversable list of function points, the business process diagram is parsed into a set of nodes and edges, and the interface prototype is parsed into a control tree structure, thus providing a basis for subsequent extraction of component elements.

[0035] Step S102: Extract the semantic concepts of each component element in the product development document and assign semantic identifiers to the component elements.

[0036] It should be noted that the semantic concepts mentioned here refer to abstract concepts extracted from the constituent elements through a semantic analysis model, representing their core functional attributes. For example, "submit order," "generate order node," and "submit button" are uniformly extracted as the semantic concept "Order_Submit." Semantic identifiers are globally unique identifiers assigned to each constituent element. They can be generated using a combination of content hash, modality type, and timestamp, and are used to uniquely index the constituent element under any modality and version.

[0037] After acquiring multimodal documents and parsing them into structured data, semantic analysis is further performed on the constituent elements of each document.

[0038] Specifically, the system invokes a pre-trained semantic analysis model (such as a BERT-based semantic understanding model or a prompt word project based on a large language model) to process the text description or label content of each component. Taking the "Submit Order" function in the product requirements document as an example, the semantic analysis model extracts its core semantic features and outputs the normalized semantic concept "Order_Submit". Similarly, for the processing node labeled "Generate Order and Verify" in the business process diagram, the semantic analysis model maps it to the same semantic concept "Order_Submit"; for the button control labeled "Submit" in the interface prototype, it is also mapped to the same semantic concept "Order_Submit".

[0039] Component elements with different modalities and varying representations are normalized to a unified semantic concept layer through a semantic analysis model. Simultaneously, the system assigns a globally unique semantic identifier to each component element. This semantic identifier can be generated using the following strategy: combining and encrypting the document identifier of the document to which the component element belongs, its modal type, the element's path location within the document, the element's content hash value, and its generation timestamp to generate a fixed-length string. For example, the semantic identifier for the "Submit Order" function in a product requirements document could be "PRD_a3_OrderSubmit_200". This semantic identifier remains unchanged throughout the entire document lifecycle; even if the element's content is modified, its semantic identifier continues to exist as a permanent identifier until the element is explicitly deleted.

[0040] Step S103: Establish cross-modal semantic binding relationships using semantic concepts and semantic identifiers.

[0041] It should be noted that the semantic binding relationship mentioned here refers to the mapping relationship formed by associating components with the same semantic concept in multiple different modalities with their respective semantic identifiers to the same semantic concept. This relationship records "which semantic identifiers represent the same business meaning" and is the data foundation for subsequent dependency tracking and cross-modal updates.

[0042] After obtaining the semantic identifiers of each component and their corresponding semantic concepts, this step establishes cross-modal semantic binding relationships.

[0043] The system maintains a semantic binding table, whose data structure is either a collection of key-value pairs or a relational database table. Each record in the table has the following core fields: a semantic concept (as the key) and a list of semantic identifiers belonging to that semantic concept (as the value). Taking "Order_Submit" as an example, the system adds the semantic identifiers of the "Submit Order" function in the product requirements document, the "Generate Order" node in the flowchart, and the "Submit" button in the interface prototype to the identifier list corresponding to that semantic concept.

[0044] Three components that originally belonged to different documents and modalities were explicitly recorded as "representing the same business meaning" through semantic binding. This binding relationship serves as the basis for all subsequent dependency tracing and consistency maintenance operations.

[0045] Step S104: Construct a cross-modal dependency graph based on semantic binding relationships and dependencies between constituent elements.

[0046] It's important to note that the cross-modal dependency graph discussed here is a directed acyclic graph (DAG) where semantic identifiers of constituent elements are nodes and various dependencies between these elements are edges. This graph is used to describe the logical structure within and between product development documents, including semantic dependencies, process dependencies, interaction dependencies, and constraint dependencies. Cross-modal dependency graphs enable the system to analyze the scope of impact of any element change through graph traversal.

[0047] After semantic binding is completed, this step constructs a cross-modal dependency graph to describe the logical and execution dependencies between the constituent elements.

[0048] The system first identifies the dependencies between its constituent elements. Types of dependencies include, but are not limited to: semantic dependencies, representing equivalent associations between semantically bound elements in different modalities (this relationship is partially recorded in the semantic binding table and represented in the diagram as bidirectional edges between nodes within the binding group); process dependencies, representing the sequential order of business operations, such as the "verify permissions" node in the flowchart depending on the "login verification" node; interaction dependencies, representing the triggering relationship between interface actions and backend logic, such as the "export button" in the interface prototype depending on the "execute export" node in the flowchart; and constraint dependencies, representing logical constraints such as data validation, access control, or exception handling, such as the "phone number field is required" constraint in the product requirements document depending on the input box control in the interface prototype.

[0049] After identifying the dependencies, the system constructs a Directed Acyclic Graph (DAG) using the semantic identifier of each component as a node and the identified dependencies as directed edges. This graph can be constructed using a graph database (such as Neo4j) or stored in memory. For example, starting from the semantic identifier of the "Export button" in the interface prototype, a directed edge is created pointing to the semantic identifier of the "Execute Export" node in the flowchart, indicating that "the Export button triggers the export process"; then, a directed edge is created from the "Execute Export" node pointing to the "Export Data Format Constraint" function point in the product requirements document, indicating that the export logic is subject to this constraint.

[0050] Step S105: In response to a change request for a first component element of a first product development document in multiple different modalities, identify at least one affected second component element in the cross-modal dependency graph.

[0051] When a user modifies the first component element in a product development document, the system automatically captures the change and obtains the first semantic identifier corresponding to the first component element. Then, starting from this first semantic identifier, the system executes a directed graph traversal algorithm in the cross-modal dependency graph, visiting all nodes reachable from the starting point along the directed edges. The component elements corresponding to these nodes are the affected second component elements.

[0052] This traversal process includes not only nodes directly connected to the first constituent element, but also all indirectly dependent nodes. Through directed traversal of the dependency graph, the scope of influence across modalities and multiple hops is automatically determined.

[0053] Step S106: Perform an update operation on the second product development document to which the second component belongs.

[0054] For each identified second component element, the system locates its corresponding second product development document and, based on the changes to the first component element, generates updated data adapted to that document format. This updated data is then applied to the corresponding editing location within the document. Application methods include automatic patching or generating modification suggestions for user confirmation. Through these operations, a change is automatically and accurately propagated to all affected cross-modal documents, maintaining semantic consistency among multimodal deliverables.

[0055] Based on the publicly available solutions described above, a deterministic semantic binding relationship across modalities is established by assigning globally unique semantic identifiers to the constituent elements of multimodal product development documents and extracting semantic concepts. On this basis, a cross-modal dependency graph is constructed using semantic identifiers as nodes and dependencies as edges, thus making the implicit dependencies between multimodal documents explicit and computable. When a constituent element in a document changes, the system can perform a directed traversal of the dependency graph starting from that element, accurately identifying all directly or indirectly affected cross-modal elements, and performing local updates on the documents to which these elements belong. This process only performs point-to-point updates on the smallest subgraph of the dependency graph affected by the change, rather than globally regenerating all documents. This effectively avoids the risk of a full-document turbulent reconstruction caused by minor changes, maintaining the overall consistency of the multimodal documents while preserving the existing state of other unaffected parts of the documents to the greatest extent possible. It fundamentally solves the problem that artificial intelligence models can generate multimodal products but cannot maintain them systematically, and realizes automatic detection of change impacts and accurate cross-modal propagation, significantly reducing the risk of omissions and manpower costs in the process of maintaining multimodal documents.

[0056] In one or more embodiments of this disclosure, such as Figure 2 This is a flowchart illustrating the semantic identifier generation method provided in an embodiment of this disclosure. Figure 2 As shown, the semantic concepts of each component element in the product development document are extracted, and semantic identifiers are assigned to the component elements, including: Step S201: Identify the component elements in different product development documents. Step S202: Extract the semantics of the component elements through a semantic analysis model to obtain semantic concepts representing the corresponding functional attributes of the component elements. Step S203: Assign a globally unique semantic identifier to each component element.

[0057] The system traverses and scans all parsed product development documents, extracting independent units as constituent elements based on the structural features of each modal document. For product requirement documents, it identifies functional points and field constraints based on chapter titles, list items, or field definition blocks; for business process diagrams, it parses node objects and their annotations in the process definition file; for interface prototypes, it parses control objects and their properties in the prototype description file. If a document cannot be automatically parsed (e.g., only an unstructured image is provided), the system can prompt the user to manually annotate or use optical character recognition (OCR) technology to assist in extraction. The AI-generated model outputs documents according to a predetermined structured format to ensure deterministic recognition.

[0058] Furthermore, each identified component element is input into a semantic analysis model. This model employs a pre-trained language model (such as BERT and its variants), taking into account the text content of the component element, the modality type of the document containing the element, and contextual information. The output is a normalized semantic concept identifier string. The semantic concept does not depend on the specific expression of the original text but rather represents the business function attribute undertaken by the element. This semantic analysis model is trained on a large amount of cross-modal labeled data, learning to map different expressions describing the same function in different modalities to the same semantic concept output. This approach directly maps elements to normalized semantic concepts through the semantic analysis model, establishing a deterministic equivalence relation independent of the original text expression. This forms the data foundation for subsequent cross-modal semantic binding.

[0059] After semantic concept extraction, the system generates a globally unique semantic identifier for each component element. The algorithm for generating this identifier is as follows: using the document identifier of the product development document to which the component element belongs, the modality type identifier, the structural path of the element in the document, the hash value of the element's content, and the generation timestamp as input parameters, a fixed-length string is calculated through a hash function. Once assigned, this identifier remains constant throughout the entire lifecycle of the component element. Even if the element's content is subsequently modified, the original semantic identifier continues to exist as a permanent identity until the component element is explicitly deleted. This solution endows each component element with deterministic indexing capabilities similar to a database primary key through semantic identifiers, which is a necessary prerequisite for subsequent dependency tracking and change propagation.

[0060] Based on the publicly available solutions described above, by systematically identifying the constituent elements in product development documents of different modalities, and using a semantic analysis model to normalize the constituent elements of different modalities and expressions into a unified semantic concept, and assigning a globally unique semantic identifier to each constituent element, a structured and computable data foundation is provided for cross-modal semantic binding and dependency tracking. This enables fine-grained functional units that were originally scattered across multiple documents to possess deterministic semantic identities and cross-document indexability.

[0061] In one or more embodiments of this disclosure, such as Figure 3 This is a flowchart illustrating the semantic binding relationship establishment method provided in this embodiment of the disclosure. Figure 3 As shown, establishing cross-modal semantic binding relationships using semantic concepts and semantic identifiers includes: Step S301: Constructing a semantic binding table. Step S302: Determining globally unique semantic identifiers corresponding to multiple different constituent elements with the same semantic concept in multiple different modalities. Step S303: Establishing semantic binding relationships between semantic identifiers of multiple different modalities and the same semantic concept in the semantic binding table.

[0062] Create a semantic binding table in memory or persistent storage. The table's data structure can be a set of key-value pairs, a relational database table, or related records in a graph database. The semantic binding table must contain at least two fields: a semantic concept field to store a normalized semantic concept identifier string; and a semantic identifier set field to store one or more semantic identifiers belonging to that semantic concept. The semantic binding table is empty during system initialization and is dynamically populated as the constituent elements of multimodal documents are processed. This table is independent of the format or version of any specific document, existing independently of individual product development documents, and serves as a centralized index for cross-modal semantic associations.

[0063] Furthermore, the system iterates through all constituent elements, and for each element, it obtains its semantic concept and semantic identifier. Then, the system groups the constituent elements according to their semantic concepts: constituent elements with the same semantic concept string are grouped together. Constituent elements within each group must originate from at least two different modal types (otherwise, there would be no need for cross-modal binding), and each constituent element corresponds to a unique semantic identifier. The system extracts the semantic identifiers of all constituent elements within each group, forming a list of semantic identifiers corresponding to that semantic concept. A single semantic concept can contain multiple constituent elements from different documents and modalities, or multiple different elements from the same document (e.g., two functional points describing the same function in the same PRD document). The system does not require each semantic concept to correspond to at least two modalities; when a semantic concept corresponds to only one constituent element, the binding relationship is still established, but the cross-modal synchronization capability is not yet utilized. In typical application scenarios, the AI ​​generation model simultaneously outputs products from three modalities; therefore, most core functional points will appear simultaneously in product requirement documents, business process diagrams, and interface prototypes, thus forming cross-modal bindings.

[0064] For each set of defined semantic concepts and their corresponding semantic identifier lists, the system performs a write operation in the semantic binding table. Specifically, it stores the list of semantic identifiers as values ​​in the table, using the semantic concept as the key (or primary key). If the semantic concept does not yet exist in the table, a new record is created; if it already exists, the semantic identifiers in the current list are merged into the existing record (duplicate removal). Through this operation, the system explicitly records that the constituent elements represented by each semantic identifier have an equivalence relationship in terms of functional attributes.

[0065] This semantic binding relationship is persistent, meaning that once established, it will not automatically become invalid due to subsequent modifications to the document content unless explicitly deleted or regenerated by the user. Furthermore, it supports bidirectional lookup; given a semantic identifier, the system can reverse-lookup the semantic binding table to determine its associated semantic concept, and then find all other semantic identifiers under that concept, thus achieving cross-modal element location. Through an independent semantic binding table, with semantic concepts as the central hub, elements scattered across different modal documents are explicitly linked, forming an independently maintainable and incrementally updatable cross-modal equivalence relation index structure. This structure is the core infrastructure for subsequent dependency graph construction and change propagation.

[0066] Based on the publicly available solutions described above, by constructing an independent semantic binding table, semantic identifiers of constituent elements with the same semantic concept from product development documents of different modalities are associated with the same semantic concept, establishing a deterministic cross-modal semantic binding relationship. This solution enables the system to accurately determine whether three different elements in three different documents represent the same business function without relying on the similarity of the original textual expressions. This semantic equivalence relationship management solution, which allows for automated construction, persistent storage, and programmatic querying, solves the problem of unified management of constituent elements across multimodal documents.

[0067] In one or more embodiments of this disclosure, such as Figure 4 This is a flowchart illustrating the cross-modal dependency graph construction method provided in this embodiment of the disclosure. Figure 4 As shown, a cross-modal dependency graph is constructed based on semantic binding relationships and dependencies between component elements, including: Step S401: Identifying dependencies between component elements, wherein the dependencies include: semantic dependencies, process dependencies, interaction dependencies, and constraint dependencies. Step S402: Constructing a cross-modal dependency graph using semantic identifiers as nodes and dependencies as edges.

[0068] The system analyzes all components that have been identified and assigned semantic identifiers, extracting four types of dependency relationships between the elements. The identification methods for each type of dependency relationship are as follows: Semantic dependencies are used to represent equivalent associations between components with the same semantic concept across different modalities. This relationship is essentially recorded in a semantic binding table. When constructing a dependency graph, the system reads the semantic binding table and, for all semantic identifiers under each semantic concept in the table, establishes a bidirectional semantic dependency edge between every two nodes corresponding to those semantic identifiers within that group, indicating that they represent the same business function. This relationship is the foundation of cross-modal mapping.

[0069] Process dependencies are used to represent the sequential order of business execution or the direction of data flow. This relationship primarily exists in business process diagrams, but may also exist across modalities between functional descriptions in the PRD and nodes in the flowchart. The system parses the flowchart definition file, extracting the labeled directed edges. Each edge, pointing from the source node to the target node, constitutes a process dependency. For functional descriptions in the PRD, if they contain explicit sequence words (such as "first…then…"), the system extracts implicit process dependencies through natural language parsing. Process dependencies are directed, with the direction representing the temporal sequence of business execution.

[0070] Interaction dependencies are used to represent the triggering relationship between interface actions and backend logic. This relationship mainly exists between the interface prototype and the flowchart, and may also extend to the functional description of the PRD. The system parses the event binding definitions in the interface prototype, such as the backend interface or process identifier called by the "click" event of a button. When the interaction event of an interface control triggers a certain process node or function, the system establishes an interaction dependency edge by pointing from the semantic identifier corresponding to the control to the semantic identifier corresponding to the triggered process node or function.

[0071] Constraint dependencies are used to represent the relationship between logical constraints such as data validation, access control, and exception handling and functional elements. This relationship can originate from field constraint descriptions in the PRD (e.g., "Mobile number required"), decision branch conditions in the flowchart (e.g., "Reject if insufficient permissions"), or input validation rules in the prototype. The system extracts constraints from the document text using a semantic analysis model and maps the target elements (validated fields, controlled nodes, restricted controls) defined by these conditions to the constraints themselves. Constraint dependency edges point from the constraint to the constrained element; the direction represents "constraint applied to the element."

[0072] The identification of the above four types of dependencies may overlap or be redundant. The system uses a dependency deduplication mechanism to ensure that each edge is recorded only once. For dependencies that cannot be automatically identified, the system can retain them for manual annotation by the user or adopt a default ignore strategy. In an optional solution, the AI ​​generation model outputs the dependency metadata between elements along with the multimodal output (i.e., product development documents), thereby ensuring the completeness and accuracy of the identification.

[0073] By systematically identifying four dependency types covering business descriptions, process logic, interaction behaviors, and constraints, the implicit business logic between and within multimodal documents is transformed into explicit, computer-processable relational data.

[0074] After dependency identification, the system constructs a directed acyclic graph. Specifically, the semantic identifier of each component element is treated as a node in the graph; each identified dependency is treated as a directed edge in the graph, with the direction of the edge determined by the logical direction of the dependency itself. Semantic dependencies are treated as bidirectional edges (equivalent to two unidirectional edges in opposite directions). All nodes and edges together constitute the cross-modal dependency graph.

[0075] The graph must be acyclic during construction. If a cycle is detected when adding an edge (e.g., A depends on B and B depends on A, but they are not bidirectional semantic equivalence relations), the system will handle it according to a pre-defined conflict resolution strategy: it may refuse to add the edge causing the cycle and prompt the user, automatically reverse the direction of the edge, or record the cycle as an anomaly for manual review. Ensuring the acyclicity of the graph is the mathematical foundation for subsequent topological sorting and influence range traversal.

[0076] This cross-modal dependency graph can be stored in a graph database or in an in-memory adjacency list structure. Each node (semantic identifier) ​​in the graph records the list of successor nodes it points to and the list of predecessor nodes it points to. The total number of nodes in the graph equals the sum of the number of constituent elements in all product development documents, and the total number of edges equals the number of all identified dependencies.

[0077] This disclosed solution constructs a dependency graph structure that can be directly manipulated by graph traversal algorithms by using semantic identifiers as stable nodes and various dependencies as directed edges. This graph integrates business logic that was originally scattered across different documents into a unified, computable semantic network.

[0078] Based on the aforementioned publicly available solution, the system identifies four types of relationships among the components of a multimodal document: semantic dependency, process dependency, interaction dependency, and constraint dependency. It then constructs a directed acyclic cross-modal dependency graph using semantic identifiers as nodes and dependency relationships as edges. This transforms the implicit business logic relationships existing in the minds of product managers or developers into an explicit, programmatically traversable graph structure. Compared to existing technologies that rely on manual experience to determine the scope of impact or use full document regeneration, this disclosure achieves a structured expression and computable storage of complex dependencies between multimodal documents. Without this dependency graph, subsequent change propagation can only be based on simple document-level comparisons, making it impossible to achieve precise element-level impact analysis and partial updates.

[0079] In one or more embodiments of this disclosure, such as Figure 5 This is a flowchart illustrating the cross-modal element modification method provided in this embodiment of the disclosure. Figure 5As shown, in response to a change request for a first component element of a first product development document in multiple different modalities, determining at least one affected second component element in a cross-modal dependency graph includes: Step S501: Obtaining a first semantic identifier corresponding to the first component element. Step S502: Traversing the cross-modal dependency graph starting from the first semantic identifier to determine at least one second semantic identifier reachable from the first semantic identifier via a directed path. Step S503: Determining the component element corresponding to at least one second semantic identifier as a second component element.

[0080] When a user initiates a change request for a specific element (i.e., the first component element) within a document (i.e., the first product development document) from multiple product development documents of different modalities, the system first captures this change event. This change request can be an operation such as text editing, attribute modification, element deletion, or addition. The system obtains the object reference of the modified first component element through the application programming interface provided by the document editor or the underlying data binding mechanism. Since each component element has been assigned a globally unique semantic identifier, and this semantic identifier is persistently bound to the component element object, the system can directly read its corresponding first semantic identifier from the first component element's data structure. If the first component element lacks a semantic identifier due to system anomalies or missing historical data, the system refuses to perform subsequent impact analysis and prompts the user to first perform semantic initialization processing on the document.

[0081] Furthermore, after obtaining the first semantic identifier, the system uses the node corresponding to that identifier as the starting point for traversal and executes a directed graph traversal algorithm on the constructed cross-modal dependency graph. The traversal algorithm can employ breadth-first search or depth-first search, and the specific implementation can be chosen based on system performance requirements and graph size. During traversal, the system moves from the current node to the successor node along the direction of the directed edges. It should be noted that the node determined by the traversal must have a directed path to the starting node; that is, there must exist a sequence of directed edges connected by at least one edge, with the same direction, that can reach the node from the starting point. This means that not only directly adjacent nodes (one hop) are included, but all nodes reachable through multi-hop directed paths are also included in the scope of influence.

[0082] This traversal process requires explicit definition of edge direction. Taking process dependencies as an example, if node A in the flowchart points to node B (meaning A executes before B), then modifying A may affect B; therefore, starting from A and following the edge direction, one can reach B. Conversely, modifying B generally does not affect A, so reverse traversal is not performed. For semantic dependencies (bidirectional edges), traversal can be performed in either direction. For interactive dependencies (from UI controls to process nodes), modifying UI controls will affect the process nodes they trigger, while modifying process nodes usually does not affect UI controls (unless the system is configured with a backpropagation strategy). The system executes according to predefined traversal direction rules, ensuring that the directionality of the impact analysis is consistent with the business logic.

[0083] The traversal can terminate under the following conditions: the current node has no outgoing edges, nodes that have already been visited are no longer visited, the system's preset maximum traversal depth is reached (to prevent an excessively large impact set), or a specific edge type filter is encountered (e.g., only traversing process-dependent edges while ignoring constraint-dependent edges). After traversal, the system collects semantic identifiers corresponding to all nodes reachable from the starting point, forming a second semantic identifier set. This set does not include the starting point itself (if the starting point needs to be included, the change processing for the starting point itself has already been completed in the change request capture phase, and there is no need to include it again in the impact set). By performing directed traversal on the directed acyclic dependency graph, cross-modal, multi-hop automated impact range determination is achieved, enabling the accurate discovery of all remote elements that have a logical transitive relationship with the changed element.

[0084] After obtaining the second set of semantic identifiers, each second semantic identifier is mapped back to its corresponding actual component object using a reverse index (e.g., a hash map) between the semantic identifiers and the component object. These components are the affected second components. Each second component necessarily belongs to a product development document (i.e., the second product development document). It is important to note that the second product development document may be the same document as the first product development document (e.g., there is a process dependency between two functional points within the same PRD document), or it may be a different modality of document (e.g., a flowchart node or prototype control).

[0085] If the traversal results do not find any second semantic identifier (i.e., the first component element has no outgoing edges in the dependency graph), it is determined that there is no affected second component element, and the change is limited to the first product development document itself. The system only performs the normal document saving operation.

[0086] Based on the aforementioned publicly available solution, by obtaining the semantic identifiers of the changed constituent elements as the starting point for graph traversal, a reachability traversal is performed along the directed edges in the cross-modal dependency graph. The constituent elements corresponding to all reachable nodes along the traversed directed paths are identified as affected elements. In this publicly available solution, the scope of the change's impact is no longer limited to the modified document itself, but can discover logically related indirectly affected elements across different modal documents. Furthermore, the determination is based on deterministic graph reachability calculations of the directed dependency graph, rather than relying on experience or probability matching, resulting in accurate and reproducible results. The traversal rules (direction, depth constraints, edge type filtering) are configurable, allowing the impact analysis to adapt to the granularity requirements of different business scenarios.

[0087] In one or more embodiments of this disclosure, such as Figure 6 This is a flowchart illustrating the cross-modal traversal method provided in an embodiment of this disclosure. Figure 6 As shown, traversing the cross-modal dependency graph starting from the first semantic identifier to determine at least one second semantic identifier reachable from the first semantic identifier via a directed path includes: Step S601: Obtaining the edge type and weight representing the dependency strength of each edge in the cross-modal dependency graph. Step S602: Filtering candidate connection edges during the traversal process based on the weights and edge types. Step S603: Determining the nodes corresponding to the filtered connection edges as at least one second semantic identifier.

[0088] In the construction of cross-modal dependency graphs, each directed edge, in addition to recording the source and target nodes, is also assigned edge type and weight attributes. The edge type identifies the type of dependency relationship it represents, including semantic dependencies, process dependencies, interaction dependencies, and constraint dependencies. The weight is a numerical value that quantifies the degree to which the dependency relationship affects change propagation; for example, it ranges from 0 to 1, with a higher value indicating a stronger dependency. Weight assignment is determined based on the dependency relationship type and business closeness: semantic dependencies (indicating two elements representing the same function) are typically assigned higher weights; in process dependencies, the weight between directly related nodes is greater than that of indirect dependencies; and in constraint dependencies, mandatory constraints have a higher weight than recommended constraints. Weights can be preset using domain knowledge during system initialization or dynamically adjusted based on the co-occurrence frequency in historical modification records using a machine learning model. By assigning quantifiable weights to each edge, the dependency graph is upgraded from a topological structure to a weighted directed graph, providing a data foundation for configurable control of the scope of influence.

[0089] During the traversal, the system retrieves all outgoing edges from the current node as candidate connection edges, and performs a filtering judgment on each candidate edge based on edge type and weight. The filtering strategy can be predefined by the system or dynamically configured by the user, and specifically includes the following methods: Weight-based threshold filtering: The system sets a lower limit for weights, retaining only edges with weights greater than or equal to this threshold. Edges with weights below the threshold are considered weak dependencies and are not included in the propagation path. This method allows the system to avoid updating a large number of irrelevant elements due to excessive propagation.

[0090] Edge type-based filtering: The system configures a set of edge types that can be allowed to propagate, and only edge types belonging to this set are retained. For example, the system can be configured to allow only process dependencies and interaction dependencies to participate in propagation, while excluding semantic dependencies and constraint dependencies, thereby customizing the impact analysis dimensions for specific business scenarios.

[0091] The system sets different weight thresholds for different edge types. For example, it uses a lower threshold for process dependency edges and a higher threshold for constraint dependency edges to express the different requirements of consistency maintenance for different dependency types.

[0092] If a candidate edge fails the screening, the traversal algorithm ignores it and does not continue exploring subsequent nodes along that edge; edges that pass the screening are retained, and the system moves along these edges to subsequent nodes and continues traversing. This screening mechanism allows the breadth and depth of the impact to be flexibly adjusted according to the business scenario.

[0093] By introducing configurable side-level filtering conditions, the system can dynamically adjust the propagation strategy according to different business scenarios and risk preferences, thus achieving refined control over the determination of the scope of influence.

[0094] After the above filtering, the system merges the semantic identifiers corresponding to the target nodes pointed to by all candidate connection edges that have passed the filtering conditions from the current node into a second semantic identifier set. Then, the filtering and traversal process is repeated starting from these nodes until no new nodes are added or a preset stopping condition is reached. Each identifier in the final second semantic identifier set represents a node that can be reached from the starting point after continuous filtering.

[0095] It's important to note that the dependencies represented by edges that fail the filtering process are not permanently deleted; rather, they are excluded from automatic update propagation during this current iteration. In subsequent consistency checks, these dependencies can still be presented to the user as suggestive hints. The primary purpose of the filtering mechanism is to control the aggressiveness of automatic updates, rather than to deny the existence of the dependencies themselves.

[0096] Based on the publicly available solutions described above, by adding weights representing edge type and dependency strength to each edge in the cross-modal dependency graph, and filtering candidate connections based on weight thresholds and edge type during traversal, configurable and hierarchical determination of the scope of change impact is achieved. By adjusting the weight thresholds, the system can adapt to different strategy requirements, from conservative to aggressive propagation; by filtering edge types, the system can distinguish the propagation priorities at different business logic levels. This gives the change propagation mechanism flexible control capabilities.

[0097] In one or more embodiments of this disclosure, such as Figure 7 This is a schematic diagram illustrating the process of changing constituent elements according to an embodiment of this disclosure. Figure 7 As shown, the update operation for the second product development document to which the second component element belongs includes: Step S701: Obtain the changed content of the first component element. Step S702: Locate the target editing position in the second product development document based on the semantic identifier of the second component element. Step S703: Map the changed content to updated data adapted to the second product development document according to the semantic binding relationship. Step S704: Apply the updated data to the target editing position.

[0098] When a user modifies the first element, the system captures the operation through the document editor's change listener interface and extracts the changed content. The changed content includes a snapshot of the content before the change, a snapshot of the content after the change, the change type (addition, deletion, modification of a field or attribute), and the change's location within the document. If the change is text editing, the changed content also includes text differences; if the change is a structured attribute modification (such as a change in the conditional expression of a node in a flowchart), the changed content includes the attribute value pairs before and after the modification. The system encapsulates this information into a change event object, which serves as input parameters for subsequent cross-modal updates.

[0099] In this disclosed solution, because each component element has an independent semantic identifier, the system can pinpoint changes precisely at the element level, rather than the entire document. The system can perceive changes at the component element level with fine granularity, rather than simply detecting document modifications at the document level.

[0100] For each affected second component element, the system obtains its semantic identifier. Since each component element, when assigned a semantic identifier, also records its structured path within its respective product development document (e.g., for product requirements documents, the XPath of the element in the chapter tree; for flowcharts, the unique ID of the node; for interface prototypes, the path of the control in the view hierarchy), the system uses the semantic identifier to reverse-engineer this path information, thereby accurately locating the target editing position within the second product development document. If the original path becomes invalid due to document version changes (e.g., elements are deleted or moved), the system uses a semantic analysis model to re-match the position of elements with the same semantic concept within the document. If the match fails, a location error is reported to the user. By combining persistent semantic identifiers with structured paths, deterministic addressing of fine-grained elements within the document is achieved. Even if the document is re-parsed or its format converted, the element can still be located as long as it has not been deleted.

[0101] The updated data generated in the previous step is applied to the target editing location in the second product development document. The specific application method depends on the document's editing interface capabilities: for document formats that support programmatic editing (such as JSON-based structured documents), the system directly modifies the values ​​of the corresponding fields; for rich text documents, the system replaces or inserts content blocks through the document object model operation interface; for graphical documents, the system calls its API to update node attributes or add new nodes. After the application operation is completed, the system saves the document changes and triggers a version record update.

[0102] If a conflict is encountered during application (e.g., the content at the target location has been manually modified by the user and is inconsistent with the updated data generated by the system), the system will handle it according to the preset conflict resolution strategy: it may pause automatic updates and prompt the user to manually merge, adopt an overwrite strategy, or retain the user's modifications and record the differences for later review. After the application operation is completed, the system can continue to perform consistency checks on the updated document.

[0103] Based on the aforementioned publicly available solution, by acquiring fine-grained changes, accurately locating target editing positions across documents using semantic identifiers, generating update data adapted to the target document format based on change type and semantic binding relationships, and applying this data at specified locations, precise, localized, and format-adapted automatic updates from a single change to corresponding elements in a multimodal document are achieved. Only affected elements are modified, while unaffected elements remain unchanged, thus preserving other customized modifications made by the user on the document. Compared to manual synchronization, this publicly available solution eliminates repetitive work and the risk of omissions across documents and modalities.

[0104] In one or more embodiments of this disclosure, such as Figure 8This is a flowchart illustrating the data update method provided in an embodiment of this disclosure. Figure 8 As shown, applying updated data to the target editing location includes: Step S801: Determining an update strategy, wherein the update strategy includes at least one of: conflict annotation, synchronous modification suggestion, and automatic patching. Step S802: When the update strategy is conflict annotation, the target editing location is highlighted in the second product development document. And / or, Step S803: When the update strategy is synchronous modification suggestion, generating modification prompt text associated with the target editing location. And / or, Step S804: When the update strategy is automatic patching, directly applying the updated data to the target editing location.

[0105] Before applying updated data to the target editing location, the strategy to be used in this update must first be determined. The determination of the update strategy can be based on a comprehensive decision based on multiple dimensions, including but not limited to: the system's preset global configuration, the strategy mode selected by the user in real time, the confidence score of the changed content, the dependency strength weight of the affected elements, and whether there are any conflicts with manual modifications made by the user at the target editing location.

[0106] Specifically, the system calculates an update confidence score, which comprehensively considers the average weight of the traversed edges, the complexity of the changed content, and the success rate of similar modifications in the historical record. If the confidence score is higher than a preset high threshold (e.g., 0.9), the system automatically selects an automatic repair strategy; if the confidence score is between a medium and low threshold, the system selects a synchronous modification suggestion strategy; if a conflict that cannot be automatically merged is detected between the content at the target editing location and the system-generated update data, the system selects a conflict labeling strategy. The update strategy can also be explicitly specified by the user through interface options when initiating a change, with user-specified strategies taking precedence over automatic determination. By introducing a multi-level strategy mechanism, the system can smoothly switch between automatic execution and manual confirmation according to different scenarios.

[0107] When the update strategy is conflict labeling, the system does not directly modify the content in the second product development document. Instead, it highlights the target editing location. Highlighting can be achieved by changing the background color, adding a border, inserting a marker icon, or adding a comment annotation. Simultaneously, the system can attach a conflict reason explanation to the highlighted area, such as "This content is inconsistent with the corresponding functional point description in the PRD; it is recommended to modify it simultaneously." The purpose of conflict labeling is to draw the user's attention, allowing the user to manually decide how to handle the inconsistency. This is suitable for high-risk or low-confidence change scenarios.

[0108] When the update strategy is to suggest synchronous modifications, the system generates a modification prompt text associated with the target editing location. This prompt text describes the suggested modifications in natural language and may include one or more action buttons (such as "Apply with one click" or "Ignore"). The prompt text can be generated using a template based on the content and semantic binding relationships of the updated data. The synchronous modification suggestion strategy lies between conflict annotation and automatic patching: users do not need to analyze what needs to be modified themselves; they only need to click confirm to execute the modifications, but the final human decision-making power is still retained.

[0109] When the update strategy is set to automatic patching, the system directly applies the generated update data to the target editing location without user intervention. Automatic patching is suitable for high-confidence, low-risk change scenarios, such as those with strong semantic binding relationships (high-weight dependency edges), no user-manually modified conflicts at the target location, and simple changes (such as text adjustments to field names). After automatic patching is completed, the system can record the operation history of this automatic modification in the log for easy user follow-up.

[0110] It should be noted that the three strategies mentioned above are not mutually exclusive. The system can use them in combination: for example, automatic patching can be used for some high-confidence target locations, conflict labeling can be used for some conflicting locations, and synchronous modification suggestions can be generated for the remaining locations. The system determines the applicable strategy for each target editing location and executes the corresponding operations in the same round of updates. When a synchronous modification suggestion is applied to a location and the user subsequently clicks to confirm the application, the system then performs the actual update data writing for that location; when the user clicks to ignore, the location remains unchanged.

[0111] Based on the publicly available solutions described above, by determining the update strategy before applying updated data and then executing conflict labeling, synchronous modification suggestions, or automatic patching operations according to the strategy, a hierarchical control mechanism is achieved, ranging from fully automated to fully manual. The system can adaptively select the most suitable strategy based on change confidence, dependency strength, and conflict status, maximizing automation while ensuring accuracy. Users can preset strategies according to their preferences and scenario risk requirements to obtain a personalized update experience. The three strategies can be used in combination within the same update, ensuring both manual review of high-risk areas and silent updates of low-risk areas.

[0112] In one or more embodiments of this disclosure, such as Figure 9 This is a flowchart illustrating the consistency detection method after updating data provided in an embodiment of this disclosure. Figure 9As shown, after applying the updated data to the target editing location, the process includes: Step S901: After applying the updated data, perform a consistency check on the updated product development documents of multiple different modalities. Step S902: Verify whether the updated components match the semantic concepts. Step S903: Verify whether the business process connections are complete and whether there are any input / output conflicts. Step S904: Verify whether the triggering relationship between the interface controls and the backend logic corresponds. Step S905: Verify whether the access control, state transition, or exception prompts are consistent with the new function.

[0113] After the update operation is completed, the system can automatically trigger a consistency check process. This process iterates through all product development documents related to the change, namely the first product development document (the source document of the change) and all updated second product development documents (of course, if multiple documents have been changed, consistency checks are performed on all documents simultaneously). The purpose of the check is to verify whether the update operation successfully maintained the semantic consistency between multimodal documents, and whether there are any new inconsistencies caused by incomplete or erroneous updates. The check results can be output as a consistency score, a list of inconsistencies, and remediation suggestions. If a serious inconsistency is detected, the system can roll back the update operation or issue an alert to prompt the user to intervene manually.

[0114] The system checks the matching relationship between each updated component and its associated semantic concept. Specifically, for each updated or affected component, the system re-invokes the semantic analysis model, extracts its current semantic concept, and compares it with the semantic concept that the component should have recorded in the semantic binding table. If they match, the verification passes; if they do not match (for example, the updated element content no longer represents the original business function), the system marks the component as "semantic drift" and prompts the user to confirm whether rebinding or correcting the updated content is necessary. This verification ensures that the update operation does not disrupt the fundamental correspondence between the element and its semantic identity.

[0115] The system checks the integrity of connections between nodes in the business process diagram and the consistency of data flow. Connection integrity verification involves traversing all nodes in the diagram to ensure that each non-starting node has at least one incoming edge, each non-ending node has at least one outgoing edge, and there are no dangling isolated nodes or broken subgraphs. If an update operation adds a new node or deletes an existing node, the system verifies whether the relevant connections have been adjusted accordingly. Input / output conflict verification involves checking whether the input and output parameters of adjacent nodes in the diagram match. For example, whether the output field name of the preceding node matches the input field name of the following node, and whether the field types are compatible. If an update causes a change in the output of a node, the system verifies whether the input of its successor nodes still matches it.

[0116] The system checks whether the correspondence between controls (such as buttons and input boxes) in the UI prototype and the background logic they trigger (such as process nodes and function points) remains valid. Specifically, for each UI control, the system obtains the target function bound to its interaction event (e.g., the semantic identifier of the process node corresponding to a button's click event), and then verifies in the current version of the product development documentation whether the target function exists and whether its semantics have changed. If the target function has been deleted or its semantic concept has shifted, the system marks the triggering relationship between the control and the background logic as invalid. This verification ensures that the linkage between the UI layer and the logic layer remains correct after changes.

[0117] The system checks whether the permission configurations, state transition rules, and exception messages related to the changed function are consistent with the requirements of the new function. Permission control verification refers to: if the updated function involves new permission requirements (e.g., batch export requires specific role permissions), the system checks whether the corresponding rules have been defined in the permission configuration document or permission table. State transition verification refers to: if the change introduces new business states or alters state transition conditions, the system checks whether the state machine definition contains new state nodes and transition edges. Exception message verification refers to: the system checks whether there are corresponding prompt controls or exception branches in the interface prototype or flowchart for exception scenarios defined in the PRD (e.g., input validation failure, insufficient permissions). If any of the above is missing, the system records the inconsistency and can generate supplementary suggestions based on preset rules. It should be noted that failure in any of the above four verifications does not affect the independent execution of the other items. The system summarizes all verification results and outputs a consistency check report.

[0118] Based on the publicly available solutions described above, a closed-loop consistency check is performed on the multimodal product development documents after the update operation is completed. This check comprehensively verifies four dimensions: semantic concept matching, process connectivity and input / output conflicts, correspondence between the interface and backend logic, and consistency of permission status anomaly prompts. This automatically detects inconsistencies between documents caused by incomplete or incorrect change propagation, preventing these inconsistencies from being ignored or amplified in subsequent collaborations.

[0119] In one or more embodiments of this disclosure, obtaining multiple product development documents of different modalities includes: receiving business requirement descriptions input by users; generating multiple initial products of different modalities based on the business requirement descriptions using an artificial intelligence generation model; the multiple initial products of different modalities include at least product requirement documents, business process diagrams, and interface prototypes.

[0120] Provide an interactive interface for users (such as product managers or business analysts) to input business requirements in natural language. The business requirements description is a summary or detailed description of the feature to be developed, such as "support batch export of reports for customers" or "add date filtering functionality to the order list." User input can be text or speech-to-text conversion. The received business requirements description will be used as input for subsequent AI model generation.

[0121] The received business requirement description is used as a prompt or input parameter to invoke a pre-trained or deployed AI generative model. This AI generative model can employ a large language model or a multimodal generative model, internally integrating three sub-modules: a product requirement document generator, a flowchart generator, and an interface prototype generator. Alternatively, it can use the same model to specify the output format in the input to generate products of different modalities. Based on the functional logic implicit in the business requirement description, the generative model outputs a structured product requirement document, a business process description file (which can be converted into a flowchart), and a description file of the interface layout and controls.

[0122] When outputting modal products, AI generative models do not output plain text or unparseable binary data. Instead, they output structured and parsable data objects in predefined formats. For example, product requirement documents are output in JSON format, where each functional point has an independent object identifier and attribute fields; business process diagrams are output as a graph structure of nodes and edges; and interface prototypes are output as a control tree structure. These structured output formats provide a direct data source for subsequent identification of constituent elements, eliminating the need for additional document parsing or format conversion.

[0123] This disclosed solution requires the AI ​​generation model to output structured data, and during the generation process, it can selectively embed the location path or initial identification information of elements, thereby ensuring from the source that the multimodal product (i.e., product development documents) has a computable internal structure. This enables the overall solution to form a complete data path from generation to maintenance, avoiding complex and unreliable reverse parsing of documents after generation.

[0124] The deliverables generated by the artificial intelligence model include at least: a product requirements document describing functional definitions, field constraints, and business rules; a business process diagram describing processing nodes and logical flow; and an interface prototype describing control layout and interactive feedback. Of course, depending on user needs, these deliverables may also include other modal documents such as data dictionaries and test cases.

[0125] Immediately after the artifacts are generated, operations such as extracting semantic concepts, assigning semantic identifiers, establishing semantic bindings, and constructing dependency graphs are performed on these artifacts. This ensures that multimodal documents already possess semantic connections and traceability from the initial moment of artifact generation, rather than having these connections manually established after generation.

[0126] By receiving user-inputted business requirement descriptions and utilizing an AI-generated model, multiple modal initial deliverables are generated in parallel, including at least product requirement documents, business process diagrams, and interface prototypes. These deliverables are in a structured data format for subsequent element-level parsing. This structured output allows for direct execution of automatic identification of constituent elements and assignment of semantic identifiers, eliminating the need for complex and unstable document reverse parsing.

[0127] Based on any of the above embodiments, this disclosure also provides a multimodal document association and update apparatus. Figure 10 This is a schematic block diagram illustrating the structure of a multimodal document association and update apparatus according to one embodiment of this disclosure. Figure 10 As shown, the multimodal document association and update device includes: an acquisition module 1001, used to acquire multiple product development documents of different modalities; an extraction module 1002, used to extract the semantic concepts of each component element in the product development documents and assign semantic identifiers to the component elements; a binding module 1003, used to establish cross-modal semantic binding relationships using semantic concepts and semantic identifiers; a construction module 1004, used to construct a cross-modal dependency graph based on the semantic binding relationships and the dependency relationships between component elements; a determination module 1005, used to determine at least one affected second component element in the cross-modal dependency graph in response to a change request for a first component element of a first product development document in multiple different modalities; and an update module 1006, used to perform an update operation on the second product development document to which the second component element belongs.

[0128] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0129] The entity executing the information sending method in the specific embodiments of this disclosure may be an electronic device such as a server (including a local server or a cloud computing platform).

[0130] Therefore, based on any of the above embodiments, this disclosure also provides an electronic device that can execute the multimodal document association update method of any of the embodiments described above.

[0131] Figure 11 This is a schematic block diagram of an electronic device according to one embodiment of the present disclosure.

[0132] The hardware architecture of the electronic device 1000 can be implemented using a bus architecture. The bus architecture can include any number of interconnect buses and bridges, depending on the specific application of the hardware and overall design constraints. Bus 1100 connects various circuits, including one or more processors 1200, memory 1300, and / or hardware modules. Bus 1100 can also connect various other circuits 1400, such as peripheral devices, voltage regulators, power management circuits, external antennas, etc.

[0133] Bus 1100 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of representation, this diagram uses only one connection line, but this does not imply that there is only one bus or one type of bus.

[0134] This disclosure also provides a readable storage medium storing a computer program that, when executed by a processor, is used to implement the methods described above. A "readable storage medium" can be any means capable of containing, storing, communicating, propagating, or transmitting a program for use by or in conjunction with an instruction execution system, apparatus, or device. More specific examples of a readable storage medium include: an electrical connection with one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable read-only memory (CDROM), etc.

[0135] This disclosure also provides a computer program product, the methods of which can be implemented wholly or partially through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented wholly or partially as a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed, all or part of the processes or functions of this disclosure are performed.

[0136] Computer programs or instructions can be stored in a readable storage medium or transferred from one readable storage medium to another. For example, the computer program or instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The readable storage medium can be any available medium capable of access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, such as a floppy disk, hard disk, or magnetic tape; an optical medium, such as a digital video optical disc; or a semiconductor medium, such as a solid-state drive. The computer-readable storage medium can be a volatile or non-volatile storage medium, or it can include both volatile and non-volatile types of storage media.

[0137] Those skilled in the art will understand that embodiments of this disclosure can be provided as methods, systems, or computer program products. Therefore, this disclosure can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this disclosure can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0138] This disclosure is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0139] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0140] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0141] In the description of this specification, the references to terms such as "one embodiment / mode," "some embodiments / modes," "example," "specific example," or "some examples," etc., refer to specific features, structures, or characteristics described in connection with that embodiment / mode or example, which are included in at least one embodiment / mode or example of this disclosure. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment / mode or example. Moreover, the specific features, structures, or characteristics described may be combined in any suitable manner in one or more embodiments / modes or examples. Furthermore, without contradiction, those skilled in the art can combine and integrate the different embodiments / modes or examples described in this specification, as well as the features of different embodiments / modes or examples.

[0142] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this disclosure, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0143] Those skilled in the art should understand that the above embodiments are merely for illustrating the present disclosure and are not intended to limit the scope of the disclosure. Those skilled in the art can make other changes or modifications based on the above disclosure, and these changes or modifications still fall within the scope of the present disclosure.

Claims

1. A multimodal document association and update method, characterized in that, The method includes: Obtain product development documents in multiple different modalities; Extract the semantic concepts of each component element in the product development document and assign semantic identifiers to the component elements; Establish cross-modal semantic binding relationships using the semantic concepts and semantic identifiers; Based on the semantic binding relationships and the dependency relationships between the constituent elements, a cross-modal dependency graph is constructed. In response to a change request for a first component element of a first product development document in the plurality of different modalities, at least one affected second component element is identified in the cross-modal dependency graph; Perform an update operation on the second product development document to which the second component belongs.

2. The multimodal document association and update method according to claim 1, characterized in that, The step of extracting the semantic concepts of each component element in the product development document and assigning semantic identifiers to the component elements includes: Identify the constituent elements in the different product development documents; The semantics of the constituent elements are extracted by a semantic analysis model to obtain semantic concepts that characterize the functional attributes of the constituent elements. Assign a globally unique semantic identifier to each of the constituent elements.

3. The multimodal document association and update method according to claim 1, characterized in that, The step of establishing a cross-modal semantic binding relationship using the semantic concept and the semantic identifier includes: Build a semantic binding table; Determine globally unique semantic identifiers corresponding to multiple different constituent elements having the same semantic concept in multiple different modalities; In the semantic binding table, establish semantic binding relationships between multiple semantic identifiers of different modalities and the same semantic concept.

4. The multimodal document association and update method according to claim 1, characterized in that, The construction of a cross-modal dependency graph based on the semantic binding relationship and the dependency relationship between the constituent elements includes: Identify the dependencies between the constituent elements, wherein the dependencies include: semantic dependencies, process dependencies, interaction dependencies, and constraint dependencies; The cross-modal dependency graph is constructed using the semantic identifiers as nodes and the dependencies as edges.

5. The multimodal document association and update method according to claim 1, characterized in that, In response to a change request for a first component element of a first product development document across the plurality of different modalities, determining at least one affected second component element in the cross-modal dependency graph includes: Obtain the first semantic identifier corresponding to the first constituent element; Starting from the first semantic identifier, traverse the cross-modal dependency graph to determine at least one second semantic identifier that is reachable from the first semantic identifier through a directed path; The constituent element corresponding to the at least one second semantic identifier is determined to be the second constituent element.

6. The multimodal document association and update method according to claim 5, characterized in that, The step of traversing the cross-modal dependency graph starting from the first semantic identifier to determine at least one second semantic identifier reachable from the first semantic identifier via a directed path includes: Obtain the edge type and weight representing the dependency strength of each edge in the cross-modal dependency graph; Based on the weights and edge types, candidate connection edges are filtered during the traversal process; The nodes corresponding to the filtered connection edges are determined as at least one second semantic identifier.

7. The multimodal document association and update method according to claim 1, characterized in that, The step of updating the second product development document to which the second component element belongs includes: Obtain the changed content of the first component element; Based on the semantic identifier of the second component element, locate the target editing position in the second product development document; Based on the type of the changed content and the semantic binding relationship, generate updated data adapted to the second product development document; Apply the updated data to the target editing location.

8. The multimodal document association and update method according to claim 7, characterized in that, After applying the updated data to the target editing location, the process further includes: After applying the updated data, a consistency check is performed on the updated product development documents of the multiple different modalities. Verify whether the updated constituent elements match the semantic concept; Verify that the business process connections are complete and that there are no input / output conflicts; Verify whether the triggering relationship between the UI controls and the backend logic corresponds; Verify that access control, status transitions, or exception messages are consistent with the new features.

9. An electronic device, characterized in that, include: The memory stores execution instructions; as well as, A processor that executes execution instructions stored in the memory, causing the processor to perform the method of any one of claims 1 to 8.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 8.

Images