A multi-modal-based intelligent software architecture design and optimization system

Abstract

The present invention discloses a multi-modal based intelligent software architecture design and optimization system, which relates to the field of intelligent design and optimization. It includes performing modal perception and preprocessing on the user's voice input, natural language text input, graphic sketches, and gesture operations, converting them into structured semantic data, and optimizing and enhancing the semantic expressions by introducing reinforcement learning strategies to form a set of structured user intention vectors; through the semantic mapping and reasoning processing unit, constructing a semantic-component alignment graph from the user intention vectors in the set of user intention vectors, mapping the user intention vectors to the target intention nodes in the semantic-component graph, and screening out a candidate component set from the architecture knowledge graph in combination with the semantic association scoring mechanism. The structure diagram generation step in the structure generation module effectively improves the rationality, diversity, and adaptation ability of the automatically generated structure, providing core support for the automatic construction of the architecture structure for target requirements.

Description

Technical Field

[0001] The present invention relates to the field of intelligent design and optimization, and in particular to a multi-modal software architecture intelligent design and optimization system. Background Art

[0002] With the increasing functional complexity of software systems, the increasingly diverse deployment environments, and the continuous evolution of user interaction methods, traditional software architecture design methods have gradually exposed limitations such as low efficiency, weak intelligence, and poor adaptability. In recent years, the rapid development of artificial intelligence, especially multimodal perception, natural language processing, and graph neural networks, has promoted the transformation of software architecture design patterns from people-centered to intelligent collaboration. Multimodal technology enables the system to recognize and integrate inputs from different forms such as voice, text, graphic sketches, gestures, etc., so as to obtain user demand intentions more comprehensively; semantic modeling and vector expression methods realize the structured transformation of complex requirements; at the same time, semantic-driven component recommendation mechanisms and graph structure modeling algorithms also provide a feasible path for automatically generating preliminary architecture diagrams. On this basis, how to achieve accurate semantic understanding of user intentions, multimodal collaborative modeling, and structural optimization control has become an important research direction for current intelligent software architecture design.

[0003] Although existing technologies have made some progress in natural language semantic parsing, component knowledge graph construction, and preliminary structure diagram generation, there are still many problems that need to be solved. For example, the current semantic fusion accuracy between multimodal inputs is insufficient, and the semantic understanding ability is difficult to reflect the user's contextual personalized behavior; the architecture candidate structure diagram lacks reliable logic and engineering consistency verification methods, and is prone to potential conflicts or weak connection points; in addition, the existing system relies on manual intervention in the structural optimization link, lacks automatic feedback mechanism and semantic closed-loop control capabilities, making it difficult to implement structural optimization suggestions. Summary of the invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a multimodal-based software architecture intelligent design and optimization system that solves the problems of inaccurate semantic understanding and insufficient consistency in structure diagram generation under multimodal user input.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a multi-modal software architecture intelligent design and optimization system, which includes:

[0008] The intention recognition module performs modality perception and preprocessing on the architecture design requirement information input by the user, converts it into structured semantic data, and optimizes and enhances the semantic expression by introducing a reinforcement learning strategy to form a structured set of user intention vectors; the structure generation module, through the semantic mapping and reasoning processing unit, constructs a semantic-component alignment graph based on the user intention semantic vectors, and uses a combination of graph structure modeling and adversarial generation algorithms to generate a set of candidate structure diagrams; the structure verification module calls an automatic structure inspection process to perform logical and engineering verification on the set of candidate structure diagrams, and automatically visualizes and marks potential errors and structural weaknesses to obtain a list of structural problems and an initial architecture sketch; the optimization output module, through the optimization logic unit, gives one-key optimization suggestions according to the list of structural problems to form a final optimized architecture solution.

[0009] As a preferred solution of the multi-modal software architecture intelligent design and optimization system described in the present invention, among them: the modality perception and preprocessing of the architecture design requirement information input by the user, and the conversion into structured semantic data are as follows.

[0010] Receive various forms of input from the user, classify and encapsulate them according to the input modality to form a set of multi-modal raw data.

[0011] Call the corresponding parsing module to perform signal recognition and conversion on the set of multi-modal raw data, and uniformly normalize it to obtain a set of semantic vectors after modality alignment.

[0012] Input the set of semantic vectors after modality alignment into the context fusion module, and combine the user's historical behavior and the modality credibility coefficient to generate structured semantic data.

[0013] As a preferred solution of the multi-modal software architecture intelligent design and optimization system described in the present invention, among them: the various forms of input from the user refer to the user's voice input, natural language text input, graphic sketches, and gesture operations.

[0014] As a preferred solution of the multi-modal software architecture intelligent design and optimization system described in the present invention, among them: the steps of calling the corresponding parsing module to perform signal recognition and conversion on the set of multi-modal raw data are as follows.

[0015] Convert the language input into text to obtain a list of intention keywords and a functional sentence structure.

[0016] Extract module nodes and connection relationships from the image sketch to obtain a set of connection paths and a set of component candidate matching semantic units.

[0017] Identify the operation semantics from the gesture operations to obtain a list of interaction operations and a correspondence table between component identifiers and action intentions.

[0018] Perform syntactic parsing, entity extraction, and structural mapping on the natural language text input to obtain a list of functional entities, a technical attribute structure, and non-functional requirement annotations.

[0019] As a preferred solution of the multi-modal based software architecture intelligent design and optimization system of the present invention, wherein: the semantic expression is optimized and enhanced by introducing a reinforcement learning strategy to form a structured set of user intention vectors, and the steps are as follows.

[0020] Encode and convert the semantic units in the structured semantic data into semantic unit vectors, and perform format alignment and semantic space mapping on the semantic unit vectors to obtain a multi-modal fusion semantic vector set.

[0021] Retrieve the historical behavior data record of the current user, and perform context fusion modeling with the multi-modal fusion semantic vector set to obtain a user context enhanced semantic vector set.

[0022] Input the user context enhanced semantic vector set into a learning model, and apply a reinforcement learning algorithm based on policy gradient to update the policy model parameters to obtain a set of user core intentions.

[0023] Perform term standardization mapping on the set of user core intentions to obtain a structured intention-term alignment list, and classify the intention-term alignment list according to intention types to obtain a structured user intention semantic vector.

[0024] As a preferred solution of the multi-modal based software architecture intelligent design and optimization system of the present invention, wherein: the combination of graph structure modeling and adversarial generation algorithm is adopted to generate a set of candidate structure diagrams, and the steps are as follows.

[0025] Use semantic mapping and reasoning to map the structured user intention semantic vector to the target intention node in the semantic-component graph, and combine the semantic association scoring mechanism to screen out a set of candidate components corresponding to the semantic node from the architecture knowledge graph.

[0026] According to the semantic mapping and reasoning processing unit, and the semantic-component association information in the architecture knowledge graph library, construct the alignment relationship between intentions and components and the structural connection relationship between components, and organize the alignment relationship between intentions and components and the structural connection relationship between components through a structured graph construction engine to obtain a set of candidate component structure diagrams.

[0027] Use a graph neural network to model each candidate structure diagram in the set of candidate component structure diagrams to obtain a set of embedded graph structure vectors.

[0028] Input the set of structure vectors of the embedded graph, generate multiple candidate structure diagrams, add semantic labels to each node in each candidate structure diagram, and perform standard formatting and encapsulation to obtain a set of candidate structure diagrams.

[0029] As a preferred embodiment of the multi-modal based intelligent software architecture design and optimization system of the present invention, wherein: according to the automatic structure inspection process, perform logical and engineering verification on the set of candidate structure diagrams, and automatically visualize and mark potential errors and structural weaknesses to obtain a list of structural problems and an initial architecture sketch. The steps are as follows:

[0030] Adopt a method based on semantic ontology mapping and structural specification template matching to convert the set of candidate structure diagrams into an internal graph modeling format, and combine the graph node type set and topological rules to parse and encapsulate the unified standard graph structure model to obtain a set of standardized structure diagram representations;

[0031] Perform logical integration and engineering verification on the set of standardized structure diagram representations to obtain a list of structural problems including the type, location, and severity level of each problem;

[0032] Combine the set of standardized structure diagram representations and the list of structural problems, mark the structural problems in the standardized structure diagram identification set, and generate a sketched structure diagram with annotations.

[0033] As a preferred embodiment of the multi-modal based intelligent software architecture design and optimization system of the present invention, wherein: through the optimization logic unit, one-key optimization suggestions are made according to the list of structural problems. The steps are as follows:

[0034] Analyze each item in the list of structural problems one by one, classify and process them according to the problem type, and match optimization suggestions for each problem to obtain a set of optimization suggestion solutions;

[0035] Determine the applicability of each optimization suggestion in the set of optimization suggestion solutions, and automatically modify the initial structure sketch according to the optimization suggestions to obtain an updated architecture structure diagram model;

[0036] Perform semantic extension, technology stack completion, and engineering encapsulation processing at the deployment level on the updated architecture structure diagram model to obtain the final optimized architecture solution.

[0037] As a preferred embodiment of the multi-modal based intelligent software architecture design and optimization system of the present invention, wherein: the determination of the applicability of each optimization suggestion in the set of optimization suggestions means that before applying each structural optimization suggestion, it is automatically determined whether the optimization suggestion is suitable for direct application in the architecture structure diagram model.

[0038] The beneficial effects of the present invention are as follows: By aligning and constructing a graph between the user intention vector and the system component knowledge base through the semantic mapping and reasoning processing unit, a semantic-component graph is established. The component dependency relationship is modeled using a graph neural network, and a diversity candidate structure diagram scheme is generated in combination with an adversarial generation mechanism. The structure generation module breaks through the traditional static structure generation method based on rules or templates, realizes the dynamic conversion from semantic abstraction to structural instances, and introduces a two-way feedback mechanism of semantic constraint and structure optimization, enabling the generated scheme to be innovative and scalable while meeting the intention expression. Finally, the structure diagram generation step in the structure generation module effectively improves the rationality, diversity, and adaptation ability of the automatically generated structure, providing core support for the automatic construction of an architecture structure oriented to target requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0040] Figure 1 Schematic diagram of a software architecture intelligent design and optimization system based on multi-modal.

[0041] Figure 2 Flowchart of the structure generation module of a software architecture intelligent design and optimization system based on multi-modal.

[0042] Figure 3 Flowchart of the structure verification module of a software architecture intelligent design and optimization system based on multi-modal.

[0043] Figure 4 Flowchart of the optimization output module of a software architecture intelligent design and optimization system based on multi-modal. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following detailed description of the specific embodiments of the present invention is made in conjunction with the accompanying drawings of the specification.

[0045] Many specific details are set forth in the following description in order to fully understand the present invention. However, the present invention can also be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0046] Second, the "one embodiment" or "embodiment" referred to herein means a specific feature, structure, or characteristic that may be included in at least one implementation manner of the present invention. The "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it an individual or alternative embodiment that is mutually exclusive with other embodiments.

[0047] Referring to Figures 1 to 4 , which is an embodiment of the present invention. This embodiment provides a multi-modal based intelligent software architecture design and optimization system, including the following steps:

[0048] Perform modal perception and preprocessing on the architecture design requirement information input by the user, convert it into structured semantic data, and optimize and enhance the semantic expression by introducing a reinforcement learning strategy to form a set of structured user intention vectors.

[0049] Receive various forms of input data streams through a unified interaction interface, including voice files (WAV), natural language texts (TXT), sketch images (PNG / SVG), and gesture sequences (JSON), automatically identify the modal types of each input data, and encapsulate and classify the various forms of input data streams according to the modal types to form a multi-modal raw data set in a unified format.

[0050] Call the modal parsing module of the object for various types of modal data to extract primary semantic information as follows. When the input modality is voice input, use the Whisper-Large speech recognition model (large speech recognition model) to obtain the transcribed text and semantic tags. When the input modality is text input, use the RoBERTa model (pre-trained model) and entity recognition technology to extract functional keywords and technical terms. When the input modality is a sketch image, first extract features of the image through a convolutional neural network, and then use a graph structure reconstruction network (such as a graph convolutional network) to reconstruct the module nodes and connection paths. When the input modality is gesture input, use a skeleton recognition model based on a vision transformer to obtain the semantic of user interaction operations. Finally, each modality is parsed into: modal type, semantic structure data, and modal semantic vector.

[0051] Unify and map the semantic vectors extracted under different modalities into a standard semantic space to achieve normalization and alignment, and obtain a set of semantic vectors after modal alignment. The linear mapping formula is as follows,

[0052] ;

[0053] Wherein, represents the number of the modality, represents the original semantic vector corresponding to the th modal input, A linear projection matrix that maps the semantic vectors of each modality to a unified semantic space, denoted as the bias vector corresponding to the th modality during the process of mapping to the unified semantic space, denoted as the semantic vector after mapping, i.e., the normalized semantic vector,

[0054] Input the set of normalized modality semantic vectors into the context fusion module, and fuse the following two types of information: the historical behavior semantic data of the current user and the credibility coefficient of each modality, and based on the weighted similarity between the semantic vector of each modality and the learning weight vector obtained through backpropagation adaptive learning, obtain a set of weighted modality feasibility coefficients through softmax calculation (exponential normalization function calculation).

[0055] The unified semantic vector generated through the fusion result is input into the semantic decoding module, and using the classifier model and the entity mapping rule library, the semantic vector is converted into structured semantic information data.

[0056] Perform modality perception and preprocessing on the architecture design requirement information input by the user, and convert it into structured semantic data. The steps are as follows,

[0057] Receive various forms of input from the user, classify and encapsulate them according to the input modality to form a multi-modal raw data set;

[0058] Call the parsing module for the multi-modal raw data set for signal recognition and semantic conversion, and uniformly normalize it to obtain a set of modality-aligned semantic vectors;

[0059] Input the set of modality-aligned semantic vectors into the context fusion module, and combine the user's historical behavior and the modality credibility coefficient to generate structured semantic data.

[0060] Receive various forms of input from the user, and label them as four categories: text, speech, sketch, and gesture respectively: Remove the background noise from the speech, and segment the sentences to ensure that each segmentation is less than 10 seconds. Then, use the speech recognition engine on the edge or in the cloud to input the cleaned audio segments one by one and return the corresponding recognized text; Extract the contours of the sketch images, and identify the connection relationship between the graphic elements and the sketch images; Perform action recognition on the gesture operation video frames and convert them into standard instruction expressions. Finally, encapsulate all the input data according to the source modality to form a unified multi-modal raw data set and label their respective modality types.

[0061] Parse the multi-modal raw data set and extract semantic units: perform syntactic analysis on the text and transcribed speech content to extract functional requirements, non-functional constraints, and component relationship descriptions; abstract the node and connection information recognized from the sketch content to form descriptive statements representing the structural relationship between components; map the gesture operation recognition results to standard design actions, and finally perform standardized conversion of the semantic content using all modal information, parsing rules, and the domain term library to form a set of semantic units.

[0062] Perform embedding transformation based on the modal type and semantic type of each semantic unit in the set of semantic units to generate a unified format of semantic representation, and align the representations of the same semantic content in different modalities, removing redundant and conflicting expressions. Subsequently, set a trust coefficient for the modal source, and fuse the aligned semantics in all modalities to obtain a set of modality-aligned semantics.

[0063] Combine the user's recent historical behavior information in architecture design, including commonly used components, design preferences, and interaction patterns, and fuse it with the modality-aligned semantics to enhance the context correlation ability of the semantics. Subsequently, group and classify the fused content semantically, and output the grouping results in the form of structured semantic data as the input basis for constructing the subsequent architecture structure diagram, finally obtaining a structured semantic data set.

[0064] Optimize and enhance the semantic expression by introducing a reinforcement learning strategy to form a set of structured user intention vectors. The steps are as follows:

[0065] Encode and convert the semantic units in the structured semantic data into semantic unit vectors, and perform format alignment and semantic space mapping on the semantic unit vectors to obtain a set of multi-modal fused semantic vectors.

[0066] Retrieve the historical behavior data record of the current user and perform context fusion modeling with the set of multi-modal fused semantic vectors to obtain a set of user context-enhanced semantic vectors.

[0067] Use the set of user context-enhanced semantic vectors as input and update the policy through a reinforcement learning algorithm based on policy gradient to obtain a set of user core intentions.

[0068] Perform term standardization mapping on the set of user core intentions to obtain a structured intention-term alignment list, and classify the intention-term alignment list according to the intention type to obtain structured user intention semantic vectors.

[0069] Encode each semantic unit of the structured semantic data using a semantic encoding model (such as BERT-base) to convert it into a fixed-length semantic vector, and then embed all semantic unit vectors into the same semantic space through linear projection to form a preliminary set of multi-modal fused semantic vectors.

[0070] Extract historical behavior records including historical architecture modification logs, operation preferences, and keyword selection frequencies from the user behavior data storage module, convert the historical behavior records into context representation vectors through behavior sequence embedding, then concatenate the fused semantic vector and the context vector, and introduce a Transformer structure for fusion modeling to obtain a set of context-enhanced semantic vectors.

[0071] Take the set of context-enhanced semantic vectors as the current state and input it into the policy learning model. Define the accuracy and precision of the user intention expression as the reward function of the policy learning model. Then apply the method based on policy gradient to update the parameters of the policy model. After iterative training in multiple rounds, select the semantic vector that can maximize the expected reward as the output, which is used as the set of user core intentions.

[0072] Refer to the standardized semantic term library, perform vector matching on each core intention vector in the set of user core intentions, and select the closest standard term based on the cosine similarity calculation. Combine it with the intention vector to form an intention-term alignment pair. Then classify the alignment pairs according to the intention type to form a structured set of user intention semantic vectors.

[0073] Through the semantic mapping and reasoning processing unit, perform vector matching and semantic reasoning on the user intention semantic vectors and the component semantics in the architecture component library, construct a semantic-component alignment graph containing nodes and edges, and combine the knowledge rule set in the optimization suggestion module. Through graph traversal and reconstruction operations, generate a set of candidate structure graphs.

[0074] Input each semantic unit in the structured semantic data into a semantic encoding model (which can be a pre-trained language model) to generate intermediate semantic vectors. Then, through linear transformation, map all semantic vectors into a unified semantic representation space to form a set of multi-modal fusion semantic vectors.

[0075] Retrieve the historical interaction semantic records of the user from the user portrait or behavior database and input them into the same semantic encoding model. Encode the historical interaction semantic records into a set of historical behavior semantic vectors, and for each current semantic vector, calculate the semantic correlation between the current semantic vector and the historical behavior semantics, construct attention weights, and perform weighted fusion on the historical behavior to form a context-enhanced semantic vector.

[0076] Use the set of context-enhanced semantic vectors as the current state information and input it into the reinforcement learning policy model to learn the optimal intention selection policy. The reinforcement learning configuration is as follows: the state is each context-enhanced semantic vector, the action is whether to select this semantic vector as the core design intention, the reward signal is generated based on the success rate score generated by the subsequent structure diagram or the user interaction feedback evaluation function, and the policy function models the selection decision in the form of probability.

[0077] Train and update the intention selection model through the policy optimization formula to generate a set of core design intentions.

[0078] Match and map the terms in the set of core intentions output in the reinforcement learning stage through the semantic term standardization module with the standard component names and domain terms in the knowledge base, and classify and process them according to the semantic types of the intentions to obtain a structured intention semantic vector data structure.

[0079] Adopt a combination of graph structure modeling and adversarial generation algorithm to generate a set of candidate structure diagrams. The steps are as follows:

[0080] Based on the structured user intention semantic vector, obtain the intention nodes of the semantic-component graph through semantic mapping reasoning, and extract the candidate component set corresponding to the semantic nodes through the semantic association scoring mechanism;

[0081] Construct the intention-component alignment relationship and component structure relationship according to the semantic mapping and reasoning processing unit and the architecture knowledge graph library, and use the structured graph construction engine to organize them to obtain a set of candidate component structure diagrams;

[0082] Use the graph neural network to extract node and edge features for each candidate structure diagram in the set of candidate component structure diagrams, and combine with the adversarial generation algorithm to obtain a set of graph structure vector embeddings;

[0083] Use the semantic vector and the set of graph structure vector embeddings as inputs to generate multiple candidate structure diagram solutions, and add semantic labels to each node in each structure diagram for standard formatting encapsulation to obtain a set of candidate architecture structure diagram solutions.

[0084] Use the structured user intention semantic information as the input, through the semantic mapping reasoning process, associate each intention information with the corresponding intention node in the semantic-component knowledge graph, and then use the weighted semantic similarity scoring mechanism to calculate the semantic correlation between each intention information and the candidate component nodes in the graph, and screen out several most matching component nodes according to the score to obtain the candidate component list matched in the preliminary semantic component graph.

[0085] Call the predefined component connection rules in the structured knowledge graph to legally combine the candidate components in the candidate component list, construct candidate structure diagrams with topological structures among multiple components, and then pass each candidate structure diagram through the structure diagram construction engine to complete the standardized connection structure arrangement, and use the graph neural network algorithm for modeling to extract the overall structure features corresponding to each structure diagram, forming multiple candidate structure diagrams and their structure embedding representation results.

[0086] Adopt the adversarial algorithm for graph structure generation, input the semantic information and the structure diagram representation results, construct multiple possible candidate structure diagram solutions, then introduce the structure rationality discrimination module to evaluate each structure diagram solution, and finally label the nodes in each structure diagram with standardized semantic labels and component identities, complete the unified encapsulation of the structure diagram format, and form an architecture and candidate solution set for subsequent verification and optimization.

[0087] Call the automatic structure inspection process to perform logical and engineering verification on the candidate structure diagram set, and automatically visualize and mark potential errors and structural weaknesses through automatic visualization to obtain a list of structure problems and an initial architecture sketch.

[0088] Receive the structured user intention semantic vector set, where each vector represents a user core intention, and analyze the structured user intention semantic vector set through the semantic mapping and reasoning processing unit to identify the semantic nodes corresponding to the intention, and extract candidate components according to the intention-component association rule to obtain the "intention-component candidate set" structure, and the semantic association scoring formula is as follows,

[0089] ;

[0090] Among them, represents the th intention vector, represents the th component semantic vector, represents the semantic matching degree between intention and component , represents the current processed user intention vector, represents the current processed candidate component semantic vector.

[0091] Call the semantic mapping and reasoning processing unit and the architecture knowledge graph library to extract the alignment relationship between each pair of intention and component and the prior structure relationship between candidate components, and organize and reconstruct them through the structured graph construction engine to form an initial connection relationship graph between candidate components.

[0092] Input the set of candidate component structure diagrams into the graph neural network model, extract the overall representation of the graph, and generate a set of embedded vector representations of the candidate structure diagram set. The node information update formula is as follows:

[0093] ;

[0094] Among them, represents the feature representation of node in the th layer in the graph neural network, represents the feature representation of node in layer, represents the set of adjacent nodes of node , represents the number of adjacent edges of node , represents the number of adjacent edges of node , represents the weight matrix in the th layer, represents the current layer number of the graph neural network, represents the target node being updated currently, represents all neighbor nodes adjacent to node .

[0095] Input the set of semantic vectors and the set of graph structure embeddings into the structure diagram generator module, and use the combined mechanism of the structure generator and the discriminator to generate multiple candidate structure diagram solutions. The generator constructs an architecture diagram with preparatory engineering logic, structural rationality, and semantic consistency based on semantic-component-structure diagram embeddings, adds semantic labels to the component nodes of each structure diagram, and performs standard format encapsulation to finally generate a set of candidate structure diagrams.

[0096] According to the automatic structure inspection process, perform logical and engineering verification on the set of candidate structure diagrams, and visually mark potential errors and structural weaknesses automatically to obtain a list of structural problems and an initial architecture sketch. The steps are as follows:

[0097] Perform structure diagram data parsing on the set of candidate structure diagrams, convert them into an internal graph modeling format, establish a unified standard graph structure model, and obtain a set of standardized structure diagram representations after parsing and encapsulating the unified standard graph structure model;

[0098] Perform logical integration and engineering verification on the set of standardized structure diagram representations to obtain a list of structural problems including the type, location, and severity level of each problem;

[0099] Combine the set of standardized structure diagram representations and the list of structural problems, perform problem annotation in the structure diagram, and generate a sketch of the structure diagram with annotations.

[0100] The structure diagram files in the candidate structure diagram set are parsed through semantic rules and structural grammar constraints to obtain component nodes, connection edges, dependency types, and interface description information in each diagram. Subsequently, the extracted graphic primitive data is input into the structure parsing engine and uniformly converted into an internal system-defined graphic modeling format including unified node numbers, connection semantic types, and deployment attribute fields. Finally, a set of standard format structure diagram models is obtained, which ensures that each sub-diagram must contain a deployment entry node, there must be a reasonable dependency path between all service components, and there must be no isolated nodes or dead-loop paths.

[0101] For each structure diagram model, a graph structure dependency matrix is constructed, and consistency analysis is performed in combination with a predefined engineering constraint rule library, which mainly includes: logical consistency analysis and engineering constraint verification. Subsequently, a unique problem type code is assigned to each type of problem, and the severity of the problem is quantitatively rated according to the following rules: Level 1 problems (critical blocking types): such as backbone service disconnection, illegal circular dependencies; Level 2 problems (structural non-standard types): such as incorrect module reuse, duplicate interfaces; Level 3 problems (optimizable types): such as unused components, duplicate logical paths. Finally, a list of structure problems including problem types, location information, and severity is obtained.

[0102] The list of structure problems is mapped to the original structure diagram, corresponding to the position nodes or connection edges of each problem, and a color and image annotation strategy is adopted according to the severity of the problem. Subsequently, the annotations in the diagram are grouped into a unified layer to form an interactive structure sketch. Each diagram is bound to the corresponding problem list, and finally, a set of initial structure sketches with annotations is generated.

[0103] Through the optimization logic unit, one-key optimization suggestions are made according to the list of structure problems to form a final optimized architecture solution.

[0104] Receive the list of structure problems, each item of which includes: problem type, location of the involved component, and severity level of the problem. Input the problem list into the optimization suggestion checking engine, call the embedded optimization knowledge base, and use semantic matching and rule mapping methods to correspond to the optimization suggestions for each problem to obtain an optimization suggestion solution set. The optimization suggestion matching score formula is as follows.

[0105] ;

[0106] Where represents the th structure problem item, represents the th optimization suggestion item, represents the rd semantic vector representation of the problem, represents the The semantic vector representation of a suggestion is expressed as the semantic similarity score between the nth question and the

[0107] For each optimization suggestion, perform applicability analysis through an applicability scoring function, screen out safe and reasonable optimization suggestions for application to the initial structural sketch, and correct operations in the triggered automatic structure diagram, so as to obtain an updated architecture model.

[0108] Input the updated structure diagram model into the deployment encapsulation engine, add deployment-related meta-information to each component (for example: computing resource labels, security policy levels), and at the same time combine historical projects and context preferences to infer and generate the runtime environment of the component. Through the engineering encapsulation in the deployment encapsulation engine, automatically generate a deployment template, and finally obtain an optimized architecture structure diagram.

[0109] In summary, the present invention: aligns and composes the user intention vector with the system component knowledge base through the semantic mapping and reasoning processing unit to establish a semantic-component graph, and uses a graph neural network to model the component dependencies, and combines an adversarial generation mechanism to generate diverse candidate structure diagram schemes. The structure generation module breaks through the traditional static structure generation method based on rules or templates, realizes the dynamic conversion from semantic abstraction to structure instance, and introduces a two-way feedback mechanism of semantic constraint and structure optimization, so that the generated scheme is innovative and scalable while meeting the intention expression. Finally, the structure diagram generation step in the structure generation module effectively improves the rationality, diversity and adaptation ability of the automatically generated structure, providing core support for the automatic construction of the architecture structure oriented to target requirements.

[0110] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. An intelligent design and optimization system for software architecture based on multi-modalities, characterized in that: Including An intention recognition module that performs modal perception and preprocessing on the architectural design requirement information input by the user, converts it into structured semantic data, and optimizes and enhances the semantic expression by introducing a reinforcement learning strategy to form a structured set of user intention vectors. The user input is various forms of input from the user, including the user's voice input, natural language text input, graphical sketches, and gesture operations. A structure generation module that, through a semantic mapping and reasoning processing unit, constructs a semantic-component alignment graph from the user intention vectors in the set of user intention vectors, and uses a combination of graph structure modeling and adversarial generation algorithms to generate a set of candidate structure diagrams. The steps are as follows: Using semantic mapping and reasoning, map the structured user intention semantic vectors to the target intention nodes in the semantic-component graph, and combine with a semantic association scoring mechanism to filter out a set of candidate components corresponding to the semantic nodes from the architectural knowledge graph. According to the semantic mapping and reasoning processing unit and the semantic-component association information in the architectural knowledge graph library, construct the alignment relationship between intentions and components and the structural connection relationship between components, and organize the alignment relationship between intentions and components and the structural connection relationship between components through a structured graph construction engine to obtain a set of candidate component structure diagrams. Use a graph neural network to model each candidate structure diagram in the set of candidate component structure diagrams to obtain a set of embedded graph structure vectors. Input the set of embedded graph structure vectors, generate multiple structure diagram candidate solutions, and add semantic labels to each node in each structure diagram candidate solution for standard formatting and encapsulation to obtain a set of candidate structure diagrams. A structure verification module that calls an automatic structure inspection process to perform logical and engineering verification on the set of candidate structure diagrams, and automatically visualize and mark potential errors and structural weaknesses to obtain a list of structure problems. An optimization output module that, through an optimization logic unit, gives one-key optimization suggestions based on the list of structure problems to form a final optimized architecture solution.

2. The multimodal-based intelligent software architecture design and optimization system according to claim 1, characterized in that: The steps of performing modal perception and preprocessing on the architectural design requirement information input by the user and converting it into structured semantic data are as follows Receive various forms of input from the user, classify and encapsulate them according to the input modality to form a set of multi-modal raw data. Call the corresponding parsing module to perform signal recognition and conversion on the set of multi-modal raw data, and uniformly normalize it to obtain a set of semantic vectors after modal alignment. Input the set of semantic vectors after modal alignment into the context fusion module, and combine with the user's historical behavior and the modal credibility coefficient to generate structured semantic data.

3. The intelligent design and optimization system for software architecture based on multi-modalities as claimed in claim 2, wherein: The steps of calling the corresponding parsing module to perform signal recognition and conversion on the set of multi-modal raw data are as follows Convert the voice input into text to obtain a list of intention keywords and a functional sentence structure. Extract module nodes and connection relationships from the image sketch to obtain a set of connection paths and a set of candidate matching semantic units for components. Identify operation semantics from gesture operations to obtain a list of interaction operations and a correspondence table between component identifiers and action intentions. Perform syntactic parsing, entity extraction, and structure mapping on the natural language text input to obtain a list of functional entities, a technical attribute structure, and non-functional requirement annotations.

4. The multimodal-based intelligent software architecture design and optimization system according to claim 1, characterized in that: The semantic expression is optimized and enhanced by introducing a reinforcement learning strategy to form a structured set of user intention vectors. The steps are as follows: Encode and convert the semantic units in the structured semantic data into semantic unit vectors, and perform format alignment and semantic space mapping on the semantic unit vectors to obtain a multi-modal fusion semantic vector set; Retrieve the historical behavior data records of the current user, and perform context fusion modeling with the multi-modal fusion semantic vector set to obtain a user context-enhanced semantic vector set; Input the user context-enhanced semantic vector set into a learning model, and apply a reinforcement learning algorithm based on policy gradients to update the policy model parameters to obtain a set of user core intentions; Perform term standardization mapping on the set of user core intentions to obtain a structured intention-term alignment list, and classify the intention-term alignment list according to intention types to obtain structured user intention semantic vectors.

5. The multimodal-based intelligent software architecture design and optimization system according to claim 1, characterized in that: The step of calling the automatic structure inspection process to perform logical and engineering verification on the candidate structure atlas, and automatically visualize and mark potential errors and structural weaknesses to obtain a list of structure problems and an initial architecture sketch is as follows: Adopt a method based on semantic ontology mapping and structure specification template matching to convert the candidate structure atlas into an internal graph modeling format, and combine the graph node type set and topological rules to parse and encapsulate the unified standard graph structure model to obtain a standardized structure graph representation set; Perform logical integration and engineering verification on the standardized structure graph representation set to obtain a list of structure problems including the type, location, and severity level of each problem; Combine the standardized structure graph representation set and the list of structure problems, mark the structure problems in the standardized structure graph representation set, and generate a sketched structure graph with annotations.

6. The multimodal-based intelligent software architecture design and optimization system according to claim 5, characterized in that: The method based on semantic ontology mapping and structure specification template matching is to automatically unify the node semantics and topological structure in the original structure graph into a standard modeling format by establishing semantic concept mapping rules and a structure graph pattern template library.

7. The multimodal-based intelligent software architecture design and optimization system according to claim 1, characterized in that: The step of making one-key optimization suggestions according to the list of structure problems by optimizing the logic unit is as follows: Analyze each item in the list of structure problems one by one, classify and process them according to the problem types, and match optimization suggestions for each item to obtain a set of optimization suggestion solutions; Determine the applicability of each optimization suggestion in the set of optimization suggestion solutions, and automatically modify the initial structure sketch according to the optimization suggestions to obtain an updated architecture structure graph model; Perform semantic extension, technology stack completion, and engineering packaging processing at the deployment level on the updated architecture structure graph model to obtain the final optimized architecture solution.

8. The multimodal-based intelligent software architecture design and optimization system according to claim 7, characterized in that: The determination of the applicability of each optimization suggestion in the set of optimization suggestion solutions means that before applying each structure optimization suggestion, it is automatically judged whether the optimization suggestion is suitable for direct application in the architecture structure graph model.

Images