A multi-modal code automatic generation and optimization system

Abstract

The present invention discloses a multimodal-based automatic code generation and optimization system, which relates to the field of automatic programming technology. The system includes a task analysis module, which combines the CLIPS rule engine and knowledge graph, analyzes task descriptions and constraints in combination with raw data, identifies task objectives and requirements, and outputs a task risk assessment report and a task intent set; an optimization decision module, which uses a particle swarm optimization algorithm and a path planning algorithm to select the best modal data subset, optimizes and adjusts it according to task requirements, and outputs a code generation strategy; a test evaluation module, which uses mutation testing to evaluate and improve unit testing and integration testing, evaluates the quality and performance of the code through continuous integration and continuous delivery, and outputs a code evaluation report and optimization suggestions. The present invention dynamically screens the best modal data subset and optimizes the data transmission path, thereby reducing computing resource consumption while ensuring code performance and achieving the global optimization of the code generation strategy.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic programming, and in particular to a multi-modal based automatic code generation and optimization system. Background Art

[0002] Current automatic code generation technologies typically use sequence-to-sequence models, abstract syntax tree generation models, or template-based semantic transformation methods to map user-entered requirement descriptions into executable code. Unstructured or semi-structured information, such as log data, historical code snippets, development documentation, and runtime feedback, is also crucial in the development process, significantly aiding task semantic understanding and building code context.

[0003] However, traditional methods struggle to accurately capture the implicit constraints and domain knowledge inherent in code generation requirements, leading to a semantic gap between the generated code and the actual requirements. During code optimization, existing static analysis tools rely on rules to detect code smells, while dynamic optimization methods often suffer from a large search space, leading to low convergence efficiency and significantly reduced code generation effectiveness. Therefore, a mechanism that optimizes modal data selection strategies is crucial. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a multimodal-based code automatic generation and optimization system to solve the problems of insufficient dynamic adaptability of multimodal data fusion and lack of cross-modal collaboration in code generation path optimization.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] The present invention provides a multimodal-based automatic code generation and optimization system, which includes: the data processing module, which collects raw data and performs data cleaning and preprocessing to construct an original data set;

[0008] The task analysis module combines the CLIPS rule engine with the knowledge graph and the original data to analyze the task description and constraints, identify the task goals and requirements, and output the task risk assessment report and task intent set;

[0009] The data fusion module selects data sources based on the mission risk assessment report and mission intent set, uses the weighted average method to perform data fusion, and outputs the fused multimodal data set;

[0010] The optimization decision module uses particle swarm optimization and path planning algorithms to select the best modal data subset, optimize and adjust according to task requirements, and output code generation strategies;

[0011] The static analysis module uses abstract syntax trees and control flow graphs to perform static analysis on the code generation strategy, identify performance bottlenecks and potential errors, optimize them, and generate preliminary code;

[0012] The test evaluation module uses mutation testing to evaluate and improve unit tests and integration tests, evaluates code quality and performance through continuous integration and continuous delivery, and outputs code evaluation reports and optimization suggestions.

[0013] As a preferred solution of the multimodal code automatic generation and optimization system of the present invention, the data processing module collects raw data and performs data cleaning and preprocessing to construct the raw data set. The specific steps are as follows:

[0014] Use the Representational State Transfer API to collect raw data, create a data directory structure archive by task ID, and output a structured raw data sample set;

[0015] Use placeholder maps to automatically fill in missing values, then use Z-score normalization and isolation forest algorithm to perform data normalization and data anomaly detection, and output a cleaned high-quality data set;

[0016] Use key phrase extraction to extract text, identify function names to extract function structure from code, and use structured extraction to parse event sequences from logs. These are then reorganized according to unified field specifications to construct data entries with a unified structure.

[0017] Generate a unique hash identifier for the data entry, combined with rule label extraction to generate the original data set.

[0018] As a preferred solution of the multimodal code automatic generation and optimization system described in the present invention, the task analysis module combines the CLIPS rule engine and the knowledge graph with the original data to analyze the task description and constraints, identify the task goals and requirements, and output the task risk assessment report and task intent set. The specific steps are as follows:

[0019] Use domain knowledge graphs and natural language processing toolkits to map text semantics to entity nodes and attribute edges in the graph to build a task semantic graph;

[0020] Use CLIPS rule templates to define behavioral goals, resource requirements, and context constraints, substitute graph entities into preset rules for reasoning, and output task intent sets;

[0021] A CLIPS-based multi-level reasoning and task dependency graph construction mechanism records the triggering path of each reasoning conclusion, identifies the source of conflict through backtracking, generates dependency chains, and outputs a structured risk node chain graph.

[0022] Combining text generation templates and structured export tools, the task intent set and risk node chain diagram are transformed into a readable report to output the task risk assessment report.

[0023] As a preferred solution of the multimodal code automatic generation and optimization system described in the present invention, the data fusion module selects a data source based on the task risk assessment report and the task intent set, uses the weighted average method to perform data fusion, and outputs the fused multimodal data set. The specific steps are as follows:

[0024] Use natural language processing methods to identify and extract task elements from task risk assessment reports and task intent sets, use dependency syntax analysis to process the relationships between task nodes, and combine knowledge graphs to output task node combinations and task node dependency graphs.

[0025] Apply relationship extraction to analyze the association between task nodes and data sources, perform priority matching through a weighted matching algorithm, and output an accurate mapping between data requirements and data sources;

[0026] Based on the combination of task nodes, task node dependency graph, and precise mapping of data requirements and data sources, weighted average fusion is used to process and fuse different modal data through time series alignment and data preprocessing to generate a multimodal dataset in a unified format.

[0027] As a preferred solution of the multimodal code automatic generation and optimization system described in the present invention, the optimization decision module uses a particle swarm optimization algorithm and a path planning algorithm to select the best modal data subset, optimize and adjust it according to task requirements, and output a code generation strategy. The specific steps are as follows:

[0028] The particle swarm optimization algorithm is used to evaluate the fused multimodal dataset and calculate the optimal modal data subset that meets the code generation task;

[0029] The A algorithm is used to plan the transmission path of the optimal modal data subset, optimize the path of the selected modal data during transmission, and output the optimized data transmission path;

[0030] Based on the selected optimal modal data subset and data transmission path, the gradient descent method is used to perform multi-objective optimization, dynamically fine-tune the feature representation according to task requirements, and output the modal data that matches the task;

[0031] Use the preset code generation template to convert the modal data and data transmission path matching the task into the code execution strategy required by the task.

[0032] As a preferred solution of the multimodal code automatic generation and optimization system described in the present invention, the static analysis module uses an abstract syntax tree and a control flow graph to perform static analysis on the code generation strategy, identify performance bottlenecks and potential errors, optimize them, and generate preliminary code. The specific steps are as follows:

[0033] Use a syntax parser to convert the code generation strategy into a weighted abstract syntax tree, parse the code structure, generate weights for each node, mark the data modal source, and output an abstract syntax tree with weights and modal labels;

[0034] Generate a control flow graph based on the abstract syntax tree, use the control flow analysis algorithm to calculate edge weights, identify execution paths, and construct a control flow graph with edge weight labels;

[0035] Based on the control flow graph, the proximal policy optimization algorithm is used to optimize the code execution strategy and output the optimized action sequence;

[0036] Using a code converter, the optimized action sequence is reconstructed into an abstract syntax tree to generate preliminary code.

[0037] As a preferred solution of the multimodal code automatic generation and optimization system described in the present invention, the test evaluation module uses mutation testing to evaluate and improve unit testing and integration testing, evaluates the quality and performance of the code through continuous integration and continuous delivery, and outputs a code evaluation report and optimization suggestions. The specific steps are as follows:

[0038] By performing syntax replacement, boundary perturbation, and control structure adjustment on the preliminary code, multiple code variants with different structures are constructed to form test evaluation samples, and a set of code variants with diverse structures is output;

[0039] Apply the existing unit test set to the code variant set, record the damage effect of the test cases and generate a coverage matrix. Through coverage clustering and hole analysis, output the reconstructed high-coverage unit test set;

[0040] Based on the reconstructed unit test set, the module control dependencies and interface call paths are extracted to generate a new control flow graph. The integration test set is automatically generated through the path-driven mechanism, and an integration test set with complete paths is output.

[0041] Through continuous integration and continuous delivery, we perform automated testing, collect mutation coverage, performance indicators, and resource utilization evaluation parameters, and generate code evaluation reports and optimization suggestions.

[0042] The beneficial effects of the present invention are: by semantically aligning natural language task descriptions with multimodal data such as code, logs, and requirement documents, the accuracy of task intent recognition is improved and the code generation deviation caused by single modal analysis is reduced; by dynamically screening the optimal modal data subset and optimizing the data transmission path, the computing resource consumption is reduced while ensuring code performance, thereby achieving the global optimization of the code generation strategy. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0044] Figure 1 Flowchart of the multimodal-based automatic code generation and optimization system.

[0045] Figure 2 This is the flow chart of the data acquisition and preprocessing module.

[0046] Figure 3 This is a diagram of multimodal data fusion and task matching.

[0047] Figure 4 Generate and optimize test flow charts for code. DETAILED DESCRIPTION

[0048] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0049] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0050] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0051] Reference Figures 1 to 4 , is an embodiment of the present invention, which provides a multimodal-based automatic code generation and optimization system, comprising the following steps:

[0052] The data processing module collects raw data and performs data cleaning and preprocessing to construct the original data set.

[0053] Furthermore, the raw data is collected using the Representational State Transfer API, a data directory structure is established and archived according to the task ID, and a structured raw data sample set is output;

[0054] Specifically, the Representational State Transfer API is used to call the data collection interface of the external platform and local service to receive data, including natural language descriptions, historical code snippets, development logs, compilation feedback information, etc. A unique task identifier is attached as a request parameter for each collection task, and the original multimodal data set organized by task ID is output; a corresponding index table is established to record the task ID, collection time, data type and file path, and task-level archive data with directory structure and metadata index is output; the data content is parsed, fields are extracted, and irregular fields are filled with default values, and the original data set with unified structure and standardized fields is output;

[0055] The best solution is to improve the efficiency and quality of data processing through multi-level data collection and reorganization processes. The use of the Representational State Transfer API for data collection, the unified definition of data formats and the archiving management of task IDs have improved the standardization level of data collection and the automation and standardization of data collection. The data directory structure constructed by task IDs ensures that each piece of data has a clear index and hierarchy, which improves maintainability and flexibility and enhances the organization and manageability of data.

[0056] Use placeholder maps to automatically fill in missing values, then use Z-score normalization and isolation forest algorithm to perform data normalization and data anomaly detection, and output a cleaned high-quality data set;

[0057] Specifically, for missing values ​​in a high-quality data set, placeholder mapping technology is used to mark them, generating a marking matrix. The placeholder mapping and matrix decomposition algorithm are used to complete the data, infer a reasonable filling scheme for the missing values, and construct a completed high-quality data set. The high-quality data set is standardized, the mean of the data is adjusted to 0, the standard deviation is adjusted to 1, and a standardized data set is output, in which the mean of each feature is 0, the standard deviation is 1, and all features are on a uniform scale. For each data point, the isolation depth in all trees is calculated using the isolation forest algorithm, and the degree of abnormality is measured by the average depth. The outliers are calculated and removed from the data set, and the cleaned high-quality data set is output.

[0058] The better approach is to use placeholder maps to automatically fill in missing values, improve the continuity and integrity of the data, and use matrix decomposition to infer the filling of missing values, retaining the inherent correlation between data to improve data quality. Z-value normalization eliminates the dimensional differences between eigenvalues, allowing data of different dimensions to be compared under the same standard, enhancing the scientific nature and reliability of data analysis. The use of the isolation forest algorithm improves the reliability and accuracy of the data and enhances the stability of data analysis.

[0059] Use key phrase extraction to extract text, identify function names to extract function structure from code, and use structured extraction to parse event sequences from logs. These are then reorganized according to unified field specifications to construct data entries with a unified structure.

[0060] Specifically, key phrase extraction is used to perform word segmentation, part-of-speech tagging and stop word filtering on the task description text, and the corresponding words of the task description text are output; the abstract syntax tree is used to perform lexical analysis and grammatical parsing on the original code text, the function definition node in the abstract syntax tree is extracted, and the function-level structured code containing the function name, function body, comments and call structure is output; the log text is read using a sliding window and processed in lines, and the template mining algorithm is applied to perform template clustering on the log format, extract the variable and constant parts, and output the formatted event log sequence; the extracted keywords, function structure and log events are mapped to a unified field respectively, and associated and integrated based on the task ID so that all modal data correspond to a task unit, and the field content is normalized to construct data entries with a unified structure;

[0061] The better approach is to use the placeholder map method to complete the structural consistency of missing items in the original data, thereby improving the integrity and processability of the data sample; use the Z-value normalization method to normalize the scale of numerical data, enhancing the comparability and computational stability between different data features; and combine the isolation forest algorithm to detect outliers in the multidimensional space of the data set, thereby improving the ability to identify outliers and abnormal distributions, thereby enhancing the robustness of the overall data quality;

[0062] Generate a unique hash identifier for the data entry, combine it with rule label extraction, and generate the original data set;

[0063] Specifically, the fields in the data entry are concatenated, the concatenated string is encoded using a standard encoding method, and the digest value is passed to a hash function to generate a unique hash identifier for the data entry. Based on network configuration, interface calls, and business logic, rule matching is performed on text fields and code fields respectively, and one or more sets of tags are assigned to data entries that meet the conditions, generating raw data with semantic context tags.

[0064] Preferably, the hash identification generation mechanism ensures the uniqueness, integrity and immutability of data entries, while providing accurate identification for the data; the rule label extraction mechanism improves the semantic hierarchical management capabilities and automatic classification efficiency of the original data, and increases the speed of indexing and refined data scheduling; the original data set constructed by combining the two is traceable, scalable and searchable.

[0065] The task analysis module combines the CLIPS rule engine with the knowledge graph, and analyzes the task description and constraints based on the original data, identifies the task objectives and requirements, and outputs a task risk assessment report and a task intent set.

[0066] Use domain knowledge graphs and natural language processing toolkits to map text semantics to entity nodes and attribute edges in the graph to build a task semantic graph;

[0067] It should be noted that the domain knowledge graph and natural language processing toolkit are used to map text semantics to entity nodes and attribute edges in the graph to construct a task semantic graph;

[0068] Use the natural language processing toolkit to identify the core semantic units in the task text, perform word segmentation, part-of-speech tagging, dependency parsing, and named entity recognition, extract task actions, objects, and conditional elements, and output a set of structured semantic units;

[0069] Specifically, the text regularization module is used to clean the input task description text, including deleting invalid characters and unifying punctuation formats. The format standardization tool is used to convert all letters to lowercase, and the expressions of numbers, units, symbols, etc. are unified. The word segmentation engine is used to finely split the text, dividing the continuous text into basic word units with semantic meaning to form a preliminary word sequence. The part-of-speech recognition tool is used to assign a part-of-speech tag to each word segmentation result. Combined with the syntactic structure analyzer, the grammatical dependency between words is identified. The entity recognition tool is used to identify proprietary entities with specific meanings in the text. Finally, three key semantic elements in the task description are extracted: task action, task object, and conditions and constraints. These semantic elements are output in a structured format to construct a set of structured semantic units.

[0070] The better ones improve the quality and consistency of the input text through task text cleaning and standardization, eliminate noise symbols and non-standard expressions, and ensure the parsability and accuracy of the text in subsequent processing. Through word segmentation and part-of-speech tagging, the ability to understand each word in the text is improved. Through the output of structured semantic units, the depth and flexibility of understanding of complex task descriptions are improved.

[0071] Combine the domain knowledge graph with the predefined term mapping rule table, bind the semantic unit set to the entity nodes in the graph, and output the list of semantic units bound to the graph entity nodes;

[0072] Specifically, a knowledge graph is used to construct knowledge resources based on a specific field and a domain knowledge graph. The entity nodes in the knowledge graph represent concepts or objects in the field, and the relationships between entity nodes are represented by edges. Based on the terms and concepts in the specific field, a term mapping rule table is established, which lists the mapping relationships between terms and entity nodes in the knowledge graph. The semantic units extracted from the task text are compared with the term mapping rule table to identify the mapping relationships with the entity nodes in the knowledge graph. The semantic units in the task text are bound to the entity nodes in the knowledge graph, and a list of semantic units bound to the entity nodes of the graph is output.

[0073] The better approach is to improve the ability to understand key information in task descriptions by combining knowledge graphs with term mapping rule tables; to improve the structuring and semantic hierarchy of task descriptions by binding semantic units to entity nodes; to improve domain adaptability by applying domain knowledge graphs, and to be able to efficiently use domain knowledge to deeply analyze task descriptions, identify the core concepts and entities involved in related tasks, and thus enhance the ability to handle complex domain tasks; and to improve the flexibility and accuracy of handling domain-specific terminology through the support of predefined term mapping rule tables, so that the terminology in tasks is consistent with the knowledge graph entity nodes.

[0074] Using the graph edge relationship template library and graph structure generation algorithm, we construct semantic relationship edges and generate a task semantic graph. We construct attribute edges based on the semantic dependencies between entities, and use the graph reasoning mechanism to complete the logical relationships and build a task semantic graph.

[0075] Specifically, call the edge relationship template library to identify the logical relationship between semantic units, construct semantic edges according to matching rules, and output a structured description of the preliminary semantic relationship edges; use the graph structure generation algorithm to organize it into a directed graph structure, generate a preliminary semantic graph, nodes represent semantic units, edges represent semantic relationships, and construct a preliminary task semantic graph structure; use a custom semantic dictionary to match and annotate the context content of each node in the preliminary semantic graph, combine with the preset conditional rule set, extract the restrictive or conditional semantics between nodes, add the extracted conditions to the graph in the form of attribute edges, and output an enhanced semantic graph structure containing conditional restrictions and task attributes; use the graph traversal algorithm to traverse the entire semantic graph, identify redundant paths, repeated nodes or redundant attribute edges in the structure, streamline multiple equivalent edges and repeated connections that do not affect the logical expression of the same semantic unit, structurally fold the links with too large a logical level span, and construct a task semantic graph with optimized structure;

[0076] The best approach is to eliminate noise, redundancy, and ambiguous expressions in descriptive texts through natural language processing and text preprocessing operations, thereby improving the purity and structure of task information extraction. By extracting and standardizing core semantic units, unstructured elements such as actions, objects, and conditions in natural language are transformed into unified and standardized structured semantic sequences, thereby improving the consistency and operability of semantic recognition. By combining domain knowledge graphs with terminology rules, the accuracy and domain specificity of semantic understanding are enhanced.

[0077] Use CLIPS rule templates to define behavioral goals, resource requirements, and context constraints, substitute graph entities into preset rules for reasoning, and output task intent sets;

[0078] Specifically, the Harbin Institute of Technology language cloud is combined with knowledge graph query technology to analyze the task semantic graph, read the entity nodes related to the task description in the graph, extract their attribute labels and relationship edges, and output a list of entity information, including structured results of graph elements such as task behavior, data resources, and environmental constraints; using the open source rule engine platform as the core reasoning tool, loading preset rule files and task conditions, defining task behavior through rule templates, and injecting the extracted graph entities as fact inputs into the CLIPS environment, outputting a list of activated rules and corresponding reasoning paths; using the built-in forward chain reasoning mechanism of CLIPS, executing the rules that meet the conditions in sequence, generating new task reasoning results, and outputting a structured task intention set that includes task goals, resource requirements, and applicable conditions;

[0079] It should be noted that the preset rules include behavioral target rules, resource requirement rules, and context constraint rules. The behavioral target rules use the conditional pattern matching mechanism of CLIPS to match entity node types in the semantic graph with their semantic attributes; the resource requirement rules match the edge types between task nodes and their dependent resource entities based on the graph relationship structure; and the context constraint rules use CLIPS to constrain matching of context fields in the rules.

[0080] The best approach is to combine knowledge graphs with rule engines to reduce semantic ambiguity in natural language parsing, enhance the logical consistency of task identification and the stability of output, and improve the accuracy and consistency of task semantic identification. The use of explicit rule-based reasoning enhances the interpretability and controllability of the task reasoning process.

[0081] A CLIPS-based multi-level reasoning and task dependency graph construction mechanism records the triggering path of each reasoning conclusion, identifies the source of conflict through backtracking, generates dependency chains, and outputs a structured risk node chain graph.

[0082] Specifically, through the built-in rule matching of CLIPS, multiple rounds of forward reasoning are carried out from the basic entity according to the rule priority and the status of the fact set. The reasoning process progresses in a hierarchical manner, and the output of each layer is used as the input of the next layer. The output generates the task objectives, constraint relationships, dependency nodes and corresponding triggered rule sequences of each stage; combining the graph structure construction algorithm with the dependency parsing strategy, the conclusions generated by each round of reasoning are used as nodes in the graph, and the causal relationships between different reasoning stages are used as edges to generate a directed dependency graph and a structured task dependency graph; during the rule execution process, the facts and preconditions on which each rule triggers are calculated. The system tracks and logs the results of each decision, forms a complete reasoning path chain, and outputs a reasoning path record table. Based on the dependency path backtracking analysis and rule conflict detection algorithm, it starts from the abnormal reasoning conclusion or resource conflict node, traces back all trigger paths along the dependency graph, analyzes rule overlap, resource duplication or logical contradictions, and outputs a list of conflict nodes. Combining graph structure visualization tools with structured data export tools, it exports the identified conflict paths, related nodes, and upstream and downstream dependencies in the form of a structured graph, generates a risk node chain diagram for further processing that can be viewed by users, and outputs a structured risk node chain diagram.

[0083] The best approach is to record each rule triggering path and its prerequisites, making the reasoning process transparent and traceable, thereby enhancing the explainability and traceability of task reasoning. The risk node chain diagram is not only highly structured but also can display high-risk paths or nodes in a targeted manner, making it targeted and practical. The CLIPS rule reasoning engine is highly combinable, and combined with the dependency graph mechanism, it enhances knowledge expansion and rule reuse capabilities. The dependency analysis and conflict identification mechanism enhances robustness and logical consistency, improving the logical consistency and fault tolerance of the overall operation.

[0084] Combine text generation templates and structured export tools to transform the mission intent set and risk node chain diagram into a readable report and output the mission risk assessment report;

[0085] Specifically, use the Microsoft document format processing tool to convert the task intent set and risk node chain diagram into unified data, establish field mapping relationships, including target descriptions, dependency paths, risk levels, conflict sources, etc., and output a standardized intermediate information representation structure; preset multiple text generation templates for different task types, risk levels and dependency chain paths, automatically fill in key content through placeholders and field mapping, and preliminarily generate task risk assessment statement paragraphs and task intent natural language descriptions; use the structured export tool to combine the task intent text, risk node chain diagram visualization chart and related annotations into a complete report template, the chart part combines the graph structure drawing tool to display the dependency relationship and risk path, and the text part uses automatic insertion to complete the organization of the text content, and output a complete task risk assessment report containing task goals, intent descriptions, risk path diagrams and structured descriptions; use document conversion and archiving tools to perform version number marking and directory archiving, support retrieval, review and version tracing, and output a structured archived task risk assessment report.

[0086] The data fusion module selects the data source based on the mission risk assessment report and the mission intent set, uses the weighted average method to perform data fusion, and outputs the fused multimodal data set.

[0087] Furthermore, natural language processing methods are used to identify and extract task elements from task risk assessment reports and task intent sets, dependency syntax analysis is used to process the relationships between task nodes, and combined with knowledge graphs to output task node combinations and task node dependency graphs;

[0088] Specifically, we use Jieba word segmentation, regular expressions and lexical filtering rules to segment the input text into sentences, remove stop words, unify terminology, and output standardized task text corpus; design a multi-layer matching template to extract task elements based on verb phrases, noun phrases and prepositional structures, and output a set of task element triples; use the Stanford syntax analyzer to analyze the grammatical structure of sentences and extract the subject, predicate, object, modifiers and parallel structures in task sentences. Through syntactic relationships, we establish node connections and generate a task node combination graph, where nodes represent task behaviors and edges represent grammatical or semantic connections, and construct a task node combination graph; perform entity matching and relationship alignment on the extracted task elements and the predefined domain knowledge graph, use string similarity calculation and attribute mapping rules to connect semantic nodes to entities in the graph and their upper and lower hierarchies, expand the logical dependency path, construct a task node dependency graph, and output task node combinations and task node dependency graphs;

[0089] The best approach is to use natural language processing methods to improve the ability to understand the semantics of task texts, especially in identifying factors related to task risks and intent, achieving high-precision structured extraction. The integration of knowledge graphs strengthens term alignment, semantic disambiguation, and context completion capabilities, enabling semantic enhancement and standardized mapping of task information. The overall composite process improves the automation level of task information extraction and the efficiency of docking with upstream and downstream modules, enhancing intelligent decision-making capabilities and generalization adaptability.

[0090] Apply relationship extraction to analyze the association between task nodes and data sources, perform priority matching through a weighted matching algorithm, and output an accurate mapping between data requirements and data sources;

[0091] Specifically, the corresponding rules between keywords and data source fields are described, the task description text is parsed, keywords and their dependencies are extracted, and the dictionary library and template rules are compared to determine which task nodes have call requirements for data sources, and a preliminary association table between task nodes and data fields is constructed; a weighted matching algorithm is used to determine the priority and optimal data source, and each pair of candidate items is weighted and scored by setting multiple matching weight factors, such as semantic similarity score, context consistency weight, historical frequency factor and data availability weight, etc. The results are sorted and the highest-scoring match corresponding to each task node is extracted, outputting an accurate mapping between data requirements and data sources;

[0092] The best approach is to combine rule matching with weighted scoring to more accurately capture the data usage intent implicit in the task description, thereby achieving a precise connection between data sources and task requirements, and achieving high-precision static matching between task requirements and data sources. The rules and weights are highly controllable and transparent, making them easy to debug and modify, and enhancing explainability and maintainability.

[0093] Based on the combination of task nodes, task node dependency graphs, and precise mapping of data requirements and data sources, weighted average fusion is used to process and fuse different modal data through time series alignment and data preprocessing to generate a multimodal dataset in a unified format;

[0094] Specifically, the task nodes are combined, the task node dependency graph is accurately mapped with the data requirements and data sources, and the format conversion and unit normalization are uniformly converted into key-value pairs or table forms, the numerical units are unified and normalized, and the timestamp format is standardized to build a multi-modal intermediate data table with consistent format and standardized units; the task node dependency graph is processed by graph traversal, and the data fusion order between modalities is established according to the logical relationship between nodes. It is clear what kind of fusion operation should be performed between each pair of task nodes, such as time synchronization alignment, field merging and splicing, or redundant field elimination. A weight value is set for each fusion relationship to indicate its importance in the fusion, and a fusion process control is generated. Table and fusion weight configuration table; use the sliding window method to match the data with timestamps, select the data with the closest time for pairing, use linear interpolation to complete the points that cannot be aligned, and output a data sample table with consistent time dimension and cross-modal alignment; perform weighted averaging calculation on data fields from different modalities according to preset weights. For non-numeric fields, a weight-based selection mechanism is implemented. The field with the largest weight is the main field, and the others are merged and annotated. The standardized sample data set after multimodal fusion is output, and the field value represents the fusion result; the field integrity of the fusion result is checked and output according to a unified structure template to generate a multimodal data set in a unified format;

[0095] The better ones improve the structural consistency and semantic coherence between multimodal data through structural unification, key-value format standardization, and time alignment; enhance the representativeness and task adaptability of the data fusion process through a weighted average fusion method based on task weight and data confidence; improve the operability and controllability of data organization and processing by clarifying the task node dependency and data source mapping path; and overall enhance the data integration capability, time series coordination capability, and high-quality input construction capability in a multi-source heterogeneous data environment.

[0096] The optimization decision module uses the particle swarm optimization algorithm and path planning algorithm to select the best modal data subset, optimize and adjust it according to task requirements, and output the code generation strategy.

[0097] Furthermore, a particle swarm optimization algorithm is used to evaluate the fused multimodal dataset and calculate the optimal modal data subset that meets the code generation task;

[0098] Specifically, different types of modal data specifications are converted into a unified structured form, and a structured unified modal feature set is output as the input basis for particle swarm search; by calculating the matching score between task requirement keywords and modal content keywords, a penalty is set according to the redundancy weight between modal contents, and the degree of support of any modal combination for the current task goal is measured. A quantifiable fitness function is output to rank the modal combinations; each particle represents a Boolean vector, encoding whether each type of modality is selected, setting the number of particles, maximum iteration rounds, speed range and convergence threshold, and in each round of iteration, performing individual and global optimal updates based on the fitness of the current modal combination, using the position and speed formula to iteratively search for the optimal modal combination, and outputting the Boolean vector of the modal combination with the best fitness, that is, the modal subset that best supports the task;

[0099] It should be noted that the fitness function expression is as follows:

[0100] ;

[0101] in, Represents a given mode The fitness of The degree of matching with the target task requirements, the higher the fitness value, the better the mode The more it meets the task requirements, the better the performance, (−∞,+ ), Represents the weight of the coverage-related part, which controls the influence of coverage in calculating fitness. (0,+ ), What is measured is the modality and target tasks The degree of match or coverage between To what extent can the task be met demand, [0,1], Represents the weight of the overlap-related part, which controls the influence of overlap in fitness calculation. It is just the name of the function and does not represent a value or constant. It must be used with input variables. and use, (0,+ ), Indicates modality The redundancy or similarity between the sub-modalities in [0,1] where 0 indicates mode With the task Completely mismatched, 1 indicates modal Complete coverage of tasks demand, It is just the name of the function and does not represent a value or constant. It must be used with input variables. use.

[0102] The A algorithm is used to plan the transmission path of the optimal modal data subset, optimize the path of the selected modal data during transmission, and output the optimized data transmission path;

[0103] It should be noted that the A algorithm is used to plan the transmission path of the optimal modal data subset, optimize the path of the selected modal data during transmission, and output the optimized data transmission path;

[0104] Furthermore, data priority is calculated based on task requirements, and the particle swarm optimization algorithm is used to evaluate the priority of different modal data and output a priority list;

[0105] Specifically, by analyzing the task description text, identifying the task objectives, constraints, and data dependencies, the task requirements are structured through the entities and attributes in the knowledge graph to form a semantic vector for the task and generate structured data of the task requirements. The characteristics of each modality are analyzed, encoded, and compared with the task requirements. The correlation and influence between the data and the task are calculated, and the feature vector of each modality data is generated to indicate its matching degree with the task and its role in task completion. The particle swarm optimization algorithm is used to sort and prioritize the feature vectors of multimodal data. After multiple iterations, the particle swarm algorithm outputs the optimal data priority sorting and finally outputs a data priority list.

[0106] The best approach is to analyze and structure task requirements to ensure a high degree of alignment between task requirements and data priorities. The particle swarm optimization algorithm is used to evaluate data priorities, and efficient multi-objective optimization improves the overall system execution efficiency. A comprehensive evaluation based on multiple factors, including task requirements, network status, and data characteristics, can intelligently adjust data priorities to ensure efficient task completion and enhance the intelligence and flexibility of the system.

[0107] Use heuristic A algorithm to perform preliminary path planning, dynamically adjust the transmission path, and output the optimized data transmission path;

[0108] Specifically, a weighted directed graph is constructed using the connection relationships between multimodal data source nodes, processing nodes, and terminal nodes. The nodes in the graph represent data sources, data processing, or target devices, and the weights of the edges in the graph are determined by real-time network status, including indicators such as bandwidth, latency, and packet loss rate. Each type of modal data is accompanied by transmission attribute labels, and a directed graph with network performance weights and modal characteristic labels is output, providing structural input for path planning. The heuristic A algorithm is used to set evaluation strategies based on different data types and output parameters for path search, allowing the algorithm to find the optimal path while meeting performance conditions. The A algorithm is applied to the constructed graph structure to search and select the path from the source to the target of the data. Starting from the starting node, the algorithm traverses adjacent nodes and selects priority nodes based on the cost function. All feasible paths are recorded while meeting the constraints. The path performance is evaluated for different modal data, and a set of feasible data transmission paths are output, which are ranked according to the comprehensive cost. For the selected path, if a network status change is found, the path update is triggered and the change frequency is recorded. The path update prioritizes the minimum range recalculation principle and outputs the optimized data transmission path.

[0109] The superior heuristic A algorithm combines multiple constraints and performance parameters to reduce invalid path traversal, shorten calculation time, and improve path planning efficiency. It uses modal data feature tags during path evaluation to ensure that different types of data are transmitted along the most suitable path, enhancing data type adaptability. Through real-time path adjustment and local reconstruction mechanisms, it can quickly reconstruct valid paths even when some network nodes fail or are congested, improving stability, robustness, and recoverability.

[0110] Based on the selected optimal modal data subset and data transmission path, the gradient descent method is used to perform multi-objective optimization, dynamically fine-tune the feature representation according to task requirements, and output the modal data that matches the task;

[0111] Specifically, the structure of the modal data subset is analyzed by combining the content header and metadata, and the word frequency inverse text frequency encoder is used to extract feature vectors for each modality. Automatic zero padding is used to uniformly map all modalities into vectors with the same feature dimension to construct a unified feature matrix; the batch gradient descent algorithm is used for iterative optimization, the gradient is automatically calculated and the feature vector matrix is ​​updated, and the optimal feature matrix is ​​output; the early fusion method is used to splice the features of different modalities at the feature level to construct a unified multimodal feature vector; the genetic algorithm is used to optimize the feature vector according to the task requirements, and the most relevant features are selected by setting the optimization goal, and the feature combination is continuously adjusted according to the optimization algorithm to output the optimal feature subset; the convolutional neural network is used to fine-tune the feature subset, and the feature expression is made more in line with the specific task requirements through end-to-end training, and the final feature representation after task fine-tuning is output; according to the task requirements, the selected feature data is mapped to the target format, the feature data is matched with the task description, data that conforms to the task format is generated, and the modal data matching the task is output;

[0112] The best approach is to improve the accuracy and stability of modal feature representation through the iterative optimization mechanism of the gradient descent method; improve the dynamic adaptability and intelligence of the response through the automated parameter fine-tuning mechanism; enhance the contextual relevance and semantic consistency of the final code generation strategy through the automatic update of the feature representation; and build a fully automated modal data optimization process to reduce the cost of manual intervention and the risk of error.

[0113] Use the preset code generation template to convert the task-matched modal data and data transmission path into the code execution strategy required by the task;

[0114] Specifically, another language recognition tool is used to automatically extract key fields, attributes and contextual semantics from modal data, and output a set of structured semantic representations; the corresponding code structure template is selected according to the task type, and a code skeleton with placeholders is constructed; the template tags are replaced with a key-value mapping table, and the extracted fields, functions, parameters, etc. are embedded in the corresponding positions of the template, and a complete target code segment is output; the Hypertext Transfer Protocol request template is combined with rule splicing to automatically insert path-related data acquisition logic into the generated code, and the target IP, port, and protocol are injected into the data sending block in the template according to the path structure, and a complete code block with path communication statements is output; the code splicing and syntax checker are used to check, organize and output the complete code structure, verify whether it contains the necessary structure, and output the code execution strategy required by the task;

[0115] Better, improve code structure consistency and semantic accuracy, improve the protocol adaptability and response efficiency of task execution by integrating multi-protocol template segment adaptation mechanism; improve generation speed and maintainability by using template-driven fast splicing method; improve code correctness and execution stability by using syntax tree parser auxiliary verification mechanism; improve transparency and traceability between task logic and code strategy by building a structured mapping mechanism.

[0116] The static analysis module uses abstract syntax trees and control flow graphs to perform static analysis on the code generation strategy, identify performance bottlenecks and potential errors, optimize them, and generate preliminary code.

[0117] Furthermore, a grammar parser is used to convert the code generation strategy into a weighted abstract syntax tree, parse the code structure, generate weights for each node, mark the data modal source, and output an abstract syntax tree with weights and modal labels;

[0118] Specifically, the code generation strategy text is divided into tokens using lexical analysis, and then converted into an abstract syntax tree according to the defined grammar rules through syntax analysis, and the basic abstract syntax tree is output; the rule library is used to generate weights according to indicators such as node type, complexity, nesting depth, frequency of use, and scope of influence, and output, and output an abstract syntax tree structure with weight fields; the modal mapping table is used to establish mapping rules between the generation strategy fields and their source modalities, and the original code generation strategy input is combined with its data source through field name, semantic annotation or context annotation, and the modal label is attached to the abstract syntax tree node as an attribute. After modal fusion, multiple modal labels are allowed to be attached, and priorities are set, and the abstract syntax tree node with fields is output; the tree structure is used for encapsulation processing, and the intermediate code structure is used to represent the entire tree. The weight and modal information are merged into each node, and the abstract syntax tree with weight and modal label is output;

[0119] The better approach is to convert the code generation strategy into a structured abstract syntax tree, thereby enhancing the comprehensive perception of the potential structure of the generated code and improving the accuracy of static analysis. The semantic weights are used to improve the pertinence and efficiency of the optimization strategy, and the convergence speed and effectiveness of the optimization algorithm. The weighted and modally labeled abstract syntax tree improves the interpretability and debuggability of code generation and enhances the transparency of the system.

[0120] Generate a control flow graph based on the abstract syntax tree, use the control flow analysis algorithm to calculate edge weights, identify execution paths, and construct a control flow graph with edge weight labels;

[0121] Specifically, an abstract syntax tree traversal algorithm is used to identify control structure nodes. Sequential statements are aggregated into a single basic block, preserving semantics, and a basic block list is output. Static semantic rules are used to determine the execution dependencies and control flow connections between basic blocks, and the control flow graph topology is output. Static analysis indicators and heuristic strategies are combined to add probabilistic weights to control flow edges to construct a basic weighted control flow graph. Basic block nodes are annotated with semantic labels, edge weights are added to the weight field, and a control flow graph with edge weight labels is output.

[0122] The best approach is to improve the ability to identify key execution paths through control dependency mapping and edge weight analysis; improve the context-awareness of static analysis through the fusion of edge weights and semantic labels; improve the controllability and explainability of code execution strategy optimization through the construction of execution path graphs; and improve reusability and scalability through the standardized output of structural information.

[0123] Based on the control flow graph, the proximal policy optimization algorithm is used to optimize the code execution strategy and output the optimized action sequence;

[0124] Specifically, the graph structure normalization method is used to normalize the weighted control flow graph, so that the identification of nodes and edges is unified, the weight information is clear and accessible, and a standardized control flow graph with clear structure and accessible edge weights is output; all feasible path fragments are extracted from the control flow graph, each path segment is represented as a state node sequence, a state space is constructed, and the state is represented as a numerical sequence using a unique encoding method, and a structured state space set and its corresponding numerical encoding are output; for the optimizable structural types in the control flow graph, optional optimization operations are defined, each type of operation is assigned a unique action identifier, rewards are set according to the policy objectives, and a discrete action set and a policy evaluation target are output; the state space is traversed, all legal actions are attempted to be executed for each state, the control flow graph transformation is simulated, and the local structure of the transformed weighted control flow graph is evaluated. If the transformation brings benefits, it is recorded as the optimal policy action, and the optimized action sequence is output; combining sequence encoding and abstract syntax mapping, the optimized actions are output in the form of a sequence, each action contains fields such as node identification, operation type, target path fragment and expected impact, a standard structured file is constructed, and the optimized action sequence is output;

[0125] The best approach is to improve the ability to analyze code execution bottlenecks through control flow graph edge weight modeling and path identification; improve the stability and convergence of code optimization through the adoption of proximal policy optimization algorithms; improve the adaptability and generalization capabilities of various optimization objectives through the discretization definition and reward of the action space; and improve the interpretability and deployment controllability of the system through rule-driven development.

[0126] Using a code converter, the optimized action sequence is reconstructed into an abstract syntax tree to generate preliminary code;

[0127] Specifically, a custom instruction parser is used to read and parse the operation types contained in the optimization action sequence, and generate an operation instruction stream that can be mapped to the abstract syntax tree structure; a syntax tree traversal algorithm is used to parse the operation instruction stream, locate the target node or substructure that needs to be modified in the syntax tree, and generate a node mapping index table; a tree structure modification library is used to perform structural addition, deletion, and modification operations on the abstract syntax tree according to the node mapping index table, and output an abstract syntax tree that has been structurally optimized; the abstract syntax tree is verified in combination with the syntax specification, and a code generator is used to generate source code text to ensure that the semantics are compatible with the language specification, without syntax conflicts or structural omissions, and to generate preliminary code;

[0128] The better ones are to improve the certainty and stability of code generation through the structured instruction mapping mechanism; improve the semantic fidelity through the precise syntax tree rewriting mechanism; improve the code generation efficiency through the node positioning and matching mechanism; improve the code usability and compilability through syntax integrity verification; and enhance the cross-modal code comprehension ability through the modal tag inheritance mechanism.

[0129] The test evaluation module uses mutation testing to evaluate and improve unit tests and integration tests, evaluates code quality and performance through continuous integration and continuous delivery, and outputs code evaluation reports and optimization suggestions.

[0130] Furthermore, by performing syntax replacement, boundary perturbation, and control structure adjustment on the preliminary code, multiple code variants with different structures are constructed to form test evaluation samples, and a set of code variants with diverse structures is output;

[0131] It should be noted that by performing syntax replacement, boundary perturbation and control structure adjustment on the preliminary code, multiple code variants with different structures are constructed to form test evaluation samples, and a set of code variants with diverse structures is output;

[0132] Furthermore, the control structures and boundary locations in the preliminary code are identified and extracted, candidate points that can be mutated are determined, and a structured list of candidate points is output;

[0133] Specifically, a syntax parser is used to parse the initially generated source code into an abstract syntax tree to obtain the structural hierarchical relationship of the source code and the complete abstract syntax tree structure; a syntax tree traversal algorithm is used to extract nodes of all control structure types, nodes that meet the control structure label are matched in the abstract syntax tree, and a set of control structure candidate points is output; comparison operations are searched in control structure and expression nodes, the relationship between constants and variables is identified, boundary values ​​are marked, and a set of boundary position candidate points is output; the control structure candidate point set and the boundary position candidate point set are combined to mark the point type, source code location, related variables and operators and other meta-information, and a structured candidate point list is output;

[0134] The better approach is to strengthen the understanding of the code semantic level by constructing an abstract syntax tree, improving the accuracy of code structure analysis; extract control structure types through static traversal and syntax node screening technology, improve the efficiency of identifying candidate mutation points, and make the goals of control structure adjustment operations clear and accurate; through a unified structured candidate point format output method, the availability and versatility of mutation point information are improved; by unifying the abstraction of control logic and boundary conditions into candidate mutation units, the construction foundation of test sample diversity is improved, and the coverage, fault triggering capability and robustness evaluation level in automated testing are enhanced;

[0135] Using mutation rules such as syntax replacement, boundary perturbation, and control structure adjustment, we generate multiple code variants with different structures and behaviors, and build a diverse set of preliminary code variants.

[0136] Specifically, the abstract syntax tree and regular expressions are used to traverse the syntax nodes in the preliminary code. The basic syntax units are identified through the abstract syntax tree. Syntax nodes that meet specific mutation rules are selected and replaced according to preset rules. Regular expressions or template matching are used to perform syntax replacement on strings, function calls, conditional expressions, and other parts of the code to generate a set of variants. Boundary conditions in the code are identified, especially boundary values ​​in loop structures and conditional judgments. Boundary values ​​are adjusted, different types of boundaries are perturbed, and multiple perturbed code variants are output. Control flow graph analysis and conditional branch rewriting are used to adjust the control structure of the program. The control flow graph of the program is analyzed, the control structure is identified, and the control structure is mutated. In the conditional judgment of the program, the branch execution order is changed. In the loop structure, the iteration method is adjusted to output multiple variants. Metadata annotation and multi-dimensional analysis are used to organize and aggregate the generated variants. All generated code variants are uniformly organized and classified according to function and execution path. Each variant is marked with a mutation point type to construct a diverse set of preliminary code variants.

[0137] The better one is that, through structural mutation technology, a set of code variants is constructed from multiple angles at the syntax, boundary, and control levels. This ensures the differentiated expression of functional behavior, improves the diversity of test samples, expands the coverage of code logic and execution paths, and enhances the discoverability of software defects and the robustness verification capabilities.

[0138] Add metadata annotations to code variants, indicating the mutation type, mutation point location, and differences before and after the mutation. Organize the variants by function and execution path, and output a set of code variants with diverse structures.

[0139] Specifically, an abstract syntax tree comparison tool is used to perform difference analysis between the original code and the variant code, traversing the abstract syntax tree nodes to compare the differences. For each mutation, the mutation type, the position of the mutation point in the source code, and the changes before and after the mutation content are extracted to build a standardized mutation metadata structure; a code comment injector is used to embed metadata information into the variant code, and comments are automatically inserted before and after the mutation point to indicate the mutation type and number, forming an annotated code variant; a static analysis tool is used to extract the function module affiliation and potential impact path of each variant, and the variants are classified according to the control flow graph, hierarchically classified by module, path, and impact area, and a structured classification index is generated; all variant codes with annotations and metadata are bound and stored with their index structure, and a set of code variants with diverse structures is output;

[0140] The best approach is to improve the traceability and explainability of code variants by using abstract syntax tree difference analysis and standardized metadata structures. The control flow approach categorizes variants by functional path, enhancing the structural organization and scheduling efficiency of test samples. Combined with structured metadata management, the test suite's ability to detect potential defects and boundary condition errors is improved, thereby enhancing test coverage and error detection capabilities.

[0141] Apply the existing unit test set to the code variant set, record the damage effect of the test cases and generate a coverage matrix. Through coverage clustering and hole analysis, output the reconstructed high-coverage unit test set;

[0142] Specifically, code variants with diverse structures are loaded into the test environment one by one, and the test cases in the existing unit test set are applied to each variant respectively. The killing effect of the variant is judged by monitoring abnormal behavior to generate a killing matrix; the code coverage information of each test case when executing the original code is recorded and summarized into a coverage matrix. The matrix elements represent whether the test case covers the code elements, and a structured coverage matrix is ​​output; a density clustering algorithm is used to perform similarity analysis on the coverage matrix, analyze the overlap between test cases, cluster highly redundant test cases into one category, retain the central test case, eliminate the redundancy, and output the clustering results; combining matrix sparsity analysis with Boolean vector inversion method to perform void detection, extract void areas in the coverage matrix, locate these void areas to control structures or logical paths, and synthesize new targeted test cases based on syntax tree analysis and symbolic execution to form a void test case patch set; uniformly select test cases with high killing rate, high path coverage and low redundancy, add void patch test cases, verify the coverage and effectiveness of the reconstructed set on the variant set and the original code, and output the reconstructed high-coverage unit test set;

[0143] The better ones improve the effectiveness and discrimination of test cases through mutation testing, compress redundancy and improve test execution efficiency through coverage clustering, strengthen coverage completeness and path diversity through hole analysis, and improve the interpretability and visibility of test evaluation through structured matrix analysis;

[0144] Based on the reconstructed unit test set, the module control dependencies and interface call paths are extracted to generate a new control flow graph. The integration test set is automatically generated through the path-driven mechanism, and an integration test set with complete paths is output.

[0145] Specifically, static analysis is performed on the tested code corresponding to the reconstructed unit test set, the control dependency edges in each module are identified, all external interface calls and cross-module dependencies are extracted, a call path graph is established, and a control flow dependency graph and an interface call path graph are constructed; the control flow dependency graph and the interface call path graph are unified and integrated into an extended control flow graph, and a path label mechanism is added to identify paths, boundary paths and risk paths. For asynchronous calls and callback chains, a delayed graph construction and back-edge reconstruction strategy are adopted to generate a path-complete control flow graph for multi-module fusion; all critical paths in the path-complete control flow graph that are not covered by existing tests are traversed, and symbolic execution and constraint solving are used to generate a test input combination that meets the constraint conditions for each path, automatically synthesize test scripts for multi-module interactions, inject verification assertions, and output a path-complete integration test set;

[0146] Through continuous integration and continuous delivery, we perform automated testing, collect mutation coverage, performance indicators, and resource utilization evaluation parameters, and generate code evaluation reports and optimization suggestions.

[0147] Specifically, use continuous integration configuration files to automatically trigger the build and test process when the code is submitted, define build tasks through class markup language configuration files, build logs, unit test logs and mutation test start signals; use open source tools for mutation testing, insert the mutation test phase into the continuous integration process, automatically perform semantic perturbations on the code and then run the test set, count the number and types of variants of the tested cases, generate mutation coverage, and output mutation test reports, coverage reports and path kill matrices; in the continuous integration and continuous delivery pipeline, embed a performance analyzer or monitoring agent when executing the test code to sample function call time, CPU usage peak, memory allocation frequency and other indicators, and output performance trend charts, resource utilization curves, and hot function call stack records; combine custom script aggregation results, aggregate test coverage, mutation kill rate, performance reports and other data sources, generate suggestions based on static rules, trigger suggestions according to the rules, and generate code evaluation reports and optimization suggestions;

[0148] Better, through the automated trigger mechanism, the real-time and consistency of code test coverage are enhanced, the mutation test indicators are improved, the lethality and defect sensitivity of the test set are improved, structured code evaluation reports and optimization suggestions are generated, and the operability and visualization of test results are enhanced.

[0149] In summary, the present invention achieves the following: semantic alignment of natural language task descriptions with multimodal data such as code, logs, and requirement documents to improve the accuracy of task intent recognition and reduce code generation deviations caused by single modal analysis; dynamically screens the optimal modal data subset and optimizes the data transmission path to reduce computing resource consumption while ensuring code performance, thereby achieving the global optimization of the code generation strategy.

[0150] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A multimodal code automatic generation and optimization system, characterized by: include, Data processing module collects raw data and performs data cleaning and preprocessing to construct the original data set; The original dataset includes natural language descriptions, historical code snippets, development logs, and compilation feedback information; The task analysis module combines the CLIPS rule engine with the knowledge graph and the original data to analyze the task description and constraints, identify the task goals and requirements, and output the task risk assessment report and task intent set; The data fusion module selects data sources based on the mission risk assessment report and mission intent set, uses the weighted average method to perform data fusion, and outputs the fused multimodal data set. The specific steps are as follows: Use natural language processing methods to identify and extract task elements from task risk assessment reports and task intent sets, use dependency syntax analysis to process the relationships between task nodes, and combine knowledge graphs to output task node combinations and task node dependency graphs. Apply relationship extraction to analyze the association between task nodes and data sources, perform priority matching through a weighted matching algorithm, and output an accurate mapping between data requirements and data sources; Based on the combination of task nodes, task node dependency graphs, and precise mapping of data requirements and data sources, weighted average fusion is used to process and fuse different modal data through time series alignment and data preprocessing to generate a multimodal dataset in a unified format; The optimization decision module uses particle swarm optimization and path planning algorithms to select the best modal data subset, optimize and adjust according to task requirements, and output code generation strategies; The static analysis module uses abstract syntax trees and control flow graphs to perform static analysis on the code generation strategy, identify performance bottlenecks and potential errors, optimize them, and generate preliminary code. The specific steps are as follows: Use a syntax parser to convert the code generation strategy into a weighted abstract syntax tree, parse the code structure, generate weights for each node, mark the data modal source, and output an abstract syntax tree with weights and modal labels; Generate a control flow graph based on the abstract syntax tree, use the control flow analysis algorithm to calculate edge weights, identify execution paths, and construct a control flow graph with edge weight labels; Based on the control flow graph, the proximal policy optimization algorithm is used to optimize the code execution strategy and output the optimized action sequence; Using a code converter, the optimized action sequence is reconstructed into an abstract syntax tree to generate preliminary code; The test evaluation module uses mutation testing to evaluate and improve unit tests and integration tests, evaluates code quality and performance through continuous integration and continuous delivery, and outputs code evaluation reports and optimization suggestions.

2. The multimodal automatic code generation and optimization system according to claim 1, wherein: The data processing module collects raw data and performs data cleaning and preprocessing to construct the original data set. The specific steps are as follows: Use the Representational State Transfer API to collect raw data, create a data directory structure archive by task ID, and output a structured raw data sample set; Use placeholder maps to automatically fill in missing values, then use Z-score normalization and isolation forest algorithm to perform data normalization and data anomaly detection, and output a cleaned high-quality data set; Use key phrase extraction to extract text, identify function names to extract function structure from code, and use structured extraction to parse event sequences from logs. These are then reorganized according to unified field specifications to construct data entries with a unified structure. The key phrase extraction is a method in the field of natural language processing, the purpose of which is to extract task description text and perform word segmentation, part-of-speech tagging and stop word filtering; The key phrase extraction refers to extracting text to perform word segmentation, part-of-speech tagging and stop word filtering; Generate a unique hash identifier for the data entry, combined with rule label extraction to generate the original data set.

3. The multimodal automatic code generation and optimization system according to claim 2, wherein: The task analysis module combines the CLIPS rule engine with the knowledge graph, analyzes the task description and constraints with the original data, identifies the task objectives and requirements, and outputs the task risk assessment report and task intent set. The specific steps are as follows: Use domain knowledge graphs and natural language processing toolkits to map text semantics to entity nodes and attribute edges in the graph to build a task semantic graph; Use CLIPS rule templates to define behavioral goals, resource requirements, and context constraints, substitute graph entities into preset rules for reasoning, and output task intent sets; A CLIPS-based multi-level reasoning and task dependency graph construction mechanism records the triggering path of each reasoning conclusion, identifies the source of conflict through backtracking, generates dependency chains, and outputs a structured risk node chain graph. Combining text generation templates and structured export tools, the task intent set and risk node chain diagram are transformed into a readable report to output the task risk assessment report.

4. The multimodal automatic code generation and optimization system according to claim 3, wherein: The optimization decision module uses the particle swarm optimization algorithm and path planning algorithm to select the best modal data subset, optimize and adjust according to the task requirements, and output the code generation strategy. The specific steps are as follows: The particle swarm optimization algorithm is used to evaluate the fused multimodal dataset and calculate the optimal modal data subset that meets the code generation task; The A algorithm is used to plan the transmission path of the optimal modal data subset, optimize the path of the selected modal data during transmission, and output the optimized data transmission path; Based on the selected optimal modal data subset and data transmission path, the gradient descent method is used to perform multi-objective optimization, dynamically fine-tune the feature representation according to task requirements, and output the modal data that matches the task; Use the preset code generation template to convert the modal data and data transmission path matching the task into the code execution strategy required by the task.

5. The multimodal code automatic generation and optimization system according to claim 4, characterized in that: The test evaluation module uses mutation testing to evaluate and improve unit testing and integration testing, evaluates the quality and performance of the code through continuous integration and continuous delivery, and outputs code evaluation reports and optimization suggestions. The specific steps are as follows: By performing syntax replacement, boundary perturbation, and control structure adjustment on the preliminary code, multiple code variants with different structures are constructed to form test evaluation samples, and a set of code variants with diverse structures is output; Apply the existing unit test set to the code variant set, record the damage effect of the test cases and generate a coverage matrix. Through coverage clustering and hole analysis, output the reconstructed high-coverage unit test set; Based on the reconstructed unit test set, the module control dependencies and interface call paths are extracted to generate a new control flow graph. The integration test set is automatically generated through the path-driven mechanism, and an integration test set with complete paths is output. Through continuous integration and continuous delivery, we perform automated testing, collect mutation coverage, performance indicators, and resource utilization evaluation parameters, and generate code evaluation reports and optimization suggestions.

6. The multimodal code automatic generation and optimization system according to claim 5, characterized in that: The A algorithm is used to plan the optimal modal data transmission path, optimize the path of the selected modal data during transmission, and output the optimized data transmission path. The specific steps are as follows: Calculate data priority based on task requirements, use particle swarm optimization algorithm to evaluate the priority of different modal data, and output a priority list; Use the heuristic A algorithm for preliminary path planning, dynamically adjust the transmission path, and output the optimized data transmission path.

7. The multimodal code automatic generation and optimization system according to claim 6, characterized in that: The domain knowledge graph and natural language processing toolkit are used to map text semantics to entity nodes and attribute edges in the graph to construct a task semantic graph. The specific steps are as follows: Use the natural language processing toolkit to identify the core semantic units in the task text, perform word segmentation, part-of-speech tagging, dependency parsing, and named entity recognition, extract task actions, objects, and conditional elements, and output a set of structured semantic units; Combine the domain knowledge graph with the predefined term mapping rule table, bind the semantic unit set to the entity nodes in the graph, and output the list of semantic units bound to the graph entity nodes; Utilizing the graph edge relationship template library and graph structure generation algorithm, semantic relationship edges are constructed and task semantic graphs are generated. Attribute edges are constructed based on the semantic dependencies between entities. The graph reasoning mechanism is combined to complete the logical relationships and construct a task semantic graph.

8. The multimodal code automatic generation and optimization system according to claim 7, wherein the steps of performing syntax replacement, boundary perturbation, and control structure adjustment on the preliminary code to construct multiple code variants with different structures, forming test evaluation samples, and outputting a set of code variants with diverse structures are as follows: Identify and extract control structures and boundary locations in the preliminary code, determine candidate points that can be mutated, and output a structured list of candidate points; Using grammatical substitution, boundary perturbation, and control structure adjustment mutation rules, we generate multiple code variants with different structures and behaviors, and construct a diverse set of preliminary code variants. The code variants are annotated with metadata to indicate the mutation type, mutation point location, and differences before and after the mutation. The variants are sorted according to function and execution path, and a set of code variants with diverse structures is output.

Images