A software requirement modeling method based on an artificial intelligence large model

Abstract

The application discloses a software requirement modeling method based on an artificial intelligence large model, relates to the technical field of requirement modeling, and comprises the following steps: converting a fused semantic element set into nodes and edges, adding an uncertainty mark to an edge with insufficient association basis in the conversion process, constructing a requirement semantic knowledge graph, constructing a requirement model generation scheme based on the requirement semantic knowledge graph, filling the requirement model generation scheme into a preset structured prompt word template, obtaining a structured prompt, driving a multi-modal large model to generate a requirement model draft by using the structured prompt, performing rule compliance checking on the requirement model draft, executing an iterative verification and correction cycle according to a checking result, and obtaining a target requirement model. The application greatly improves the automation efficiency of software requirement modeling while improving the accuracy, completeness and usability of the target requirement model by generating the requirement model draft.

Description

Technical Field

[0001] This invention relates to the field of requirements modeling technology, and in particular to a software requirements modeling method based on a large artificial intelligence model. Background Technology

[0002] With the deep integration of software engineering and artificial intelligence technologies, software requirements engineering is undergoing a paradigm shift from manual analysis to intelligent assisted modeling. Software requirements modeling, as a key bridge connecting users' original needs and system design, focuses on transforming unstructured business descriptions into formal model representations. In the early stages of technological development, requirements modeling mainly relied on analysts writing requirements specifications in natural language. In recent years, with breakthroughs in deep learning, especially multimodal models, using generative AI to automatically understand requirements documents and generate code or models has become a research hotspot. This end-to-end generation method has shown potential in simple scenarios, greatly improving the efficiency of preliminary modeling and bringing new possibilities to the early stages of the software development lifecycle.

[0003] Nevertheless, existing requirements modeling methods still have room for improvement. First, they lack the ability to deeply integrate multimodal requirements materials and fail to establish deep semantic alignment between textual semantics and visual elements. This results in a disconnect between the generated model and the interface interaction logic and business rules, making it impossible to accurately reflect the real requirement of "consistency between text and graphics". Second, they lack intermediate verification mechanisms. Once a logical error occurs in the generated result, it is difficult to locate the root cause of the error and automatically correct it due to the lack of an interpretable intermediate structure. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a software requirements modeling method based on a large artificial intelligence model to solve the problems of disconnect between the generated model and the business rules, as well as the difficulty in locating the root cause of errors and automatically correcting them.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] This invention provides a software requirement modeling method based on a large artificial intelligence model, which includes receiving multimodal software requirement materials provided by users, and using a large multimodal model to parse them to obtain the initial semantic elements corresponding to each modality.

[0008] Perform two-layer semantic alignment reconstruction on the initial semantic elements of different modalities to obtain a fused semantic element set;

[0009] The set of fused semantic elements is converted into nodes and edges. During the conversion process, uncertainty markers are added to edges with insufficient association basis to construct a demand semantic knowledge graph.

[0010] A demand model generation scheme is constructed based on a demand semantic knowledge graph.

[0011] The requirement model generation scheme is filled into the preset structured prompt word template to obtain structured prompts, and the structured prompts are used to drive the multimodal large model to generate a requirement model draft;

[0012] After performing a rule compliance check on the draft requirement model, an iterative verification and correction loop is executed based on the check results to obtain the target requirement model.

[0013] As a preferred embodiment of the software requirement modeling method based on a large artificial intelligence model described in this invention, the multimodal software requirement materials include natural language text and software requirement visualization images.

[0014] As a preferred embodiment of the software requirements modeling method based on a large artificial intelligence model described in this invention, the step of obtaining the initial semantic elements corresponding to each modality specifically involves:

[0015] Natural language text and software requirement visualization images are input into a multimodal large model. The multimodal large model performs entity recognition, relation extraction and semantic role labeling on the natural language text to obtain the initial semantic elements of the text.

[0016] The multimodal large model performs visual element recognition on the software requirement visualization image, extracts the user interface elements, the relationships between elements, and the structured text of the related business function descriptions, and obtains the initial semantic elements of the image.

[0017] As a preferred embodiment of the software requirements modeling method based on a large artificial intelligence model described in this invention, wherein: obtaining the fused semantic element set specifically involves:

[0018] The initial semantic elements of the text and the initial semantic elements of the image are vectorized and encoded to obtain text semantic vectors and image semantic vectors;

[0019] Calculate the semantic similarity between text semantic vectors and image semantic vectors, and construct cross-modal candidate association pairs by combining semantic element type constraints;

[0020] For each cross-modal candidate association pair, the consistency of attributes, behavior, and context dependency is used to determine the cross-modal candidate association pair, and the cross-modal candidate association pair is divided into consistent semantic pairs and conflicting semantic pairs.

[0021] Consistent semantic pairs are merged and reconstructed to generate core fused semantic elements;

[0022] For conflicting semantic pairs, retain the conflict marker, conflict source, and corresponding semantic content, and generate semantic elements to be fused.

[0023] The core fusion semantic elements, the fusion semantic elements to be determined, and the initial semantic elements that have not formed cross-modal candidate association pairs are combined to form a fusion semantic element set.

[0024] As a preferred embodiment of the software requirement modeling method based on a large artificial intelligence model described in this invention, the construction of the requirement semantic knowledge graph specifically includes:

[0025] Extract each fused semantic element from the fused semantic element set;

[0026] Based on the semantic category of the fused semantic elements, the fused semantic elements are mapped to corresponding type nodes in the knowledge graph, and attributes are assigned to the nodes;

[0027] Analyze the semantic relationships between different fused semantic elements, and establish a corresponding edge between the two corresponding nodes based on the relationship type;

[0028] During the edge establishment process, association determination is performed, and an uncertainty marker is added to edges with insufficient association basis;

[0029] The connections between nodes and edges are summarized to form a semantic knowledge graph of requirements.

[0030] As a preferred embodiment of the software requirement modeling method based on a large artificial intelligence model described in this invention, the requirement model generation scheme is constructed by reading nodes and edges in the requirement semantic knowledge graph and constructing the model based on the semantic category of the nodes and the association type of the edges.

[0031] As a preferred embodiment of the software requirement modeling method based on artificial intelligence large model described in this invention, the acquisition of structured prompts refers to extracting the model type, node semantics, relationship organization, and unconfirmed relationships from the requirement model generation scheme, and filling them into the corresponding fields in the structured prompt word template to obtain structured prompts.

[0032] As a preferred embodiment of the software requirements modeling method based on a large artificial intelligence model described in this invention, the step of using structured prompts to drive the generation of a draft requirements model from a multimodal large model specifically involves:

[0033] The structured prompts are submitted as input instructions to the multimodal large model;

[0034] The multimodal large model performs forward inference computation based on the modeling task, modeling elements and modeling logic required in the structured prompts;

[0035] Based on the pre-defined modeling language, the forward inference calculation results are organized into a draft requirement model.

[0036] As a preferred embodiment of the software requirements modeling method based on a large artificial intelligence model described in this invention, the step of performing a rule compliance check on the draft requirements model specifically includes:

[0037] The requirement model draft is input into the formal rule checker, which performs syntax parsing on the requirement model draft and converts it into an internal abstract syntax tree representation.

[0038] It iterates through each node of the internal abstract syntax tree, matching and logically judging the node's type, attributes, and relationships with preset rules one by one, recording all instances of rule violations, and generating an inspection result report.

[0039] As a preferred embodiment of the software requirement modeling method based on a large artificial intelligence model described in this invention, the step of performing an iterative verification and correction loop based on the inspection results to obtain the target requirement model specifically includes:

[0040] Determine if the inspection result report is empty. If the inspection result report is empty, then use the current requirement model draft as the target requirement model.

[0041] If the inspection result report is not empty, then based on the violation description in the inspection result report, locate the violation fragment in the requirement model draft and map it to the requirement semantic knowledge graph to obtain a local subgraph;

[0042] Based on the violation descriptions and local sub-graphs in the inspection results report, generate correction instructions;

[0043] Extract the prompt content corresponding to the violation fragment from the structured prompts, combine it with the correction instructions to form a corrected prompt, and use the corrected prompt to drive the multimodal large model to perform local regeneration to obtain the corrected violation fragment;

[0044] The corrected violation fragments were then added back to the draft requirements model to form a new draft requirements model.

[0045] The new draft demand model will undergo another rule compliance check until the check result report is empty.

[0046] The beneficial effects of this invention are as follows: By using a two-layer semantic alignment and reconstruction process, the requirement information in natural language text and software requirement visualization images is uniformly aligned and fused, improving the semantic consistency of multimodal software requirement materials, reducing the impact of fragmented text and image information on requirement understanding, converting the fused semantic element set into a requirement semantic knowledge graph, and adding uncertainty markers to edges with insufficient association basis, so that candidate associations in the requirements can be distinguished from explicit associations, thereby improving the clarity and completeness of the requirement semantic organization process. Furthermore, a formal rule checker is used to check the rule compliance of the requirement model draft, and iterative verification and correction loops are executed in combination with the check result report, so that the non-compliant fragments in the requirement model draft can be located, corrected and backfilled, thereby improving the accuracy, completeness and usability of the target requirement model. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart of a software requirements modeling method based on a large-scale artificial intelligence model.

[0049] Figure 2 A flowchart for generating a draft requirements model.

[0050] Figure 3 A flowchart for obtaining initial semantic elements of text and initial semantic elements of images.

[0051] Figure 4 A flowchart for generating an inspection result report. Detailed Implementation

[0052] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0053] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0054] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0055] Reference Figures 1-4 This is one embodiment of the present invention, which provides a software requirements modeling method based on a large artificial intelligence model, including the following steps:

[0056] S1. Receive the multimodal software requirement materials provided by the user, and use the multimodal large model to parse them to obtain the initial semantic elements corresponding to each modality.

[0057] S1.1 It should be noted that the system receives multimodal software requirement materials submitted by users through file upload interfaces and graphical interfaces, including software requirement visualizations and natural language text. Natural language text includes requirement specification documents, user story descriptions, business rule descriptions, and process description texts; software requirement visualizations include interface prototypes, page sketches, system screenshots, and process interface diagrams.

[0058] S1.2. It should be noted that historical multimodal software requirement materials should be collected, including historical natural language text and historical software requirement visualization images. The natural language text should be annotated, including user roles, functional items, business objects, interactive actions, constraints, and process sequences, forming text supervision labels. The software requirement visualization images should be annotated, including interface control categories, page area categories, jump relationships, input / output items, and function entry points, forming image supervision labels. Simultaneously, template filling records, requirement generation instruction records, or historical human-computer interaction prompt logs saved from historical software projects, along with corresponding historical requirement model text, should be collected. These records should be organized into historical requirement prompt samples. Historical natural language text, historical software requirement visualization images, text supervision labels, image supervision labels, historical requirement prompt samples, and historical requirement model text should be paired to form a joint training set.

[0059] S1.3 It should be noted that the multimodal large model adopts a dual-branch coding and cross-modal alignment fusion architecture, including a text input embedding layer, a text coding layer, an image input embedding layer, a visual coding layer, a cross-modal alignment layer, a semantic parsing output layer, and a structured generation output layer.

[0060] The word embedding dimension of the text input embedding layer is set to 768 dimensions, and the position encoding dimension is set to 768 dimensions.

[0061] The text encoding layer is set to consist of 12 layers of Transformer encoding structure stacked sequentially. Each Transformer encoding structure includes a multi-head self-attention computation structure, a residual connection structure, a layer normalization structure, and a feedforward neural network. The number of attention heads in the multi-head self-attention computation structure is set to 12, the hidden dimension is set to 768, and the intermediate layer dimension of the feedforward neural network is set to 3072.

[0062] The image input embedding layer is used to segment the software requirement visualization image into multiple image blocks. The image block size is set to 16×16 pixels, the image block embedding dimension is set to 768 dimensions, and the two-dimensional position encoding dimension is set to 768 dimensions.

[0063] The visual coding layer is set to consist of 12 layers of Transformer coding structure stacked sequentially; each Transformer coding structure includes a multi-head self-attention computation structure, a residual connection structure, a layer normalization structure, and a feedforward neural network; the number of attention heads in the multi-head self-attention computation structure is set to 12, the hidden dimension is set to 768, and the intermediate layer dimension of the feedforward neural network is set to 3072.

[0064] Two cross-modal alignment layers are set and inserted between the 10th and 11th layers of the text encoding layer and between the 10th and 11th layers of the visual encoding layer. Each cross-modal alignment layer adopts a bidirectional cross-attention structure, including a text-to-image cross-attention sub-layer and an image-to-text cross-attention sub-layer, and includes a residual connection structure, a layer normalization structure, and a feedforward neural network. The number of attention heads in the cross-attention is set to 12, the hidden dimension is set to 768, and the intermediate layer dimension of the feedforward neural network is set to 3072.

[0065] The semantic parsing output layer includes a text semantic element output layer and an image semantic element output layer; the output head dimension of the text semantic element output layer is set to 11 dimensions; the image semantic element output layer includes a category recognition output head and a relationship recognition output head, with the category recognition output head dimension set to 8 dimensions and the relationship recognition output head dimension set to 6 dimensions.

[0066] The structured output layer is composed of six stacked Transformer decoding structures. Each Transformer decoding structure includes a masked multi-head self-attention computation structure, a cross-attention computation structure, a residual connection structure, a layer normalization structure, and a feedforward neural network. The masked multi-head self-attention computation structure and the cross-attention computation structure each have 12 attention heads and 768 hidden dimensions. The intermediate layer of the feedforward neural network has 3072 dimensions, and the decoding method is an autoregressive decoding method.

[0067] For the joint training set, the historical natural language text and historical software requirement visualization images from each batch are input into the corresponding encoding paths to obtain text prediction results and image prediction results, respectively. For the historical requirement prompt samples from each batch, the historical requirement prompt samples are input into the text input embedding layer, text encoding layer, and structured generation output layer to obtain requirement prediction results. The text prediction results are compared with the text supervision labels, and the text loss value is calculated using the cross-entropy loss function. The image prediction results are compared with the image supervision labels, and the image loss value is calculated using the smoothing L1 loss function. The requirement prediction results are compared with the corresponding historical requirement model text position by position to calculate the sequence prediction loss value. The text loss value, image loss value, and sequence prediction loss value are weighted and summed to obtain the total loss value. Based on the total loss value, the backpropagation algorithm is used to calculate the gradient of the learnable parameters in the multimodal large model. According to the gradient, the Adam optimizer is used to update the learnable parameters of the multimodal large model. The forward propagation calculation, loss calculation, and parameter update process are repeated until the maximum number of iterations is reached to obtain the trained multimodal large model.

[0068] Furthermore, the training process corresponding to historical natural language text and historical software requirement visualization images is used to improve the multimodal large model's ability to identify and fuse requirement semantic elements. The training process corresponding to historical requirement prompt samples and historical requirement model text is used to improve the multimodal large model's ability to generate requirement model drafts based on structured prompts, thereby ensuring that the condition generation path in the training phase is consistent with the path generated by subsequent inference based on structured prompts.

[0069] Text loss and image loss are used to constrain the model's ability to represent natural language text features and image features, respectively. They are given the same weight, which helps to maintain the balance of the multimodal feature learning process and avoid the model from being overly biased towards a single modality during training. For example, the weight is set to 0.3. The sequence prediction loss directly corresponds to the accuracy of subsequent entity boundary recognition, semantic category determination, and sequence output results. It has a more direct impact on the initial semantic element extraction of text and the subsequent demand modeling results. Therefore, it has the highest weight, such as 0.4, to balance the multimodal semantic parsing ability and the demand model draft generation ability.

[0070] In one embodiment, the maximum number of iterations can be set to 15 rounds, which can fully optimize the text loss value, image loss value and sequence prediction loss value, thereby avoiding output fluctuations caused by too few training rounds.

[0071] The weights for text loss, image loss, sequence prediction loss, and the maximum number of iterations are not fixed values, but can be adjusted according to the size of the training samples, the data quality of different modalities, the required model complexity, and the model convergence.

[0072] S1.4 It should be noted that word segmentation is performed sequentially on the natural language text to obtain a sequence of word segmentation units arranged in the processing order. Size normalization and image block division are performed on the software requirement visualization image to obtain a sequence of image blocks arranged in spatial position. The word segmentation unit sequence is input into the trained multimodal large model. The text input embedding layer of the multimodal large model maps each word segmentation unit to a word embedding vector through the word embedding matrix, and generates a position embedding vector for each word segmentation position through the position embedding matrix. Then, the word embedding vector and the corresponding position embedding vector are summed element by element to form a text input vector sequence. The image patch sequence is input into the trained multimodal large model. The image input embedding layer of the multimodal large model maps each image patch to a visual embedding vector through a linear projection weight matrix, and generates a position embedding vector for each image patch position through a position embedding matrix. The visual embedding vector is summed element-wise with the corresponding position embedding vector to form the image input vector sequence. The first text encoding layer of the multimodal large model performs multi-head self-attention calculation on the text input vector sequence. The calculation process is as follows: the text input vector sequence is multiplied by the query projection weight matrix, the key projection weight matrix, and the value projection weight matrix respectively to obtain the query matrix, the key matrix, and the value matrix. The query... After performing a dot product with the transpose of the key matrix, a quotient operation is performed with the scaling factor. The Softmax function is applied to the quotient result to obtain the text attention weights. The text attention weights are then weighted and summed with the value matrix to obtain the text attention output vector sequence. The text attention output vector sequence is then residually concatenated with the text input vector sequence and layer normalization is performed to obtain the normalized text vector sequence. The normalized text vector sequence is then input into a feedforward neural network for nonlinear mapping to obtain the feedforward text output vector sequence. The feedforward text output vector sequence is then residually concatenated with the normalized text vector sequence and layer normalization is performed again to obtain the output vector sequence of the first text encoding layer.

[0073] The first visual coding layer of the multimodal large model performs multi-head self-attention computation, residual connection, layer normalization, and feedforward neural network operations on the image input vector sequence, which are the same as those in the text coding layer, to obtain the output vector sequence of the first visual coding layer. The output vector sequence of the first text coding layer is used as the input of the next text coding layer, and the output vector sequence of the first visual coding layer is used as the input of the next visual coding layer. After processing through all text coding layers and all visual coding layers in sequence, the final text output vector sequence and the final image output vector sequence are obtained.

[0074] The final text output vector sequence and the final image output vector sequence are input into the cross-modal alignment layer. The cross-modal alignment layer first performs text-to-image cross-attention calculation, using the final text output vector sequence as the query matrix and the final image output vector sequence as the key and value matrices, calculating the cross-attention weights of text positions to image patch positions, resulting in an image-aware text vector sequence. Simultaneously, it performs image-to-text cross-attention calculation, using the final image output vector sequence as the query matrix and the final text output vector sequence as the key and value matrices, calculating the cross-attention weights of image patch positions to text positions, resulting in a text-aware image vector sequence. The image-aware text vector sequence is concatenated and linearly mapped with the final text output vector sequence to form a text fusion vector sequence. The text-aware image vector sequence is then concatenated and linearly mapped with the final image output vector sequence to form an image fusion vector sequence. Finally, the text semantic element output layer of the multimodal large model performs text initialization on the text fusion vector sequence. The semantic element extraction calculation specifically involves the fully connected linear layer mapping each position vector in the text fusion vector sequence to a semantic category logical score vector. Semantic categories include user roles, functional items, business objects, interactive actions, constraints, and process order. Then, the optimal label sequence corresponding to each position is determined through a sequence labeling decoding process. Based on the label combination results of consecutive positions in the optimal label sequence, the text segment boundaries and semantic categories corresponding to user roles, functional items, business objects, interactive actions, constraints, and process order are determined. Entities obtained through the sequence labeling decoding process in the text fusion vector sequence are paired to form entity position pairs. The semantic categories, text segment boundaries, text sequence relationships, and contextual content between the two entities in each entity position pair are read. Based on the semantic categories, text sequence relationships, and contextual content, the relationship type between the entities is determined. The text segment boundaries, semantic categories, and relationship types of the entities are then associated and organized to form the initial semantic elements of the text.

[0075] The software requirements visualization image is used to call an OCR tool to detect and recognize text regions in the image, outputting the text content and position coordinates of each text region to obtain the text recognition results. The multimodal large model's image semantic element output layer performs initial semantic element extraction calculations on the image fusion vector sequence. Specifically, the image semantic element output layer maps the image fusion vector sequence to candidate bounding box coordinates and interface control category scores, and after filtering, obtains the interface control positions and categories; the image semantic element output layer maps the image fusion vector sequence to page region category distribution to obtain page regions; the image semantic element output layer calculates the relationships between elements based on the interface control positions, page region positions, and cross-modal alignment results to obtain jump relationships and input / output item relationships; the image semantic element output layer generates semantic labels corresponding to function entry points based on the image fusion vector sequence and the text recognition results in the software requirements visualization image, forming the initial semantic elements of the image.

[0076] Furthermore, the cross-modal alignment layer is used to establish semantic relationships between text features and image features in a continuous semantic vector space; the two-layer semantic alignment reconstruction is used to perform consistency determination, conflict identification and fusion reconstruction of the initial semantic elements of the text and the initial semantic elements of the image at the discrete semantic element level.

[0077] S2. Perform two-layer semantic alignment reconstruction on the initial semantic elements of different modalities to obtain a fused semantic element set.

[0078] S2.1 It should be noted that the text fusion vector sequence and the image fusion vector sequence are read from the cross-modal alignment layer of the multimodal large model respectively; based on the text fusion vector sequence, the text segment boundaries of each initial semantic element of the text in the natural language text are read, and the text segment boundaries include the start position and the end position; according to the start position and the end position corresponding to each initial semantic element of the text, the continuous position vectors of the corresponding intervals are extracted from the text fusion vector sequence to form a text fusion vector subsequence that corresponds one-to-one with each initial semantic element of the text.

[0079] Based on the image fusion vector sequence, the position boundaries of the initial semantic elements of each image in the software requirement visualization image are read. The position boundaries include the position of the interface control bounding box, the position of the page area, or the position of the function entry area. According to the position boundaries corresponding to the initial semantic elements of each image, the set of image block vectors covering the corresponding position range is extracted from the image fusion vector sequence and arranged into an image fusion vector subsequence.

[0080] For each text fusion vector subsequence, mean pooling is performed to obtain the corresponding text semantic representation. For each image fusion vector subsequence, mean pooling is performed to obtain the corresponding image semantic representation. Fully connected linear transformation layers are used to map each text semantic representation and each image semantic representation to the same dimension, resulting in multiple sets of text semantic vectors and image semantic vectors. Each text semantic vector corresponds to an initial text semantic element, and each image semantic vector corresponds to an initial image semantic element. The cross-modal correspondence between the text side and the image side is not directly determined in the subsequence segmentation stage, but is established later when constructing cross-modal candidate association pairs by combining semantic similarity and semantic element type constraints.

[0081] S2.2 It should be noted that after pairing all text semantic vectors and image semantic vectors, cosine similarity is calculated to obtain semantic similarity. The formula is as follows:

[0082] ;

[0083] in, Indicates the first The semantic vector of the first text and the first Semantic similarity between image semantic vectors Indicates the first A text semantic vector, Indicates the first Image semantic vectors;

[0084] The process involves filtering based on semantic element type constraints and semantic similarity. Semantic element type constraints limit the combinations of semantic categories that can be associated. Specifically, functional items are associated only with functional entry points, business objects are associated only with input / output items, interactive actions are associated only with the relationship type between elements, and constraints are associated only with page area categories or interface control categories. User roles and process order are not included in the image-side candidate association construction. For text semantic vectors and image semantic vectors that meet the semantic element type constraints and have a semantic similarity higher than the similarity threshold, the corresponding initial text semantic element identifier, initial image semantic element identifier, and semantic similarity are recorded. The recorded results are used as a cross-modal candidate association pair. The similarity calculation, type constraint filtering, and threshold determination are repeated for all text semantic vectors and image semantic vectors to obtain a set of cross-modal candidate association pairs.

[0085] Furthermore, the similarity threshold is set to determine whether the matching degree between the text semantic vector and the image semantic vector meets the requirements for candidate association construction. The similarity of all text semantic vectors and image semantic vectors is summed, and the mean of the summation is calculated. This mean is used as the similarity threshold.

[0086] S2.3. It should be noted that, for each cross-modal candidate association pair, the corresponding initial text semantic elements and initial image semantic elements are read respectively; the semantic category, relationship type, and semantic role in the initial text semantic elements, and the interface control category, page area category, element relationship type, function entry point, and input / output item in the initial image semantic elements are extracted. An attribute consistency determination is performed, checking the correspondence between the semantic category of the initial text semantic elements and the category attributes of the initial image semantic elements. Specifically, when the semantic category of the initial text semantic elements satisfies the preset category mapping relationship with the function entry point, input / output item, page area category, interface control category, or element relationship type of the initial image semantic elements, the attribute consistency result is recorded as consistent; otherwise, it is recorded as inconsistent. The category mapping relationship includes: the correspondence between function items and function entry points, the correspondence between business objects and input / output items, the correspondence between constraints and page area categories or interface control categories, and the correspondence between interactive actions and element relationship types.

[0087] For cross-modal candidate association pairs formed by the interaction actions in the initial semantic elements of text and the relationship types between elements in the initial semantic elements of images, a behavior consistency determination is performed. Specifically, the action semantics corresponding to the interaction action and the relationship semantics corresponding to the relationship types between elements are read. When the action semantics and relationship semantics are consistent in the meaning of interactive behaviors such as triggering, jumping, submitting, selecting, entering, displaying, or switching, the behavior consistency determination result is recorded as consistent; otherwise, the behavior consistency determination result is recorded as inconsistent. For cross-modal candidate association pairs formed by function items and function entry points, cross-modal candidate association pairs formed by business objects and input / output items, and cross-modal candidate association pairs formed by constraints and page area categories or interface control categories, no behavior consistency determination is performed, and the behavior consistency determination result is recorded as inapplicable. The system reads the sentence and paragraph numbers of the initial semantic elements in the natural language text and combines this with the positional relationships of the initial semantic elements in the image within the page area to determine the contextual dependency consistency of object dependencies and process sequence relationships in the semantic context. Specifically, when the object dependencies in the text semantic context are consistent with the element hierarchy relationships in the image semantic structure, and the process sequence relationships in the text semantic context are consistent with the page switching order or interaction triggering order in the image semantic structure, the contextual dependency consistency result is recorded as consistent; otherwise, it is recorded as inconsistent. When cross-modal candidate association pairs belong to candidate associations formed by the correspondence between interaction actions and element relationship types... When the attribute consistency judgment result, behavior consistency judgment result, and context dependency consistency judgment result are all consistent, the corresponding cross-modal candidate association pair is marked as a consistent semantic pair. When the cross-modal candidate association pair belongs to the candidate association pair formed by the correspondence between function item and function entry, the candidate association pair formed by the correspondence between business object and input / output item, or the candidate association pair formed by the correspondence between constraint condition and page area category or interface control category, and the attribute consistency judgment result and context dependency consistency judgment result are all consistent, the corresponding cross-modal candidate association pair is marked as a consistent semantic pair. In other cases, the corresponding cross-modal candidate association pair is marked as a conflict semantic pair, and a conflict mark is generated.

[0088] S2.4. It should be noted that the initial text semantic elements and initial image semantic elements corresponding to the consistent semantic pairs should be read separately, and the corresponding semantic content should be aligned and organized. For consistent semantic pairs formed by the correspondence between interactive actions and element relationship types, when the attribute consistency judgment result, behavior consistency judgment result, and context dependency consistency judgment result are all consistent, the corresponding initial text semantic elements and initial image semantic elements should be merged at the field level. For consistent semantic pairs formed by the correspondence between function items and function entry points, consistent semantic pairs formed by the correspondence between business objects and input / output items, and consistent semantic pairs formed by the correspondence between constraints and page area categories or interface control categories, when the attribute consistency judgment result and context dependency consistency judgment result are all consistent, the corresponding initial text semantic elements and initial image semantic elements should be merged at the field level. Using the main semantic content in the initial semantic elements of the text as the basis for semantic description, and the interface structure attributes and spatial location information in the initial semantic elements of the image as supplementary interface expression, a unified semantic category field, a unified semantic role field, and a unified location description field are formed. For semantically equivalent fields in the initial semantic elements of the text and the initial semantic elements of the image, only one set of merged normalized field values ​​is retained. For complementary fields in the initial semantic elements of the text and the initial semantic elements of the image that respectively represent semantic content and interface performance, they are retained and written into the same semantic record to obtain the core fused semantic elements.

[0089] For conflicting semantic pairs, the corresponding initial text semantic elements and initial image semantic elements are read separately. The semantic category, relationship type, semantic role, sentence number, and paragraph number from the initial text semantic elements, and the interface control category, page area category, element relationship type, positional relationship, input / output items, and function entry point from the initial image semantic elements are retained as corresponding semantic content. Simultaneously, conflict markers are generated based on the consistency judgment results applicable to the corresponding cross-modal candidate association pair type. Specifically, when the cross-modal candidate association pair corresponding to the conflicting semantic pair belongs to a candidate association pair formed by the interaction action and the element relationship type, the conflict marker is used to represent attribute conflict, behavior conflict, context dependency conflict, or composite conflict; when the conflicting semantic pair corresponds to... When cross-modal candidate association pairs belong to candidate association pairs formed by the correspondence between functional items and functional entry points, candidate association pairs formed by the correspondence between business objects and input / output items, or candidate association pairs formed by the correspondence between constraints and page area categories or interface control categories, conflict markers are used to characterize attribute conflicts, context dependency conflicts, or compound conflicts. The source information of the initial semantic elements of the text and the initial semantic elements of the image are recorded as conflict sources. Conflict sources include the start position, end position, sentence number, and paragraph number of the initial semantic elements of the text in the natural language text, and the bounding box position and page area position of the initial semantic elements of the image in the software requirement visualization image. Conflict markers, conflict sources, and corresponding semantic content are combined according to field order to form the fusion semantic elements to be determined. The core fusion semantic elements, the fusion semantic elements to be determined, and the initial semantic elements that have not formed cross-modal candidate association pairs together constitute the fusion semantic element set.

[0090] S3. Convert the fusion semantic element set into nodes and edges. During the conversion process, add uncertainty markers to edges with insufficient association basis to construct a demand semantic knowledge graph.

[0091] S3.1 It should be noted that, based on each fusion semantic element in the fusion semantic element set, the semantic category, relationship type, semantic role, location description, input / output items, function entry, source information, and conflict marker in the fusion semantic element are read; fusion semantic elements with the semantic category of user role are mapped to user role nodes, fusion semantic elements with the semantic category of function item are mapped to function item nodes, fusion semantic elements with the semantic category of business object are mapped to business object nodes, fusion semantic elements with the semantic category of interaction action are mapped to interaction action nodes, fusion semantic elements with the semantic category of constraint condition are mapped to constraint condition nodes, and fusion semantic elements with the semantic category of process sequence are mapped to process sequence nodes.

[0092] For the image-side interface semantic content retained in the fused semantic elements, if it includes interface control categories and bounding box positions, it is mapped to interface control nodes; if it includes page area categories and page area positions, it is mapped to page area nodes; if it includes function entry semantic tags, it is mapped to function entry nodes; if it includes input / output item names or input / output field identifiers, it is mapped to input / output item nodes.

[0093] After completing the node type mapping, node attributes are written for the corresponding type of node according to the field correspondence rules. Specifically, the semantic main content in the fused semantic elements is written into the node name attribute, the semantic role is written into the role attribute, the location description content is written into the location attribute, the interactive semantic content is written into the interaction attribute, the function entry content is written into the function entry attribute, the input and output item content is written into the input and output attribute, the source information is written into the source attribute, and the conflict marker is written into the conflict state attribute, thereby obtaining a knowledge graph node that corresponds one-to-one with each fused semantic element.

[0094] S3.2 It should be noted that, based on various nodes in the demand semantic knowledge graph, the semantic associations, interface associations, and process associations between different integrated semantic elements are analyzed. The semantic roles, relationship types, semantic categories, interface control categories, page area categories, element relationship types, function entry points, and input / output items related to the corresponding nodes are read. Combined with the attribute consistency judgment results, behavior consistency judgment results, and context dependency consistency judgment results, the association types between the corresponding nodes are matched and judged according to the semantic associations, interface associations, and process associations. After the association type judgment is completed, corresponding type edges are established between the corresponding nodes according to the determined association types. Specifically, when the semantic role corresponding to the user role node matches the relationship type corresponding to the function item node, and the context dependency consistency judgment result indicates that the user role participates in the function item, the association type between the user role node and the function item node is determined to be a role participation relationship, and a role execution edge is established between the user role node and the function item node; when the relationship type corresponding to the function item node matches the semantic category corresponding to the business object node, and the attribute consistency judgment result and the context dependency consistency judgment result indicate that the function item performs processing on the business object, the association type between the function item node and the business object node is determined to be a business processing relationship, and a function processing edge is established between the function item node and the business object node; when the relationship type corresponding to the function item node matches the semantic category corresponding to the interaction action node, and the behavior consistency judgment result indicates that the interaction action is triggered by the function item, the association type between the function item node and the interaction action node is determined to be an operation triggering relationship, and an interaction triggering edge is established between the function item node and the interaction action node.

[0095] When the relationship type corresponding to an interactive action node matches the category of a UI control, function entry point, or input / output item, and the behavior consistency judgment result indicates that the interactive action acts on the UI control, function entry point, or input / output item, the association type between the interactive action node and the UI control node, function entry point node, or input / output item node is determined to be a UI interaction relationship, and a UI interaction edge is established between the interactive action node and the corresponding UI control node, function entry point node, or input / output item node. When the UI control category corresponding to a UI control node matches the page area category corresponding to a page area node, or the function entry point or input / output item corresponding to a function entry point node or input / output item node matches the UI control node or page area node, and the attribute consistency judgment result indicates a subordinate relationship, the association type between the corresponding nodes is determined to be a UI affiliation relationship, and a UI affiliation edge is established between the corresponding nodes. When the relationship type or element relationship type corresponding to a function entry point node, UI control node, or page area node matches, and the behavior consistency judgment result and context dependency consistency judgment result indicate page switching, page migration, or step jump, the association type between the corresponding nodes is determined to be a page switching relationship, page migration relationship, or step jump relationship, and a page jump edge is established between the corresponding nodes.

[0096] When the semantic category corresponding to a business object node matches the input / output item corresponding to an input / output item node, and the attribute consistency determination result represents the field carrying relationship, input relationship, display relationship, or echo relationship, the association type between the business object node and the input / output item node is determined to be an input / output association relationship, and an input / output association edge is established between the business object node and the input / output item node. When a process sequence node represents the sequential relationship between multiple function item nodes, interactive action nodes, function entry nodes, or page area nodes, the association type between the process sequence node and the corresponding node is determined to be a process sequence relationship, and a process sequence edge is established between the process sequence node and the corresponding node. When the pointing relationship between a constraint condition node and a function item node, business object node, interactive action node, interface control node, function entry node, input / output item node, or page area node represents a constraint action relationship, the association type between the constraint condition node and the corresponding node is determined to be a constraint dependency relationship, and a constraint dependency edge is established between the constraint condition node and the corresponding node.

[0097] Furthermore, during the edge establishment process, the two nodes corresponding to the edge are read, and the semantic role, position description, input / output items, function entry, source information, and conflict marker of the nodes in the fused semantic elements are extracted. When both nodes originate from the core fused semantic elements and there are no conflict markers in the corresponding fused semantic elements, it indicates that the association basis between the corresponding nodes is clear, and no uncertainty marker is added to the established edge. When at least one of the two nodes originates from the fused semantic elements to be determined, or there are conflict markers in the corresponding fused semantic elements, it indicates that although the corresponding nodes can establish edges based on semantic association, interface association, or process association, the association basis of the edge still contains conflict information that needs further confirmation. Therefore, the association basis of the edge is insufficient, and an uncertainty marker is added.

[0098] By repeatedly performing semantic association analysis, edge type determination, and uncertainty tagging on all nodes, role execution edges, function processing edges, interaction triggering edges, interface interaction edges, interface affiliation edges, page jump edges, input / output association edges, process sequence edges, and constraint dependency edges are formed; the connection relationships between various types of nodes and various types of edges together constitute the requirement semantic knowledge graph.

[0099] S4. Based on the semantic knowledge graph of requirements, construct a requirement model generation scheme.

[0100] It should be noted that all nodes, edges, and corresponding uncertainty markers in the requirement semantic knowledge graph are read; when a user role node is connected to a function item node through a role execution edge, and a function item node is connected to a business object node through a function processing edge, the user role node is treated as a use case participant, the function item node as a use case behavior, and the business object node as a use case processing object; using the participation relationship represented by the role execution edge and the business processing relationship represented by the function processing edge, the relationships between use case participants, use case behaviors, and business objects are organized to generate a use case model.

[0101] When a functional item node is connected to an interactive action node via an interactive trigger edge, and the interactive action node is connected to a UI control node, functional entry node, or input / output item node via a UI interaction edge, or to a page area node via a page jump edge, the functional item node and the interactive action node are treated as interactive step nodes, and the UI control node, page area node, functional entry node, and input / output item node are treated as UI interactive elements. Using the interactive logic relationships represented by interactive trigger edges, UI interaction edges, page jump edges, and UI belonging edges, the interaction sequence, triggering relationship, and page migration relationship between interactive step nodes and UI interactive elements are organized to generate an interactive flow model.

[0102] When a process sequence node connects multiple function item nodes, interactive action nodes, function entry nodes, or page area nodes through process sequence edges, the corresponding nodes are sorted according to the order represented by the process sequence edges, and a process model is generated based on the sorting results.

[0103] When a constraint node is connected to a function item node, business object node, interactive action node, interface control node, function entry node, input / output item node, or page area node through a constraint dependency edge, the constraint node is used as the constraint description content, the connected node is used as the constraint object, and a constraint model is generated based on the constraint relationship represented by the constraint dependency edge.

[0104] Use case models, interaction flow models, process models, and constraint models together constitute the requirements model generation scheme.

[0105] Furthermore, when the edges used to organize model relationships have uncertainty markers, the corresponding relationship expressions are preserved, and the corresponding relationships are marked as relationships to be confirmed.

[0106] S5. Fill the generated solution from the demand model into the preset structured prompt template to obtain structured prompts.

[0107] It should be noted that the use case model, interaction flow model, process model, and constraint model should all use the same structured prompt template with the same field framework. The structured prompt template employs a field-based organization method, including model type fields, node semantic fields, relationship organization fields, and fields for confirmation of relationships.

[0108] For use case models, interaction flow models, process models, and constraint models, extract the corresponding model type fields, node semantic fields, relationship organization fields, and unconfirmed relationship fields respectively; when the model type is a use case model, fill the corresponding fields in the structured prompt word template with use case actors, use case behaviors, processing objects, participation relationships, and business processing relationships.

[0109] When the model type is an interaction flow model, read the interaction step nodes, interface interaction elements, and the triggering relationship represented by the interaction triggering edge between nodes, the page migration relationship represented by the page jump edge, the interface operation relationship represented by the interface interaction edge, and the interface hierarchy relationship represented by the interface belonging edge in the interaction flow model; fill the interaction step nodes, interface interaction elements, and corresponding relationships into the corresponding fields in the structured prompt word template to represent the operation sequence, page migration logic, interface element role relationship, and interface hierarchy structure in the interaction flow model.

[0110] When the model type is a process model, read the process sequence nodes, process step nodes, and process sequence relationships in the process model; among them, process step nodes include functional item nodes, interactive action nodes, functional entry nodes, or page area nodes connected by process sequence edges; fill the corresponding fields in the structured prompt word template with the process sequence nodes, process step nodes, and process sequence relationships to represent the step organization method and execution sequence relationship in the process model.

[0111] When the model type is a constraint model, the constraint condition nodes, constraint objects, and constraint dependencies in the constraint model are read. Constraint objects include functional item nodes, business object nodes, interactive action nodes, UI control nodes, function entry nodes, input / output item nodes, or page area nodes connected by constraint dependency edges. The constraint condition nodes, constraint objects, and constraint dependencies are then populated into the corresponding fields in the structured prompt template to represent the constraints, constraint objects, and constraint application methods in the constraint model. When a relationship needs to be confirmed, it is simultaneously written into the "Confirmed Relationship" field in the corresponding structured prompt template. After filling in all fields, the model type, node semantics, relationship organization, and pending confirmation relationships are arranged and separated according to the field order in the structured prompt template, with field titles corresponding to field content, to obtain the structured prompt.

[0112] S6. Utilize structured prompts to drive the generation of a draft requirement model from a multimodal large model.

[0113] It should be noted that the structured prompts are submitted as input instructions to the trained multimodal large model; the text input embedding layer of the multimodal large model encodes the structured prompts into a prompt input vector sequence; the prompt input vector sequence is input into the multi-layer Transformer encoding structure of the multimodal large model, which performs multi-head self-attention calculation, residual connections, layer normalization, and feedforward neural network operations on the prompt input vector sequence, resulting in a prompt semantic representation vector sequence after processing by the entire Transformer encoding structure; the prompt semantic representation vector sequence is input into the structured generation output layer, which reads the model type field, node semantic field, relation organization field, and... The system generates a requirement model draft by generating a sequence of conditions according to the syntax rules of the preset modeling languages ​​(PlantUML and Mermaid) based on the node organization rules and relation expression rules corresponding to the current model type. This process progressively outputs the node definitions, relation definitions, sequence definitions, constraint definitions, and pending confirmation marker definitions in the requirement model draft, resulting in a requirement model draft that conforms to the syntax requirements of the modeling language. It should also be noted that when the structured prompts contain pending confirmation relation fields, the structured generation output layer outputs pending confirmation markers, candidate relation markers, or uncertainty markers at the corresponding relation positions to retain the uncertainty relation information passed from previous steps and to ensure that the requirement model draft is consistent with the pending confirmation relations in previous steps.

[0114] S7. After performing a rule compliance check on the draft requirement model, execute an iterative verification and correction loop based on the check results to obtain the target requirement model.

[0115] S7.1 It should be noted that after the requirement model draft is input into the formal rule checker, the formal rule checker first reads all the text content in the requirement model draft and performs lexical segmentation on the requirement model draft according to the modeling language syntax rules, identifying keywords, identifiers, relation symbols, hierarchy symbols and delimiters; after completing the lexical segmentation, the formal rule checker performs syntactic parsing on the segmentation results, identifying node definition statements, relation definition statements, sequence definition statements and constraint definition statements in the requirement model draft layer by layer, and establishing hierarchical subordinate relationships and connection relationships between statements; during the syntactic parsing process, use case actors, use case behaviors, processing objects, interaction steps, interface interaction elements, process sequence nodes, process step nodes, constraint condition nodes and their corresponding relationships are respectively converted into nodes in the internal abstract syntax tree, and the participation relationship, business processing relationship, interaction logic relationship, process sequence relationship and constraint dependency relationship are converted into edges in the internal abstract syntax tree, and the nodes and edges are organized according to the syntactic subordinate relationship and semantic connection relationship to obtain the internal abstract syntax tree representation used for subsequent rule compliance checks.

[0116] S7.2 It should be noted that the node type, node attributes, and relationships between nodes in the internal abstract syntax tree are read. At the same time, a matching and logical judgment are performed one by one according to the preset type constraint rules, attribute constraint rules (including integrity rules and uniqueness rules), and relationship constraint rules (including connection rules and direction rules). Specifically, when the use case actor node represents a user role, it matches the type constraint rule; when the use case behavior node represents a function item, it matches the type constraint rule; when the business object node represents a business object, it matches the type constraint rule.

[0117] For node attributes, when the use case participant name is not empty, it matches the integrity rule; when the process step number is not repeated, it matches the uniqueness rule.

[0118] For relationships between nodes, when the participation relationship is from the use case participant node to the use case behavior node, it matches the connection rule and direction rule; when the business processing relationship is from the use case behavior node to the business object node, it matches the connection rule and direction rule; when the process sequence relationship is from the preceding process step node to the following process step node, it matches the connection rule and direction rule.

[0119] When the node type, node attribute, or relationship between nodes does not match the corresponding rule, record the instance of the violation and write the violation node identifier, violation relationship identifier, violation rule type, violation location, and violation reason. After completing the traversal, matching, and logical judgment of all nodes in the internal abstract syntax tree, summarize all instances of rule violations and generate an inspection result report.

[0120] Furthermore, the process of setting the preset type constraint rules, attribute constraint rules, and relation constraint rules is as follows: Based on the model syntax definition corresponding to the target requirement model, the allowed node types, the node attributes that each node should have, and the allowed connection relationships between nodes are organized to form type constraint rules, attribute constraint rules, and relation constraint rules; among them, the type constraint rules are used to limit the node type correspondence of different semantic nodes in the internal abstract syntax tree, the attribute constraint rules are used to limit the necessary attributes that each node should contain and the attribute value requirements, and the relation constraint rules are used to limit the connection relationships that can be established between different nodes and the connection direction.

[0121] S7.3 It should be noted that the inspection result report is checked to see if it is empty. If the inspection result report is empty, the current requirement model draft is used as the target requirement model.

[0122] If the inspection result report is not empty, the violation node identifier, violation relationship identifier, violation rule type, violation location, and violation reason in the inspection result report are read. Based on the violation location, the corresponding node definition fragment, relationship definition fragment, sequence definition fragment, or constraint definition fragment in the requirement model draft is located to obtain the violation fragment. Based on the violation node identifier and violation relationship identifier, the nodes and edges corresponding to the violation fragment are retrieved in the requirement semantic knowledge graph. The adjacent nodes, adjacent edges, and uncertainty markers directly connected to the corresponding nodes and edges are further read. The retrieved and read nodes, edges, and connection relationships constitute a local subgraph.

[0123] Furthermore, the "relationship to be confirmed" is used to characterize uncertain relationships retained in previous steps due to insufficient evidence. A relationship to be confirmed does not automatically constitute a rule violation. When a relationship to be confirmed is expressed in the draft requirement model according to the preset confirmation marking rules and does not violate the preset type constraint rules, attribute constraint rules, and relationship constraint rules, the relationship to be confirmed will not be written into the inspection result report. An empty inspection result report means that there are no nodes, attributes, or relationship instances in the draft requirement model that violate the preset rules. It does not require the relationship to be confirmed field to be empty, nor does it require that all relationships to be confirmed be deleted from the draft requirement model.

[0124] S7.4. It should be noted that the violation type, violation reason, violation node identifier, and violation relationship identifier in the inspection result report should be read, and combined with the semantic category, node attributes, edge association type, and uncertainty marker of the corresponding node in the local subgraph, the object to be corrected, the content to be corrected, and the rule requirements to be met should be determined. Specifically, when the violation type is node type error, the correct semantic category and allowed node type of the corresponding node in the local subgraph should be written into the correction instruction; when the violation type is missing or duplicate node attributes, the attribute content to be supplemented, deleted, or replaced for the corresponding node in the local subgraph should be written into the correction instruction; when the violation type is relationship direction error, relationship connection error, or constraint dependency error, the starting node, target node, or edge type to be adjusted for the corresponding edge in the local subgraph should be written into the correction instruction; when the violation type is process sequence error, the sequential relationship that should be satisfied between the corresponding process step nodes in the local subgraph should be written into the correction instruction; the correction object, correction content, rule requirements, and relevant semantic context in the local subgraph should be organized according to the preset instruction template to form the correction instruction. The system extracts the corresponding prompts from the structured prompts and combines them with correction instructions to form corrected prompts. These corrected prompts then drive a multimodal large-scale model to perform local regeneration, resulting in the corrected violation fragment. The preset instruction template includes fields for violation type, correction object, correction content, rule requirements, context, and prompt content. The order of these fields in the instruction template matches the order of input elements required by the multimodal large-scale model during local regeneration, improving the accuracy of the model's understanding of the violation fragment correction task and the stability of the local regeneration results.

[0125] In the draft requirement model, the identified violation fragments are replaced with the corrected violation fragments to obtain a new draft requirement model. The new draft requirement model is then subjected to rule compliance checks again until the check result report is empty, at which point the target requirement model is obtained.

[0126] This embodiment also provides a computer device applicable to the software requirements modeling method based on a large artificial intelligence model, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the software requirements modeling method based on a large artificial intelligence model as proposed in the above embodiment.

[0127] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0128] This embodiment also provides a storage medium storing a computer program, which, when executed by a processor, implements the software requirements modeling method based on a large artificial intelligence model as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0129] In summary, this invention improves the semantic consistency of multimodal software requirement materials and reduces the impact of fragmented text and image information on requirement understanding through a two-layer semantic alignment and reconstruction process. It transforms the fused semantic element set into a requirement semantic knowledge graph and adds uncertainty markers to edges with insufficient association basis, enabling the distinction between candidate associations and explicit associations in the requirements. This improves the clarity and completeness of the requirement semantic organization process. Furthermore, it utilizes a formal rule checker to perform rule compliance checks on the requirement model draft and executes iterative verification and correction loops based on the check result report, allowing non-compliant segments in the requirement model draft to be located, corrected, and backfilled, thereby improving the accuracy, completeness, and usability of the target requirement model.

[0130] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A software requirements modeling method based on a large-scale artificial intelligence model, characterized in that, include, Receive multimodal software requirement materials provided by users, and use the multimodal large model to parse them to obtain the initial semantic elements corresponding to each modality; Perform two-layer semantic alignment reconstruction on the initial semantic elements of different modalities to obtain a fused semantic element set; The set of fused semantic elements is converted into nodes and edges. During the conversion process, uncertainty markers are added to edges with insufficient association basis to construct a demand semantic knowledge graph. A demand model generation scheme is constructed based on a demand semantic knowledge graph. The requirement model generation scheme is filled into the preset structured prompt word template to obtain structured prompts, and the structured prompts are used to drive the multimodal large model to generate a requirement model draft; After performing a rule compliance check on the draft requirement model, an iterative verification and correction loop is executed based on the check results to obtain the target requirement model.

2. The software requirements modeling method based on a large artificial intelligence model as described in claim 1, characterized in that, The multimodal software requirements materials include natural language text and software requirements visualization images.

3. The software requirements modeling method based on a large artificial intelligence model as described in claim 2, characterized in that, The specific steps for obtaining the initial semantic elements corresponding to each modality are as follows: Natural language text and software requirement visualization images are input into a multimodal large model. The multimodal large model performs entity recognition, relation extraction and semantic role labeling on the natural language text to obtain the initial semantic elements of the text. The multimodal large model performs visual element recognition on the software requirement visualization image, extracts the user interface elements, the relationships between elements, and the structured text of the related business function descriptions, and obtains the initial semantic elements of the image.

4. The software requirements modeling method based on a large artificial intelligence model as described in claim 3, characterized in that, The acquisition of the fused semantic element set specifically involves: The initial semantic elements of the text and the initial semantic elements of the image are vectorized and encoded to obtain text semantic vectors and image semantic vectors; Calculate the semantic similarity between text semantic vectors and image semantic vectors, and construct cross-modal candidate association pairs by combining semantic element type constraints; For each cross-modal candidate association pair, the consistency of attributes, behavior, and context dependency is used to determine the cross-modal candidate association pair, and the cross-modal candidate association pair is divided into consistent semantic pairs and conflicting semantic pairs. Consistent semantic pairs are merged and reconstructed to generate core fused semantic elements; For conflicting semantic pairs, retain the conflict marker, conflict source, and corresponding semantic content, and generate semantic elements to be fused. The core fusion semantic elements, the fusion semantic elements to be determined, and the initial semantic elements that have not formed cross-modal candidate association pairs are combined to form a fusion semantic element set.

5. The software requirements modeling method based on a large artificial intelligence model as described in claim 4, characterized in that, The construction of the requirement semantic knowledge graph specifically involves: Extract each fused semantic element from the fused semantic element set; Based on the semantic category of the fused semantic elements, the fused semantic elements are mapped to corresponding type nodes in the knowledge graph, and attributes are assigned to the nodes; Analyze the semantic relationships between different fused semantic elements, and establish a corresponding edge between the two corresponding nodes based on the relationship type; During the edge establishment process, association determination is performed, and an uncertainty marker is added to edges with insufficient association basis; The connections between nodes and edges are summarized to form a semantic knowledge graph of requirements.

6. The software requirements modeling method based on a large artificial intelligence model as described in claim 5, characterized in that, The aforementioned demand model generation scheme is constructed by reading nodes and edges in the demand semantic knowledge graph and constructing the model based on the semantic category of the nodes and the association type of the edges.

7. The software requirements modeling method based on a large artificial intelligence model as described in claim 6, characterized in that, The process of obtaining structured prompts refers to extracting the model type, node semantics, relationship organization, and unconfirmed relationships from the requirement model generation scheme, and then filling them into the corresponding fields in the structured prompt word template to obtain structured prompts.

8. The software requirements modeling method based on a large artificial intelligence model as described in claim 7, characterized in that, The method of using structured prompts to drive the generation of a draft requirement model from a multimodal large model specifically includes: The structured prompts are submitted as input instructions to the multimodal large model; The multimodal large model performs forward inference computation based on the modeling task, modeling elements and modeling logic required in the structured prompts; Based on the pre-defined modeling language, the forward inference calculation results are organized into a draft requirement model.

9. The software requirements modeling method based on a large artificial intelligence model as described in claim 8, characterized in that, The rule compliance check on the draft demand model specifically includes: The requirement model draft is input into the formal rule checker, which performs syntax parsing on the requirement model draft and converts it into an internal abstract syntax tree representation. It iterates through each node of the internal abstract syntax tree, matching and logically judging the node's type, attributes, and relationships with preset rules one by one, recording all instances of rule violations, and generating an inspection result report.

10. The software requirements modeling method based on a large artificial intelligence model as described in claim 9, characterized in that, The step of performing iterative verification and correction loops based on the inspection results to obtain the target requirement model is as follows: Determine if the inspection result report is empty. If the inspection result report is empty, then use the current requirement model draft as the target requirement model. If the inspection result report is not empty, then based on the violation description in the inspection result report, locate the violation fragment in the requirement model draft and map it to the requirement semantic knowledge graph to obtain a local subgraph; Based on the violation descriptions and local sub-graphs in the inspection results report, generate correction instructions; Extract the prompt content corresponding to the violation fragment from the structured prompts, combine it with the correction instructions to form a corrected prompt, and use the corrected prompt to drive the multimodal large model to perform local regeneration to obtain the corrected violation fragment; The corrected violation fragments were then added back to the draft requirements model to form a new draft requirements model. The new draft demand model will undergo another rule compliance check until the check result report is empty.

Images