Intelligent generation method and system for executable process and detection item based on industry agent

Through an intelligent generation method based on industry agents, a multimodal vertical model is used to process unstructured data, and an XML file that complies with the BPMN2 specification is generated, which solves the problem of complex process automation design and domain knowledge embedding, and improves the simulation of business processes and optimization capabilities achieve efficient intelligent generation effects.

CN119046618APending Publication Date: 2024-11-29DMAI (GUANGZHOU) CO LTD

Patent Information

Application Number
CN202411253255.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-09-09
Publication Date
2024-11-29

AI Technical Summary

Technical Problem

The existing intelligent generation methods are difficult to fully automated design of complex processes, difficult to effectively embed domain knowledge, and weak ability to adapt to changing business environments.

Method used

Using an intelligent generation method based on industry agents to obtain unstructured data, preprocess and multimodal data fusion, and using multimodal vertical model for training and fine-tuning to generate BPMN2-compliant Standardized XML files and in-depth analysis and compliance detection through automated verification modules.

Benefits of technology

Realize natural language interpretation and intelligent generation of complex business logic, improve the simulation and optimization capabilities of business processes, enhance the professional accuracy and adaptability of the model, reduce human errors, and improve the accuracy and reliability of process definition. .

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119046618A_ABST
    Figure CN119046618A_ABST
Patent Text Reader

Abstract

The invention discloses an intelligent generation method and system for an executable process and a detection item based on an industry agent, and the method comprises the following steps: obtaining unstructured data which comprises text data and image data containing the related information of the executable process; preprocessing the collected unstructured data, and generating multi-modal data fused by text data and image data; multi-modal data obtained after preprocessing is input into the trained multi-modal vertical class large model, and a corresponding XML file is generated; and verifying the generated XML file to obtain a final XML file passing the verification. According to the method and the system, the service flow can be simulated and optimized in a larger range, and the complex process and the accurate detection item of the specific industry can be generated.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and more specifically to a method and system for intelligently generating executable processes and detection items based on industry intelligent agents. Background Art

[0002] At present, after years of research and practice, the intelligent generation technology of executable processes and test items has gradually built a set of universal method systems, covering multiple key links from demand analysis, rule definition, algorithm design to automatic generation. This series of processes aims to use advanced information technology, especially artificial intelligence and machine learning technology, to automatically extract effective process logic and test specifications from a large amount of business data and expert experience, and transform them into actual executable process models and test item lists.

[0003] Traditional manual design of processes and inspection items is not only time-consuming and labor-intensive, but also easily affected by subjective factors, resulting in inefficiency and inconsistency. Intelligent generation methods, on the other hand, use complex mathematical models and optimization algorithms to achieve process automation design, such as using theories based on constraint satisfaction problems (CSP), formal methods, or Petri nets to characterize process structure and dynamic behavior. For inspection items, with the help of deep learning and pattern recognition technology, potential rules can be discovered from historical case libraries, and checklists suitable for different scenarios can be automatically generated.

[0004] Nevertheless, current intelligence generation methods still face a series of challenges, such as the difficulty in designing complete automation of complex processes, the difficulty in effectively embedding domain knowledge, and the weak ability to adapt to the ever-changing business environment.

[0005] Therefore, how to achieve natural language interpretation and intelligent generation of complex business logic and simulate and optimize business processes on a larger scale is a technical problem that needs to be urgently solved in this field. Summary of the invention

[0006] In view of this, the present invention provides a method and system for intelligently generating executable processes and detection items based on industry intelligent agents, which are used to at least solve some of the technical problems in the background technology.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] The present invention discloses a method for intelligently generating executable processes and detection items based on an industry agent, comprising the following steps:

[0009] Acquiring unstructured data, wherein the unstructured data includes text data and image data containing information related to the executable process;

[0010] Preprocess the collected unstructured data and generate multimodal data that integrates text data and image data;

[0011] Input the preprocessed multimodal data into the trained multimodal vertical model to generate the corresponding XML file;

[0012] The generated XML file is verified to obtain a final XML file that passes the verification.

[0013] Furthermore, the collected unstructured data is preprocessed, including: data cleaning, data standardization and data format conversion.

[0014] Furthermore, the multimodal vertical model is trained by the following steps:

[0015] Obtaining a training data set, wherein the training data set includes industry knowledge base data and manually labeled industry positive sample data;

[0016] The industry knowledge base data is abstracted into several knowledge units. The multimodal vertical model learns based on the knowledge units and establishes in-depth insights into the corresponding industry field at the underlying knowledge level.

[0017] Convert manually labeled industry positive sample data into several business process instances F j and the corresponding standardized process description P j ,The multi-modal vertical large model enhances the understanding of the inherent rules of the business process by comparing and learning ,business process instances and the corresponding standardized process descriptions;

[0018] During the model training process, the model parameters θ are continuously updated iteratively through the back propagation algorithm and the Adam optimizer to minimize the prediction process description With standardized process description P j The loss function between .

[0019] Furthermore, the prediction process is described as Compared with the actual marking P j The loss function between , specifically includes the following functions:

[0020] Furthermore, it also includes fine-tuning the trained multi-modal vertical model, including:

[0021] Input the original document into the multimodal vertical model to generate the corresponding model output XML file;

[0022] Manually verify and optimize the original document to obtain the corresponding manually optimized XML file;

[0023] By comparing the model output XML files and manually optimized XML files, the ability of multimodal vertical large models to simulate real business processes can be improved.

[0024] Furthermore, the multimodal data obtained after preprocessing is input into the trained multimodal vertical model to generate the corresponding XML file, which specifically includes:

[0025] Input the multimodal data into the trained multimodal vertical model to generate a corresponding XML file containing the main process definition information. The main process definition information includes activity nodes Task, events Event, gateway Gateway, data objects Data Object and the relationship chain between them.

[0026] Furthermore, verifying the generated XML file specifically includes the following steps:

[0027] Validate the received XML files according to preset structural specifications, logical rules and business requirements.

[0028] The present invention also discloses an intelligent generation system of executable processes and detection items based on industry intelligent agents, including an unstructured data acquisition module, a data preprocessing module, a multi-modal vertical model and an automated verification module;

[0029] Wherein, the unstructured data acquisition module is used to acquire unstructured data, and the unstructured data includes text data and image data;

[0030] The data preprocessing module is used to preprocess the collected unstructured data and generate multimodal data fused with text data and image data;

[0031] The multimodal vertical model is used to receive the preprocessed multimodal data and generate a corresponding XML file based on the multimodal data;

[0032] The automatic verification module is used to verify the generated XML file.

[0033] Preferably, the above system also includes a model training module, which is used to train the multimodal vertical classification model using a training data set.

[0034] Preferably, the training process of the multimodal vertical model specifically includes the following steps:

[0035] Obtaining a training data set, wherein the training data set includes industry knowledge base data and manually labeled industry positive sample data;

[0036] The industry knowledge base data is abstracted into several knowledge units. The multimodal vertical model learns based on the knowledge units and establishes in-depth insights into the corresponding industry field at the underlying knowledge level.

[0037] Convert manually labeled industry positive sample data into several business process instances F j and the corresponding standardized process description P j ,The multi-modal vertical large model enhances the understanding of the inherent rules of the business process by comparing and learning ,business process instances and the corresponding standardized process descriptions;

[0038] During the model training process, the model parameters θ are continuously updated iteratively through the back-propagation algorithm and the Adam optimizer to minimize the loss function between the predicted process description and the actual annotation.

[0039] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a method and system for intelligently generating executable processes and detection items based on industry intelligent agents, which has the following beneficial effects:

[0040] 1. The unstructured data acquisition module is able to process text and image data in various formats, which provides a large amount of raw data for subsequent analysis.

[0041] 2. Data preprocessing: The data preprocessing module ensures data quality through operations such as cleaning, standardization, and format conversion, providing accurate input for model training.

[0042] 3. Rich sample data sets: The sample data sets include industry knowledge bases, manually annotated positive sample data, and XML files generated and optimized by the model. This combination helps the model to comprehensively learn domain knowledge and business processes.

[0043] 4. Model training and fine-tuning: The model training / fine-tuning module trains and fine-tunes the large language model based on the sample data set, improving the professional accuracy and adaptability of the model.

[0044] 5. Standardized process output: The multimodal vertical model can generate XML files that comply with the BPMN2 specification. These files describe the main process and sub-process information in detail, ensuring the standardization and universality of the process definition.

[0045] 6. Automated Verification: The automated verification module uses formal methodologies and technical means to conduct in-depth analysis and compliance testing of the generated XML files, reducing human errors and improving the accuracy and reliability of process definitions.

[0046] 7. Graphical review and optimization interface: The process and inspection item review / editing module converts XML files into graphical flow charts, which facilitates experts or business personnel to conduct intuitive review and optimization, improving the interactivity and user-friendliness of process design.

[0047] 8. Closed-loop mechanism for continuous improvement: The audited and optimized XML files are fed back into the sample dataset for further fine-tuning of the model, forming a closed-loop mechanism for continuous learning and improvement, enabling the model to continuously improve performance and accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0049] Figure 1 A schematic diagram of the overall process flow of the intelligent generation method of executable processes and detection items based on industry intelligent agents provided by the present invention.

[0050] Figure 2 A schematic diagram of the architecture of the intelligent generation system for executable processes and detection items based on industry intelligent agents provided by the present invention.

[0051] Figure 3 A flow chart of the unloading operation procedures of an LNG storage and distribution station provided in an embodiment of the present invention.

[0052] Figure 4 The detection item provided for the embodiment of the present invention is a schematic diagram of the main process nodes of the operation when the tank truck is pressurized. DETAILED DESCRIPTION

[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0054] like Figure 1 As shown, the present invention first discloses an intelligent generation method of executable processes and detection items based on industry intelligent agents, comprising the following steps:

[0055] Acquire unstructured data, where the unstructured data includes text data and image data;

[0056] Preprocess the collected unstructured data and generate multimodal data that integrates text data and image data;

[0057] Input the preprocessed multimodal data into the trained multimodal vertical model to generate the corresponding XML file;

[0058] The generated XML file is verified to obtain a final XML file that passes the verification.

[0059] Based on the above method, the present invention also discloses an intelligent generation system of executable processes and detection items based on industry intelligent agents, including an unstructured data acquisition module, a data preprocessing module, a multi-modal vertical model and an automated verification module;

[0060] Wherein, the unstructured data acquisition module is used to acquire unstructured data, and the unstructured data includes text data and image data;

[0061] The data preprocessing module is used to preprocess the collected unstructured data and generate multimodal data fused with text data and image data;

[0062] The multimodal vertical model is used to receive the preprocessed multimodal data and generate a corresponding XML file based on the multimodal data;

[0063] The automatic verification module is used to verify the generated XML file.

[0064] In a specific embodiment, the intelligent generation system of executable processes and detection items based on industry intelligent agents proposed by the present invention is as follows: Figure 2 As shown in the figure, it can be seen that the system includes 4 processing modules, namely data preprocessing module, model training / fine-tuning module, multimodal vertical model and automated verification module. It also includes 3 types of input information, namely unstructured data acquisition module, sample data set and output data structure and standard requirements. The output information in the invention is the process and detection item review / editing module. The following explains these 8 contents in the present invention respectively.

[0065] (1) Unstructured data collection module

[0066] The unstructured data acquisition module plays a vital role in the data analysis and processing system. It focuses on obtaining original information resources that are not presented in a fixed pattern or relational structure. The core task of this module is to effectively capture and integrate unstructured data in the form of text and images, thereby providing a rich and diverse basic data source for subsequent data preprocessing, analysis and mining, and decision support.

[0067] First, for the collection of text data, this module is compatible with and can process a variety of file formats, including but not limited to PDF documents, plain text (TXT) files, and office document formats such as Microsoft Word (DOC or DOCX). These text data contain rich business logic, professional knowledge and even potential market trend information. For PDF documents, the text content may be extracted through OCR technology and text parsing algorithms; for Word documents, they are directly read and converted into a processable text format. The specific formula or algorithm to implement this process can use text extraction technology in the field of natural language processing (NLP), such as rule-based methods, statistical learning methods, or deep learning models, to ensure efficient and accurate capture and conversion of various types of text data.

[0068] Secondly, the collection of image data is also a key link. The unstructured data collection module can adapt to various common image formats, such as JPEG (.jpg), PNG (.png), BMP (.bmp) and some other professional image formats. These image data cover a variety of scenarios from scanned images, surveillance videos and system screenshots, each of which contains valuable information. The processing of image data usually involves computer vision technologies such as image recognition, image classification, and target detection, and the image content is understood and interpreted by training a special neural network model. For example, in the process of image recognition, a convolutional neural network (CNN) can be used to extract features from the image, and then the softmax function or other classifiers can be used to output the category label of the image, which is expressed in mathematical expression as:

[0069]

[0070] Among them, I represents the input image, y is the target category corresponding to the image, θ represents the network parameters, z is the score vector of each category calculated by the network, and K is the total number of categories.

[0071] Finally, the successfully collected unstructured text and image data will be transmitted to the data preprocessing module, which will perform a series of operations such as cleaning, standardization, and format conversion to make the original data meet the needs of the subsequent analysis module. For example, text data may need to be preprocessed such as word segmentation, stop word removal, and stem extraction, while image data may undergo grayscale, normalization, and resizing operations. The entire unstructured data collection and preprocessing process together constitutes the first stage of the data analysis pipeline, laying a solid foundation for subsequent in-depth mining and utilization of data value.

[0072] (2) Data preprocessing module

[0073] The data preprocessing module plays a vital role as a bridge between big data analysis and artificial intelligence applications. It receives rich and diverse raw data transmitted by the unstructured data acquisition module, including text data (such as PDF, TXT, Word documents and other formats) and image data (such as JPG, PNG and other common image formats). Only after preprocessing can these unstructured data meet the data input requirements of subsequent advanced analysis tools - multi-modal vertical large models.

[0074] In terms of text data preprocessing, the module first performs data cleaning to remove irrelevant noise and outliers, such as HTML tags, special characters, and whitespace. Next, text standardization is performed, including but not limited to unifying upper and lower case, stem extraction (such as using the Porter Stemmer algorithm), and removing stop words (such as "the", "a", "an" and other words without actual meaning in English). Bag of Words (BoW) or TF-IDF vectorization can also be performed to convert text data into a numerical matrix for subsequent model processing. In terms of formula expression, TF-IDF can calculate the importance of a word t in a document d, that is:

[0075] TF-IDF(t,d,D)=TF(t,d)×IDF(t,D)

[0076] Among them, TF(t,d) is the term frequency of word t in document d, and IDF(t,D) is the inverse document frequency of word t in document set D.

[0077] In the image data preprocessing stage, image enhancement is first performed to improve image quality and consistency, which includes brightness / contrast adjustment, grayscale processing, histogram equalization and other operations. Subsequently, size normalization is performed to scale the image to a fixed size suitable for model input, such as using interpolation methods for scaling. Next, the image data may also need to be normalized, that is, all pixel values ​​are mapped to between 0 and 1, or whitened by subtracting the mean and dividing by the standard deviation. In terms of formula expression, normalization can be written as:

[0078]

[0079] Where x is the original pixel value, μ is the mean of the data set, and σ is the standard deviation of the data set.

[0080] Finally, for large multimodal vertical models, the data preprocessing module also needs to complete the task of data fusion, that is, to effectively combine the preprocessed text data and image data to form a multimodal representation, such as splicing or fusing text feature vectors and image feature vectors through a joint embedding space, thereby meeting the requirements of large models to process multiple types of data simultaneously.

[0081] In short, the data preprocessing module not only improves the data quality through meticulous processing and conversion of unstructured data, but also provides standardized inputs that are easy to understand and parse for large multimodal vertical models, thereby supporting the model to achieve efficient and accurate analysis, prediction and decision support.

[0082] (3) Sample Dataset

[0083] Sample datasets are an indispensable basic resource for building high-performance large language models. They first undergo a rigorous data preprocessing process, which is performed by a dedicated data preprocessing module to convert raw data into a structured and standardized form that can be used for model training. This preprocessing stage includes multiple links such as data cleaning, format unification, missing value processing, and outlier detection to ensure that the final dataset is of high quality and conducive to model learning.

[0084] The sample dataset is broken down into three key components:

[0085] 1. Industry knowledge base content: This part constitutes the first pillar of the data set, which contains rich industry background knowledge and practice rules. For example, legal and regulatory texts, industry consensus standards, various operation manuals, and safety management system documents, etc., are all important knowledge carriers in the field. It can be expressed as K = {k1, k2, ..., kn}, where each

[0086] ki corresponds to a specific knowledge item, such as a regulatory document or a work instruction document.

[0087] Example: An industry knowledge base entry (k_j) might be a specific Building Safety Act text or a detailed Chemical Storage Safety Guide.

[0088] 2. Manually labeled positive sample data: This part of the data plays the role of "textbook" and is the key to the model learning how to correctly understand and generate the target output. It includes examples with clear labels or corresponding structured representations, such as safety guidance documents for high-altitude operations and their XML representation files that follow the Business Process Modeling Notation (BPMN) 2.0 standard. These paired data are in the form of (F j ,P j ), where Fj is the original text guide, and P j It is the corresponding standard XML file.

[0089] Example: Positive sample pair (F j ,P j ), where (F j ) is the operation procedure text of aerial work, and (P j ) is an XML flowchart file written according to the BPMN2.0 specification.

[0090] 3. A collection of XML files generated by a multimodal vertical model: This part of the data is a new sample generated based on an existing model, usually used in the fine-tuning stage of the model. The model automatically generates a series of XML files that conform to the BPMN2.0 specification, as well as the original documents corresponding to these XML files and optimized XML file versions after manual review and correction. This set of data triplets can be written as (O1, R1, E1), where O1 is the XML file generated by the model, R1 is the corresponding real-world original document, and E1 is a more accurate XML file obtained after manual review of O1.

[0091] Example: Model generation sample (O1, R1, E1), where O1 is the XML process description generated by the model based on the original document R1, and E1 has been manually verified and optimized to improve the accuracy of the process description.

[0092] In the actual large language model training process, the contents of the first and second parts above mainly serve the "model training module". Through the input of a large amount of industry knowledge base content and positive sample data with clear annotations, the model can learn the standardized expression of the characteristics and standard processes of various industries. The third part of the data set focuses on the use of the "model fine-tuning module". By introducing the model's own generated samples and manual optimization results, the model can continuously improve its refined processing capabilities and generation accuracy in specific vertical tasks in a closed-loop feedback mechanism, and achieve efficient migration and generalization from theoretical knowledge to practical applications.

[0093] (4) Model training / fine-tuning module

[0094] The model training and fine-tuning modules play a vital role in the process of building multimodal vertical models. They make full use of carefully designed sample data sets to conduct targeted training and adjustments on large-scale language models, aiming to enable them to deeply master the knowledge structure of specific fields and accurately simulate the professional qualities of human experts in business process description. This process not only relies on a large amount of rich and diverse data input, but also requires a set of rigorous methodologies and technical means.

[0095] The design of the sample dataset is crucial and covers three core components. First, the content of the industry knowledge base constitutes the basic part of the dataset, which includes but is not limited to various legal and regulatory texts, industry general standards, specific operation guidelines, safety management procedures, etc. These documents are rich in deep professional knowledge in the field and can be abstracted into groups of knowledge units K. i By learning these knowledge units, the model gradually builds in-depth insights into the field at the underlying knowledge level:

[0096] K={K1,K2,…,Km}

[0097] Secondly, manually annotated positive sample data provides clear and accurate learning examples for model training. For example, in the field of aerial work, such data may include detailed work instructions and corresponding BPMN 2.0 standard XML files. For each specific business process instance F j , there is a corresponding standardized process description P j , the model learns F by contrast j and P j To strengthen the understanding of the inherent laws of business processes:

[0098] Dtrain={(F1,P1),(F2,P2),…,(Fn,Pn)}

[0099] During the model training process, the model parameters θ are continuously updated iteratively through the back-propagation algorithm and optimizers such as the Adam optimization algorithm to minimize the loss function between the predicted process description and the actual annotation:

[0100]

[0101] Finally, the model fine-tuning phase focuses on the model's expressiveness in generating XML files that conform to the BPMN 2.0 specification. The dataset for this phase includes the XML files generated by the model itself, as well as the original documents corresponding to these files and the XML files optimized after manual review. By comparing the model output with the expert-level optimization results, the model can further improve its ability to simulate real business processes:

[0102] Dfine-tune={(O1,R1,E1),(O2,R2,E2),…,(Ok,Rk,Ek)}

[0103] Among them, O i Represents the XML file generated by the model, R i is the original document, E i It is an ideal XML file after manual optimization.

[0104] When fine-tuning the model, the loss function is also used to measure the gap between the XML file generated by the model and the manually optimized version, and detailed adjustments are made on this basis:

[0105]

[0106] To summarize, the model training and fine-tuning module integrates these three types of sample data to gradually improve the adaptability and professional accuracy of the language big model in specific vertical fields, and finally successfully builds a multimodal vertical big model. It can not only deeply understand and grasp the professional knowledge in the field, but also accurately reproduce the expression details of various business processes in a nearly expert level manner, greatly broadening the practicality and effectiveness of the model in vertical application scenarios.

[0107] (5) Multimodal vertical large model

[0108] As an advanced AI technology solution, multimodal vertical large models are designed and implemented to enable efficient learning and reasoning across multiple data modalities. The core function of this type of model is to process and integrate data from different sources and in different forms, such as text, voice, images, and even sensor data, in order to more fully understand and simulate complex real-world situations. When the model is applied to the field of business process management, it can connect to the results output by a carefully designed data preprocessing module. This preprocessing process is extremely critical, and it covers steps such as data cleaning, standardization, integration, and conversion to ensure that the model can effectively digest and absorb various types of unstructured and structured information.

[0109] The network structure in the multimodal vertical model may include BERT, ViT, CNN, GNN, LSTM, etc., in which the input layer is adjusted according to different modalities, and the visual features and audio features are associated for fusion, and specific homeostatic functions are used to optimize the tasks in this field.

[0110] After completing the data preprocessing phase, the multimodal vertical model will receive the formalized data that has been converted into data suitable for model processing, and generate an XML file based on this data that complies with the Business Process Model and Notation (BPMN) 2.0 standard. BPMN 2.0 is an internationally recognized graphical business process modeling language. Its XML format can describe the process structure and interaction logic in detail, thereby realizing visual management and automated execution of business processes.

[0111] The XML file generated by the multimodal vertical model usually contains detailed main process definition information, such as activity nodes (Task), events (Event), gateways (Gateway), data objects (Data Object) and the relationship chain between them. A simplified example may be as follows:

[0112]

[0113] The XML file contains a macro view of the entire process - the main process and its nested sub-processes. The main process depicts the main trajectory of the business process from the beginning to the end, while the sub-process is refined into the various sub-stages within the main process. Each sub-process consists of a series of interrelated activities (test items). The "activities" or "test items" here are actually specific tasks or operation units to be performed in the process. Their precise definition and arrangement directly affect the effectiveness and efficiency of the entire process.

[0114] The activities (detection items) in the XML file have a rigorous structure and rich attributes, which helps to clearly define the key elements of each task. The reference example is as follows:

[0115] 1. Name: The name of the activity (detection item) is a concise and clear description of the task, which is easy to understand and identify, such as "document review", "order confirmation", etc.

[0116] 2. Type: The activity (detection item) type is used to distinguish tasks of different natures, such as manual tasks (performed manually), automatic tasks (automatically executed by the system), script tasks (executed based on program code), etc.

[0117] 3. Preconditions: Preconditions specify the prerequisites for the execution of an activity, which are usually based on the state of certain variables or business rules. They can be expressed in Boolean expressions or other logical forms. For example:

[0118]

[0119] This means that the activity can only be started if the "approved by manager" condition is met.

[0120] 4. Operation steps: The operation steps list in detail the specific operation sequence required to perform the activity, which can include user operation instructions, system processing logic, or interaction processes with other systems. In the actual XML file, this part of the content may not be directly reflected as a single element, but will be indirectly reflected in the extended element that describes the activity behavior.

[0121]

[0122] 5. Post-action: The post-action after the activity is completed usually refers to the processing and response to the execution results of the activity, such as updating the database status, triggering the next activity, sending notification emails, etc.

[0123] Similarly, post-actions can also be expressed through extended attributes or associated elements of XML tags.

[0124]

[0125] After the XML file is generated, it will be parsed by a specially designed BPMN 2.0 process engine, which can interpret the structure and logic in the XML file and convert the abstract process model into an actual executable process instance. In this process, users can edit and optimize the activities (detection items) in the main process and sub-process according to their needs. The methodologies and tools used may include process simulation and emulation, Lean Six Sigma, decision tree analysis, machine learning-assisted optimization, etc.

[0126] For example, in the process optimization stage, simulation software can be used to simulate the process described in the XML file, evaluate the execution time, resource usage and bottlenecks of each activity (detection item), and then use queuing theory, Petri net theory and other methodologies to analyze and optimize the timing relationship and resource allocation of the process. At the same time, combined with actual operation data, statistical methods are used to study the dependencies between activities (detection items), and algorithms such as genetic algorithms and particle swarm optimization are used to find the best process improvement plan.

[0127] In short, after receiving pre-processed data and generating BPMN2.0-compliant XML files, the multimodal vertical model lays a solid foundation for the automated execution and continuous optimization of business processes. By carefully defining and managing each link of activities (detection items) in the main process and sub-process in the XML file, organizations can more effectively control and improve business processes during the digital transformation process, thereby improving efficiency, reducing costs and enhancing competitiveness.

[0128] (6) Output data structure and standard requirements

[0129] The concept of output data structure and standard requirements has far-reaching significance in the field of computer science and process modeling, especially when following the internationally recognized Business Process Model and Notation (BPMN) 2.0 specification, which provides indispensable guidelines and reference architecture for building and parsing corresponding XML files. This rigorous data structure and standard requirements not only define the organization and element composition of XML files, but also become the key basis for the automated verification module to verify the output results of multi-modal vertical large models.

[0130] Under the BPMN 2.0 specification, the XML file generated by the multimodal vertical model fully reflects the meticulousness and complexity of business process design. In terms of file structure, it not only includes the main process at the macro level, but also describes the embedded subprocesses in detail. The main process consists of a series of key components, which can be represented by mathematical set theory and tree diagram models, including:

[0131] 1. Activity node (Task): denoted as T∈T, where T is the set of all activity nodes. Each node represents a specific business operation and has a unique ID identifier, name, and type attributes. It can be accessed through XML tags. <bpmn:task>Give a description.

[0132] 2. Event: denoted as E∈E, where E is the set of all events. Events trigger the transition of process states, such as start events, end events, intermediate catch events, etc., which can be used <bpmn:event>Type tag to encapsulate.

[0133] 3. Gateway: denoted as G∈G, where G contains different types of gateway sets, such as parallel gateways, inclusive gateways, etc. They play a decision-making role in the process flow. <bpmn:gateway>Label indicates.

[0134] 4. Data Object: D∈D, where D represents a collection of data objects. Data objects are used to transfer, store and reference business data in the process.

[0135] <bpmn:dataobject>Tag definition.

[0136] In addition, there are clear chains of relationships between these components, which can be expressed in the form of directed graphs, such as sequence flows ( <bpmn:sequenceflow>) connects the activity nodes to show the execution order of the business process; while the association relationship may be through the message flow ( <bpmn:messageflow>) or other types of relationship lines.

[0137] As for sub-processes, they can be regarded as refinements and extensions of the main process, which are aggregated by a series of interrelated activity nodes (i.e. detection items). Each "activity" or "detection item" is essentially an abstraction of the task or operation unit to be executed in the process, and its detailed information includes but is not limited to:

[0138] 1. Name: used to identify and describe the core content of the activity;

[0139] 2. Type: Indicates the nature of the activity, such as user task, service task, script task, etc.

[0140] 3. Preconditions: The prerequisites for the execution of an activity, which can be defined by constraint expressions or logical judgments;

[0141] 4. Operation Steps: The specific steps for implementing the activity, which can be further divided into subtasks;

[0142] 5. Post-actions: actions that need to be performed or states that need to be changed after the activity ends.

[0143] In summary, "Output Data Structure and Standard Requirements" not only provides a strict framework for building XML files that comply with the BPMN 2.0 specification, but also plays a core role in the entire automated verification process, ensuring that the XML files generated by the multimodal vertical large model can accurately reflect the real business process, thereby effectively driving the development of process automation and intelligence.

[0144] (7)Automated Verification Module

[0145] As an important component in modern software engineering and process control, the automated verification module shoulders the important responsibility of comprehensively and accurately verifying the XML files generated by multi-modal vertical models. Its working principle is to strictly follow the pre-set output data structure and standard requirements, apply advanced formal methodologies and technical means, and conduct in-depth structural analysis and comprehensive compliance review of the XML files generated by the model.

[0146] First of all, it is very important to understand the "output data structure and standard requirements", which is the benchmark reference for the legality of XML files. The XML data structure defined in BPMN 2.0 or other industry standards includes a series of detailed requirements such as the use of element tags, the provisions of hierarchical relationships, and the legality restrictions of attribute values. For example, a qualified XML file should clearly reflect the various components in the flow chart and their relationships, including activity nodes (Task), events (Event), gateways (Gateway), data objects (Data Object), etc., and ensure that the nesting levels, link relationships, and attribute filling of these elements all conform to the preset standard structure.

[0147] In the actual verification process, the "Automated Verification Module" uses a series of formalized technical means and algorithm strategies. For example, for the overall structural consistency of XML files, XML Schema Definition (XSD) can be used for static verification, and the XML instance document can be compared with the XSD schema file to confirm whether the file follows the predefined data structure:

[0148]

[0149] To verify more complex business logic, such as the correctness of process flow, graph traversal algorithms in graph theory, such as depth-first search (DFS) or breadth-first search (BFS), may be used to ensure that each link in the flowchart can be connected and converted as expected:

[0150]

[0151] At the same time, the rule engine also plays an important role in this process. It can handle unstructured or semi-structured data constraints and business rules. By parsing the element content in the XML file, it uses a series of expressions and logical judgments to ensure that the preconditions of the activity node are met and the subsequent actions are executed as planned, such as:

[0152]

[0153] In addition, in order to cope with the complex situations that may arise from large multimodal vertical models, the automated verification module also needs to be flexible and intelligent enough to adapt to changes in the output formats of various models, and to discover potential structural anomalies or logical contradictions through deep learning and pattern recognition of XML files, thereby achieving efficient and accurate automated compliance detection.

[0154] To summarize, the "automated verification module" closely combines the output data structure and standard requirements, uses formal methodologies and a variety of technical means including algorithms and rule engines to comprehensively and deeply analyze and verify the XML files generated by multimodal vertical large models, ensuring that they are highly compliant in terms of data structure, business logic and regulatory requirements, and strongly supports high-precision and high-efficiency large-scale automated process management.

[0155] (8) Process and test item review / editing module

[0156] In the design of intelligent systems, the "Process and Test Item Review / Editing Module" plays a key role. It works closely with the "Automated Verification Module" to achieve efficient processing and optimization of XML files. The specific workflow is as follows:

[0157] First, the "Automated Verification Module" conducts a comprehensive and detailed verification of the received XML file according to the preset structural specifications, logical rules and business requirements. Only when the XML file meets all the set conditions, that is, it is deemed to be qualified for verification, will it be transferred to the "Process and Inspection Item Review / Editing Module". This link ensures the accuracy and consistency of the input data, which is the basis for the effective operation of the subsequent process.

[0158] Next, after receiving the verified XML file, the "Process and Detection Item Review / Editing Module" uses advanced data parsing technology and visualization algorithms to convert and render the abstract XML data into an intuitive and easy-to-understand graphical flowchart. This graphical presentation not only clearly reveals each step of the process and the relationship between each detection item, but also provides a friendly interface for non-technical personnel such as business personnel and experts, allowing them to examine the entire process from a macro perspective and conduct in-depth review and optimization based on actual business needs.

[0159] Experts or business personnel can efficiently adjust the process sequence, add or delete inspection items, etc. by directly operating and modifying the graphical flowchart. These changes will be synchronized back to the XML file in real time to form a new version of the XML file that has been professionally optimized by humans.

[0160] Finally, these deeply reviewed and optimized XML files will be fed back into the system's sample data set as valuable training materials for the later fine-tuning of the "multi-modal vertical model". With its strong learning ability and adaptability, this model can continuously improve its ability to understand and generate complex business processes by learning from a large amount of optimized XML sample data, thereby providing more accurate and efficient solutions in various business scenarios.

[0161] In summary, the system realizes the full-process management of XML files through the close integration of multiple links such as automated verification, graphical display, and manual review and optimization, and effectively drives the continuous iteration and optimization of the "multimodal vertical large model".

[0162] The intelligent generation system of operation flow and inspection items. As a highly integrated intelligent system, the intelligent generation system of operation flow and inspection items mainly includes three core components, each of which carries specific functions and key technical applications.

[0163] Part 1: Knowledge and Data Management Module This part is the cornerstone of the entire system, focusing on the construction and maintenance of the industry knowledge base. This module builds a solid foundation for the knowledge system by collecting, organizing and updating industry-related regulations, standards, best practices and other professional knowledge, and storing them in a structured manner. In addition, it also undertakes the task of integrating business data, including but not limited to the collection, cleaning, integration and standardization of raw data, and uses the database management system (DBMS) to achieve effective organization and management of data. At the same time, this module must also have strong data analysis and mining capabilities, conduct application analysis based on existing data, and support the data basis required for intelligent decision-making and generation processes, which can be expressed as: Knowledge and Data Management = Industry Knowledge Integration + Business Data Integration + Data Analysis Application.

[0164] Part 2: Industry Agent Training and Fine-tuning Module This part focuses on the development and optimization of artificial intelligence models and builds agent models with industry characteristics. The core of the agent is to accurately simulate the professional qualities of human experts in business process description, generate accurate operation processes and their detection items; and rely on high-quality sample data sets for training and fine-tuning. This module is not only responsible for the screening, labeling and management of sample data, but also effectively allocates computing resources to ensure the efficient operation of model training. Its main functions can be summarized as: Industry Agent Management = Model Training + Model Fine-tuning + Sample Data Management + Training Resource Scheduling.

[0165] Part 3: Process and test item visualization and system management module This module is the key window for the system to interact with users. It displays complex work processes and test items in a visual way, so that business experts and managers can intuitively understand, review and optimize them. Using graph theory and data visualization technology, the intelligently generated processes and test items are converted into easy-to-interpret charts, such as flow charts, state machines or timing diagrams. In addition, other important system management functions are also included, such as permission management, logging, version control, etc., to ensure the safe and stable operation of the system: visualization and system management = process and test item visualization + system operation and maintenance management.

[0166] In summary, the intelligent generation system of operation processes and inspection items fully integrates knowledge management, data-driven intelligent model optimization and humanized visual interactive design through these three collaborative modules, realizing full-chain intelligent services from data integration, intelligent decision-making to results display, and effectively promoting the innovation of business processes and efficiency improvement.

[0167] The specific implementation plan is as follows:

[0168] In the actual operation of this patent, energy safety-related knowledge and sample data are input to train a large language model around work safety scenarios in the industrial field, so as to obtain a multimodal vertical large model suitable for vertical fields; the multimodal vertical large model analyzes and processes the work safety operating procedures / manuals in the energy safety field and outputs XML files, and the activiti software parses the XML files and outputs the corresponding graphical work flow charts and inspection items. At the same time, activiti also supports editing and optimization of work flow charts and inspection items.

[0169] The operation flow chart mainly refers to the main operation flow chart, and the detection items refer to the specific activities or task items in the sub-process; Figure 3 , Figure 4 Taking the unloading operation procedures of the LNG storage and distribution station as an example, the main process refers to six main operation process nodes, including pre-takeover preparation, purging and replacement, tank truck pressurization, liquid inlet pipeline pre-cooling, tank truck liquid unloading and unloading completion.

[0170] Taking the main process node of tank truck pressurization as an example, the detection item refers to the specific activities in the tank truck pressurization link. For example, there are three activities for tank truck pressurization, namely tank pressurization, pressurization using BOG main pipe and self-pressurization. The name, type, restrictions, operating steps and subsequent actions of each activity are different, and it is necessary to dynamically match the optimal solution according to the site environment and tank truck type.

[0171] Finally, the job flow charts and detection items generated by the multimodal vertical large model were applied to actual job safety management and control scenarios. Through continuous feedback from experts and application practices, the availability and accuracy of the job flow charts and detection items generated by the multimodal vertical large model were optimized, and good results were achieved.

[0172] 1. Integrated processing framework: The invention designs four closely coordinated processing modules, including data preprocessing module, model training / fine-tuning module, multimodal vertical large model and automated verification module, forming a complete workflow from data collection to result verification and application.

[0173] 2. Unstructured data collection and preprocessing: Through the unstructured data collection module, the system can automatically acquire various unstructured data such as text and images, and perform data cleaning, standardization and format conversion through the data preprocessing module to provide high-quality input data for subsequent model training and analysis.

[0174] 3. Construction and hierarchical utilization of sample data sets: The sample data sets include industry knowledge base content, manually annotated positive sample data, and samples automatically generated by the model and manually reviewed, achieving efficient integration and utilization of data resources. Among them, the industry knowledge base and manually annotated data are used for model training, while the samples automatically generated by the model and manually reviewed are used for fine-tuning. This hierarchical data usage strategy helps to improve the professionalism and accuracy of the model.

[0175] 4. Multimodal vertical model training and fine-tuning: The model training module uses the industry knowledge and manually annotated samples in the sample data set to enable the language model to establish an in-depth understanding of specific fields at the underlying knowledge level. The model fine-tuning module further uses the model's self-generated data to simulate the way human experts express business processes, thereby enhancing the model's adaptability and professional accuracy.

[0176] 5. Generate BPMN2 standard XML from multimodal vertical large model: This model can not only process pre-processed data, but also generate XML files that comply with BPMN2 specifications, describing the main process, sub-process, activities (detection items) and their attributes (name, type, preconditions, operation steps, post-actions) in detail to achieve structured modeling of business processes.

[0177] 6. Output data structure and standard definition: The data structure and standard requirements of XML files that comply with the BPMN 2.0 specification are clearly proposed, providing a benchmark for automated verification and ensuring that the XML files generated by the model are standardized and readable.

[0178] 7. Automated verification module: Based on predefined standards, using formal methodologies and technical means, the XML files generated by the model are fully checked for compliance to ensure the quality and consistency of the output results, reducing the workload and potential errors of manual inspection.

[0179] 8. Process and test item review / editing module: This module converts the verified XML files into intuitive flowcharts, which are convenient for experts and business personnel to review and optimize. At the same time, the feedback data after review is included in the sample data set, forming a closed loop of model training and optimization, and continuously improving the performance and applicability of the system.

[0180] To sum up, the innovative points of the present invention are concentrated on the integrated processing architecture, effective use of unstructured data, hierarchical training strategy of sample data sets, construction and application of multimodal vertical large models, standardized XML output, automated verification technology, and human-machine collaborative process review and optimization mechanism.

[0181] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0182] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.< / bpmn:messageflow> < / bpmn:sequenceflow> < / bpmn:dataobject> < / bpmn:gateway> < / bpmn:event> < / bpmn:task>

Claims

1. A method for intelligently generating executable processes and detection items based on industry agents, characterized in that: The following steps are involved: Acquiring unstructured data, wherein the unstructured data includes text data and image data containing information related to the executable process; Preprocess the collected unstructured data and generate multimodal data that integrates text data and image data; Input the preprocessed multimodal data into the trained multimodal vertical model to generate the corresponding XML file; The generated XML file is verified to obtain a final XML file that passes the verification.

2. According to claim 1, a method for intelligently generating executable processes and detection items based on industry agents, characterized in that: Preprocess the collected unstructured data, including data cleaning, data standardization and data format conversion.

3. According to claim 1, a method for intelligently generating executable processes and detection items based on industry agents, characterized in that: The multimodal vertical model is trained through the following steps: Obtaining a training data set, wherein the training data set includes industry knowledge base data and manually labeled industry positive sample data; The industry knowledge base data is abstracted into several knowledge units, and the multimodal vertical model is learned based on the knowledge units; Convert manually labeled industry positive sample data into several business process instances F j and the corresponding standardized process description P j ,The multi-modal vertical large model enhances the understanding of the inherent rules of the business process by comparing and learning ,business process instances and the corresponding standardized process descriptions; During the model training process, the model parameters θ are continuously updated iteratively through the back propagation algorithm and the Adam optimizer to minimize the difference between the predicted process description and the standardized process description P j The loss function between .

4. According to claim 3, a method for intelligently generating executable processes and detection items based on industry agents is characterized in that: Prediction Process Description With standardized process description P j The loss function includes the following functions:

5. According to claim 3, a method for intelligently generating executable processes and detection items based on industry agents is characterized in that: It also includes fine-tuning the trained multi-modal vertical model, including: Input the original document into the multimodal vertical model to generate the corresponding model output XML file; Manually verify and optimize the original document to obtain the corresponding manually optimized XML file; By comparing the model output XML file and the manually optimized XML file, we fine-tune the learning rate, training batch size, number of training iterations, type of loss function, weight decay parameter, and type and momentum of the optimizer of the multimodal vertical large model to improve the ability of the multimodal vertical large model to simulate real business processes.

6. According to claim 1, a method for intelligently generating executable processes and detection items based on industry agents, characterized in that: The pre-processed multimodal data is input into the trained multimodal vertical model to generate the corresponding XML file, which specifically includes: Input the multimodal data into the trained multimodal vertical model to generate a corresponding XML file containing the main process definition information. The main process definition information includes the activity node Task, event Event, gateway Gateway, data object DataObject and the relationship chain between them.

7. According to claim 1, a method for intelligently generating executable processes and detection items based on industry agents, characterized in that: Verifying the generated XML file specifically includes the following steps: Validate the received XML files according to preset structural specifications, logical rules and business requirements.

8. An intelligent generation system for executable processes and detection items based on industry agents, characterized in that: It includes unstructured data acquisition module, data preprocessing module, multi-modal vertical model and automated verification module; Wherein, the unstructured data acquisition module is used to acquire unstructured data, and the unstructured data includes text data and image data; The data preprocessing module is used to preprocess the collected unstructured data and generate multimodal data fused with text data and image data; The multimodal vertical model is used to receive the preprocessed multimodal data and generate a corresponding XML file based on the multimodal data; The automatic verification module is used to verify the generated XML file.

9. The intelligent generation system of executable processes and detection items based on industry agents according to claim 8 is characterized in that: It also includes a model training module, which is used to train the multimodal vertical model using a training data set.

10. The intelligent generation system of executable processes and detection items based on industry agents according to claim 9, characterized in that: The training process of the multimodal vertical model specifically includes the following steps: Obtaining a training data set, wherein the training data set includes industry knowledge base data and manually labeled industry positive sample data; The industry knowledge base data is abstracted into several knowledge units. The multimodal vertical model learns based on the knowledge units and establishes in-depth insights into the corresponding industry field at the underlying knowledge level. Convert manually labeled industry positive sample data into several business process instances F j and the corresponding standardized process description P j ,The multi-modal vertical large model enhances the understanding of the inherent rules of the business process by comparing and learning ,business process instances and the corresponding standardized process descriptions; During the model training process, the model parameters θ are continuously updated iteratively through the back-propagation algorithm and the Adam optimizer to minimize the loss function between the predicted process description and the actual annotation.

Citation Information

Patent Citations

  • Medical data mining method and system fused with deep learning

    CN117292840A

  • Method and device for generating workflow

    CN117472358A

  • Auxiliary retrieval method fusing knowledge graph and large language model

    CN117633252A

  • Intelligent question and answer implementation method and system based on big data

    CN118211648A

  • Domain question and answer set automatic construction method and construction system thereof

    CN118245588A

Cited By

  • Automatic generation method supporting MBSE system model and design document

    CN119989699A

  • Inductive process mining optimization method and system based on multi-agent cooperation

    CN121436623A

  • Multi-agent task arrangement method and system

    CN121455636A

  • Method and system for digital transformation and integration

    EP4769264A1