Intelligent business optimization control method and system based on multi-modal large model

Abstract

The application provides an intelligent business optimization control method and system based on a multi-modal large model, which first generates a business operation intervention node set based on a dynamic correlation relationship of a whole business operation process, then collects multi-element multi-modal information resources, performs cross-modal coupling processing through a multi-modal large model to generate multiple sets of business intervention execution schemes, then dynamically adapts and adjusts the schemes according to real-time multi-modal feedback data, selects a target business intervention execution scheme, and finally embeds the target business intervention execution scheme into a business operation process to drive adjustment of operation modes of each link of the business, so that the optimization control of the business operation is realized. The application can improve the timeliness and pertinence of business intervention, and enhance the efficiency and quality of business operation.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more specifically, to an intelligent business optimization control method and system based on a multimodal large model. Background Technology

[0002] In today's complex and ever-changing business environment, efficient operation and precise control of business are key to maintaining a company's competitiveness. Traditional business optimization and control methods often have many limitations.

[0003] On the one hand, most existing business intervention mechanisms are based on fixed business rules and preset process nodes, lacking a deep understanding of the dynamic relationships between the entire business operation process and the ability to respond flexibly. During the operation of a business, there are complex and dynamically changing relationships between various links, and traditional methods are unable to capture these dynamic changes, making it impossible to determine key control nodes and the triggering conditions for linkage between nodes in a timely and accurate manner, which greatly reduces the timeliness and effectiveness of business intervention.

[0004] On the other hand, the information resources generated during business operations are rich and diverse, including text, structured data, real-time sensor data, images, voice, and external environmental data. However, traditional methods typically process these diverse and multimodal information resources separately, lacking cross-modal fusion and coupling capabilities. This makes it difficult to fully utilize the inherent connections and complementarities between these information resources, thus hindering the generation of comprehensive, accurate, and targeted business intervention execution plans. Furthermore, traditional methods lack effective dynamic adaptation mechanisms when facing dynamic changes in business states, making it difficult to optimize business intervention plans in a timely manner based on real-time feedback data, and thus failing to meet the needs of rapidly changing business operations. Summary of the Invention

[0005] In view of the aforementioned problems, and in conjunction with the first aspect of the present invention, embodiments of the present invention provide an intelligent service optimization control method based on a multimodal large model, the method comprising: Based on the dynamic correlation of the entire business operation process, a set of business operation intervention nodes is generated. The set of business operation intervention nodes includes key control nodes of each business link and the linkage triggering conditions between nodes. Collect diverse and multimodal information resources related to business operations. These diverse and multimodal information resources include text-based business rule information, structured historical business execution information, real-time business status sensor data, image-based business credential information, voice-based business command information, and multi-dimensional data of the external environment. The multimodal large model performs cross-modal coupling processing on the set of business operation intervention nodes and the multi-dimensional multimodal information resources to generate multiple sets of business intervention execution schemes, each set of business intervention execution schemes corresponding to different node triggering sequences; Based on real-time multimodal feedback data of business operations, the multiple sets of business intervention execution plans are dynamically adapted and adjusted to select the target business intervention execution plan that matches the current business status. The target business intervention execution plan is embedded into the business operation process, driving each link of the business to adjust its operation mode according to the requirements of the target business intervention execution plan, thereby completing the optimized control of business operation.

[0006] In another aspect, embodiments of the present invention also provide an intelligent business optimization control system based on a multimodal large model, including a processor and a machine-readable storage medium. The machine-readable storage medium is connected to the processor. The machine-readable storage medium is used to store programs, instructions, or code. The processor is used to run the programs, instructions, or code in the machine-readable storage medium to implement the above-described method.

[0007] Based on the above, this embodiment of the invention generates a set of business operation intervention nodes by dynamically linking the entire business operation process. This allows for precise location of key control nodes in each business stage and clarification of the triggering conditions for linkage between nodes. Then, it collects diverse and multimodal information resources related to business operation and performs cross-modal coupling processing through a large multimodal model. This fully explores the inherent connections and complementarities between different modal information. Based on real-time multimodal feedback data of business operation, it dynamically adapts and adjusts multiple sets of business intervention execution plans, selecting the target business intervention execution plan that matches the current business state. This achieves real-time matching between the business intervention plan and the actual business state, enhancing the flexibility and adaptability of business intervention. By embedding the target business intervention execution plan into the business operation process, it drives each business stage to adjust its operating mode as required, completing the optimized control of business operation, effectively improving the efficiency and quality of business operation, and reducing business risks. Attached Figure Description

[0008] Figure 1 This is a schematic diagram of the execution flow of the intelligent business optimization control method based on a multimodal large model provided in an embodiment of the present invention.

[0009] Figure 2 This is a schematic diagram of the hardware architecture of the intelligent business optimization control system based on a multimodal large model provided in an embodiment of the present invention. Detailed Implementation

[0010] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1 This is a flowchart illustrating an intelligent service optimization control method based on a multimodal large model, according to an embodiment of the present invention. The following is a detailed description of this intelligent service optimization control method based on a multimodal large model.

[0011] Step S110: Based on the dynamic correlation of the entire business operation process, generate a set of business operation intervention nodes, which includes key control nodes of each business link and the linkage triggering conditions between nodes.

[0012] In the application scenario of a multimodal AI-based intelligent enterprise management system, the entire business operation process encompasses multiple stages, including market research, product development, manufacturing, supply chain management, marketing, and sales service. These stages are interconnected in complex and dynamic ways. For example, market research results directly influence product development direction, manufacturing progress constrains supply chain management inventory strategies, and marketing effectiveness, in turn, affects the allocation of sales and service resources. To achieve intelligent optimization and control of enterprise operations, it is first necessary to generate a set of business operation intervention nodes based on these dynamic relationships. Key control nodes refer to operational points in the business process that can significantly impact the overall operational effect, while the triggering conditions for node interaction specify under what circumstances these nodes will interact, trigger sequentially, or work collaboratively.

[0013] Step S111: Analyze the entire business operation process, traverse all execution stages from start to finish, define the core functions and operational requirements of each execution stage, and form a list of business process stages.

[0014] This embodiment analyzes the entire business operation process of a manufacturing company as an example. Business operations begin with market demand analysis and end with the completion of after-sales service. The entire process includes, in sequence, market demand analysis, product design, raw material procurement, production planning, production and processing, quality inspection, warehousing and logistics, marketing, sales order processing, product delivery, and after-sales service. For each execution stage, its core functions and operational requirements are defined. For example, the core function of the market demand analysis stage is to collect and analyze market information, identify customer needs and market trends, and its operational requirement is to obtain accurate, timely, and comprehensive market data. The core function of the product design stage is to design the product's structure, performance, and appearance based on the market demand analysis results, and its operational requirement is to design a product solution that meets market demands, is competitive, and is highly manufacturable. After organizing the core functions and operational requirements of the above stages, a business process stage list is formed.

[0015] Step S112: Analyze the interaction between each execution stage and other execution stages, identify the multimodal information transmission paths and dependencies between execution stages, and construct a business stage association network.

[0016] The market demand analysis stage has a close interactive relationship with the product design stage. The market demand analysis stage outputs multimodal information such as market demand reports and customer preference data (including text-based analysis reports, chart-based trend data, and audio-based customer interview records), which is then transmitted to the product design stage as input. This constitutes a multimodal information transmission path from market demand analysis to product design. Simultaneously, the design solutions from the product design stage may also be fed back to the market demand analysis stage to verify whether the design meets market expectations, forming a two-way information transmission. In terms of dependencies, the product design stage relies on accurate demand information provided by the market demand analysis stage; without this information, product design would lose direction. The raw material procurement stage relies on the raw material demand list output by the production planning stage; only by clarifying the types, quantities, and timing of required raw materials can procurement proceed in an orderly manner. By analyzing the above-mentioned interactive relationships between each execution stage and other stages, all multimodal information transmission paths and dependencies are identified. Then, using stages as nodes and information transmission paths and dependencies as directed edges, a business stage relationship network is constructed.

[0017] Step S1121: For each execution stage, define the types of multimodal input information that it needs to receive and the types of multimodal result information that it needs to output during business operation, and form a multimodal information interaction list for each stage.

[0018] Taking the production and processing stage as an example, the multimodal input information types that this stage needs to receive during business operations include: structured production work orders output from the production planning stage (containing data such as product model, production quantity, and production time nodes); drawing-type design documents output from the product design stage (image-type information such as two-dimensional engineering drawings and three-dimensional model diagrams); raw material quality inspection reports provided from the raw material procurement stage (text reports and related image data); and equipment operation status sensor data from the equipment management system (such as real-time monitoring data such as temperature, pressure, and speed). The multimodal result information types that need to be output include: real-time progress data during the production process (structured table format); images of abnormal situations generated during the production process (such as images of product defects); and preliminary results of the quantity and quality inspection of completed products (text and numerical reports). The above definitions are applied to each execution stage, thus forming a multimodal information interaction list for each stage.

[0019] Step S1122: Based on the business process definition and data flow logs, determine the upstream generation link corresponding to the multimodal input information of each execution link, thereby determining the starting link and ending link of multimodal information transmission and forming a multimodal information transmission path record.

[0020] Based on the enterprise business process definition documents and historical data flow logs, for the structured production work order input received in the production and processing stage, tracing the data flow logs reveals that its upstream generation stage is the production planning stage. Therefore, the starting point of this information transmission is the production planning stage, and the ending point is the production and processing stage. Similarly, for the drawing-type design documents output from the product design stage, they are transmitted to the production and processing stage and the quality inspection stage. Thus, the starting point of this information transmission is the product design stage, and the ending points include the production and processing stage and the quality inspection stage. Following this method, the corresponding upstream generation stage is determined for each type of multimodal input information in each execution stage, clarifying the starting and ending points of information transmission and forming a multimodal information transmission path record.

[0021] Step S1123: Identify the receiving link of the multimodal information output by each execution link, define the downstream link of multimodal information transmission, supplement and improve the multimodal information transmission path record, and form a multimodal information transmission path diagram.

[0022] Continuing with the production planning stage as an example, the structured production orders output from this stage are not only transmitted to the production and processing stage but also to the warehousing and logistics stage, allowing them to prepare and schedule raw materials in advance. By analyzing business processes and data flows, all receiving stages of the output information from the production planning stage are identified, namely the production and processing and warehousing and logistics stages. These receiving stages are the downstream stages of the output information transmission. This downstream stage information is then added to the previous multimodal information transmission path record. This process is repeated for all execution stages. Finally, all information transmission paths are graphically displayed, using arrows to indicate the direction of information transmission and connecting the starting and ending stages to form a multimodal information transmission path diagram.

[0023] Step S1124: Analyze whether the operation of each execution stage depends on the execution results of other stages. If there is a dependency, determine the specific stage and the multimodal result type of the dependency, and form a list of stage dependencies.

[0024] Analysis of the sales order processing process reveals its reliance on customer intent data from the marketing stage and real-time inventory data from the product inventory stage. Without customer intent data from the marketing stage, the sales order processing process cannot accurately grasp customer demand and potential order sources; without real-time inventory data from the product inventory stage, it cannot determine whether customer order demand can be met. Therefore, the sales order processing process depends on the marketing and product inventory stages, relying on the following multimodal output types: customer intent text reports and related statistical charts (graphical) output from the marketing stage, and structured inventory data tables output from the product inventory stage. Similar analysis is performed on each execution stage to identify its specific dependencies and corresponding multimodal output types, forming a list of stage dependencies.

[0025] Step S1125: Refine the list of dependencies between links, distinguishing between direct dependencies and indirect dependencies. Direct dependencies refer to the fact that the operation of one link directly requires the immediate result of another link, while indirect dependencies refer to dependencies formed through multiple intermediate links.

[0026] In the list of dependencies, the production and processing stage directly depends on the production work orders from the production planning stage. This is a direct dependency because the production and processing stage requires immediate production work orders directly provided by the production planning stage to start production. However, the after-sales service stage's dependency on the product design stage is an indirect dependency. The after-sales service stage needs to understand the product's design details to answer customer technical questions or handle product malfunctions. This dependency is not direct but is formed through intermediate stages such as product manuals (output from the product design stage to the sales and service stage, and then to the after-sales service stage). Each dependency in the list is further refined and differentiated in this way to clearly identify which are direct and which are indirect dependencies.

[0027] Step S1126: Based on the multimodal information transmission path graph and the refined list of process dependencies, construct an initial framework for business process association. In the initial framework, nodes represent execution processes, and directed edges represent multimodal information transmission paths and dependencies.

[0028] Each execution stage, such as market demand analysis, product design, and raw material procurement, is represented as a node and plotted on a plane. Based on the multimodal information transmission path diagram, directed edges with arrows connect the starting and ending nodes of information transmission, with the arrow direction indicating the direction of information transmission. Simultaneously, based on the refined list of stage dependencies, direct dependencies are also represented by directed edges pointing from the dependent stage node to the dependent stage node, labeled "Direct Dependency"; indirect dependencies are similarly connected by directed edges and labeled "Indirect Dependency." This constructs the initial framework for the association of business stages, which visually demonstrates the information transmission and dependencies between each execution stage.

[0029] Step S1127: Add an attribute identifier to each directed edge to define the type of multimodal information or dependency type represented by the directed edge.

[0030] For directed edges that transmit information from the market demand analysis stage to the product design stage, add attribute labels such as "Market Demand Report (Text)," "Customer Preference Chart (Image)," and "Customer Interview Transcription (Text)" to clarify the type of multimodal information transmitted by the edge. For directed edges indicating dependencies, such as a direct dependency edge from the production and processing stage to the production planning stage, add the attribute label "Depends on Production Work Order (Structured Data)"; for an indirect dependency edge from the after-sales service stage to the product design stage, add the attribute label "Indirectly Depends on Product Design Details (Text, Image)." By adding the above attribute labels to each directed edge, the specific content represented by the edge is further clarified.

[0031] Step S1128: Analyze whether there are circular dependencies or multimodal information transmission loops in the initial framework of business process association. If so, parse the relationship between the links in the multimodal information transmission loop, define the operation logic and triggering conditions of the multimodal information transmission loop, and correct unreasonable parts in the business process association network.

[0032] In the initial framework, we check for circular dependencies or information loops, such as A depending on B, B depending on A, or A transmitting information to B, B transmitting information to C, and C transmitting information back to A. For example, we might find that the production planning stage depends on order data from the sales order processing stage, which in turn depends on capacity forecast data from the production planning stage, creating a circular dependency. In this case, we need to analyze the relationships between the stages in this loop and define its operational logic. For instance, is it first performing a preliminary capacity forecast based on historical order data, then adjusting the production plan upon receiving new orders, or are there other triggering conditions? If such a circular dependency causes the business process to fail to start or operate chaotically, it needs to be corrected. For example, we could introduce an independent capacity assessment stage to provide capacity forecast data, breaking the circular dependency and correcting unreasonable parts of the business process's interconnected network.

[0033] Step S1129: Optimize and organize the nodes and edges in the initial framework of business process association, merge nodes with duplicate functions, simplify redundant multimodal information transmission paths, and form a simplified initial business process association network.

[0034] In the initial framework, there may be some nodes with similar or mergeable functions. For example, if "Raw Material Quality Inspection" and "Semi-finished Product Quality Inspection" both belong to the quality inspection department and their functions can be integrated, then these two nodes can be merged into a single "Quality Inspection" node. For redundant information transmission paths, such as when stage A transmits the same type of information to stage B through two different paths that are essentially identical, one main path can be retained, and the redundant path deleted. Through this optimization and reorganization, the number of nodes and edges is reduced, making the business process relationship network clearer and simpler, resulting in a streamlined initial business process relationship network.

[0035] Step S11210: Verify the completeness and accuracy of the simplified initial business process association network, check whether all execution processes are included in the business process association network, whether all multimodal information transmission paths and dependencies are accurately expressed, correct any missing or incorrect content, and form the business process association network.

[0036] A comprehensive review of the streamlined initial business process network was conducted to verify that all execution stages, such as market demand analysis and product design, were included as nodes in the network, and that no stages were missing. Simultaneously, it was checked whether each multimodal information transmission path accurately reflected the information flow between stages, whether dependency labels were correct, and whether the attribute identifiers of directed edges were accurate. If any execution stage was found to be missing, it should be added to the network promptly; if the direction of an information transmission path was reversed or dependency labels were incorrect, they should be corrected. Through this verification and correction process, an accurate and complete business process network was ultimately formed.

[0037] Step S113: Based on the business process association network, extract the operation points that meet the predefined key criteria in each execution process, and use the extracted operation points as candidate initial control nodes.

[0038] Predefined critical criteria include the degree of impact of operation points on business processes, the irreplaceability of operation points, and the degree of harm to the overall business when an operation point malfunctions. Taking the production and processing stage as an example, in this stage, the start-up and shutdown of production equipment, the setting and adjustment of key process parameters, and the monitoring and adjustment of production progress directly affect production efficiency and product quality, thus meeting the predefined critical criteria. By analyzing each execution stage in the business process association network, these standard-compliant operation points are identified and extracted as candidate initial control nodes.

[0039] Step S114: Based on predefined business indicators, sort the candidate initial control nodes by function priority and select core control nodes. The predefined business indicators include the number of downstream links affected by the node and the degree of interruption of the overall process due to node failure.

[0040] Among the predefined business metrics, the more downstream links a node affects, the wider its influence within the business process; the higher the impact of a node failure on the overall process, the greater its importance. For example, the candidate initial control node "Production Work Order Generation" in the production planning stage affects multiple downstream links such as production processing, raw material procurement, and warehousing logistics. If this node fails, it will prevent many subsequent links from starting normally, resulting in a high degree of disruption. In contrast, the candidate initial control node "Daily Equipment Cleaning" in the production processing stage affects relatively fewer downstream links; its failure mainly affects the operation of equipment within this stage, resulting in a lower degree of disruption to the overall process. By quantifying and scoring each candidate initial control node according to the aforementioned business metrics, and then prioritizing them based on the scoring results, the nodes with the highest ranking and priority are selected as core control nodes.

[0041] Step S115: For each core control node, analyze its activation conditions in business operation, define the prerequisite business states and multimodal external input information required for node startup, and form a node activation condition description.

[0042] Taking the core control node of "Production Work Order Generation" as an example, we analyze its activation conditions. For this node to start, the preceding business states include: the sales order processing stage has completed order aggregation; the product inventory stage has provided real-time inventory data and the inventory is insufficient to meet order demand; and the market demand analysis stage has confirmed that current market demand is stable. Multimodal external input information includes the structured order summary table (text and numerical types) output from the sales order processing stage, the inventory status report (text and chart types) output from the product inventory stage, and the demand stability assessment report (text type) output from the market demand analysis stage. By defining the above-mentioned preceding business states and multimodal external input information, we clarify under what circumstances the "Production Work Order Generation" node can be activated, thus forming the activation condition description for this node.

[0043] Step S1151: Trace the position of each core control node in the business process, define the business stage and related links before and after the core control node, and determine the business scenario background of the core control node's operation.

[0044] The core control node for "production work order generation" is located in the production planning stage, which is the business phase after product development is completed and before production begins. Its preceding related stages include sales order processing, product inventory management, and market demand analysis; its subsequent related stages are production processing and raw material procurement. The business scenario for this node is that the company receives a certain number of sales orders, and after analyzing inventory and market demand, it needs to formulate a specific production plan to meet the order demand.

[0045] Step S1152: Analyze the core functions of the core control node, define the business operations that need to be completed and the expected business effects after the core control node is started, and deduce the functional requirements for activating the core control node based on this.

[0046] The core function of the "Production Work Order Generation" node is to formulate detailed production plans based on sales orders, inventory status, and market demand, generating production work orders that include information such as product model, production quantity, production batch, completion time, and required raw materials. After activation, the required business operations include data collection (collecting order, inventory, and demand data), data processing and analysis (analyzing order priority, inventory gaps, and production capacity), and work order creation (creating specific production work orders based on the analysis results). The expected business outcome is to ensure that production work orders are accurate and reasonable, guiding the orderly progress of production and processing, and meeting the delivery requirements of sales orders. Based on these core functions and expected outcomes, the functional requirement for its activation is to obtain accurate and complete order, inventory, and market demand data, and to have the ability to effectively analyze this data and create work orders.

[0047] Step S1153: Based on the functional requirements of the core control node, determine the internal business state required for its startup. The internal business state includes the execution results of the preceding steps, the current business progress status, and the reserve status of relevant resources.

[0048] Based on the functional requirements of the "Production Work Order Generation" node, the internal business states required for its activation include: the upstream sales order processing stage has completed the aggregation and confirmation of all pending orders, outputting accurate order data; the product inventory stage has completed the latest inventory count, providing real-time and accurate inventory data, including the current inventory quantity and in-transit inventory of each product; the current business progress status is in the production planning stage, that is, the product design has been completed and the technical data required for production has been prepared; in terms of the reserve status of relevant resources, the production equipment is in normal and usable condition, the production personnel have been arranged in place, and the basic resource conditions for starting production are available.

[0049] Step S1154: Determine the multimodal external input information relied upon during the startup process of the core control node. The multimodal external input information includes text-based instruction information, image-based verification credentials, voice-based authorization instructions, and structured environmental data.

[0050] The multimodal external input information relied upon during the “Production Work Order Generation” node startup process includes: a text-based order summary report output from the sales order processing stage, which details the product information, quantity, and delivery date of customer orders; a graphical inventory report provided by the product inventory stage, which displays the inventory change trend and current inventory level of each product in chart form; a voice-based authorization command input by the production department manager to authorize the start of the production work order generation process; and structured production environment data, such as structured data tables of current production line equipment operating parameters and production personnel shift schedules.

[0051] Step S1155: Provide a detailed description of the identified preceding business status, and define the specific manifestation and judgment criteria of each preceding business status.

[0052] For the preceding business status of "Order summary completed in the sales order processing stage," the specific manifestation is that the sales order processing system generates a "Order summary completed" status indicator, and all pending orders in the system are in the "Confirmed" status. The criteria for judgment are: an order summary report has been generated, the order quantity in the report matches the actual order quantity in the system, and all order information (product model, quantity, customer information, delivery date, etc.) is complete and accurate. For the preceding business status of "Real-time inventory data provided in the product inventory stage," the specific manifestation is that the inventory management system updates the inventory data in real time, and the last inventory update time is no more than a preset threshold (e.g., 2 hours) from the current time. The criteria for judgment are: the quantity, location, and other information of each product in the inventory data are accurate, and the error between the data and the actual inventory count results is within the allowable range (e.g., error rate less than 1%).

[0053] Step S1156: Classify the extracted multimodal external input information and define the source, format requirements and reception time window for each type of multimodal external input information.

[0054] The multimodal external input information for the "Production Work Order Generation" node is categorized as follows: Text-based order summary reports, sourced from the sales order processing system, in PDF or Word format, including fields such as order number, customer name, product details, quantity, and delivery date, received within 24 hours before the production planning stage begins; Image-based inventory reports, sourced from the product inventory management system, in JPG or PNG format, including product name, inventory quantity, and inventory warning line, received within 12 hours before the production planning stage begins; Voice-based authorization instructions, sourced from the production department manager, in clear audio recordings (e.g., MP3 format), containing explicit authorization instructions and the authorizing person's identity information, received within 1 hour before the production work order generation node begins; Structured production environment data, sourced from the production management system and human resource management system, in Excel spreadsheets or database tables, including fields such as equipment number, operating status, personnel name, and shift schedule, received within 6 hours before the production work order generation node begins.

[0055] Step S1157: Analyze the logical relationship between the preceding business states, determine whether the preceding business states are satisfied simultaneously or selectively, and define the combination conditions between the preceding business states.

[0056] The prerequisite business states for the "Production Work Order Generation" node include: sales order processing has completed order aggregation; product inventory has provided real-time inventory data; the current business progress is in the stage awaiting production planning; and relevant resource reserves meet production requirements. These prerequisite business states are logically related to "simultaneous fulfillment," meaning all prerequisite business states must be met for the node to be eligible to start. The combined conditions are: (sales order processing completed) AND (real-time inventory data provided) AND (business progress in the stage awaiting planning) AND (resource reserves meet requirements). Only when all four conditions are met simultaneously is the node's prerequisite business state considered satisfactory.

[0057] Step S1158: Analyze the correlation between multimodal external input information, determine the priority and complementarity of different multimodal external input information, and define the order of receiving multimodal external input information and the alternative conditions.

[0058] In the multimodal external input information of the "Production Order Generation" node, the text-based order summary report is the core input information with the highest priority because it directly determines the product type and quantity of the production order. Structured production environment data has the next highest priority, as it affects the production scheduling of the production order. Image-based inventory reports and voice-based authorization instructions have relatively lower priority but are equally indispensable. They are complementary; for example, image-based inventory reports can intuitively aid in understanding inventory status, supplementing the shortcomings of text-based data; voice-based authorization instructions are the necessary authorization to start the process. The order of receipt is as follows: first, receive the text-based order summary report and image-based inventory report, based on which inventory gaps and production needs are analyzed; then, receive the structured production environment data to formulate specific production scheduling plans; finally, receive the voice-based authorization instructions to complete the final authorization. Regarding alternatives, if the voice-based authorization instructions cannot be received on time, in emergency situations, the production department manager can submit a text-based authorization instruction through the system as a substitute, but this must be accompanied by the manager's digital signature.

[0059] Step S1159: Integrate the combined conditions of the preceding service status with the requirements for receiving multimodal external input information to form a complete condition rule for node activation. The complete condition rule for node activation defines all the necessary conditions for the core control node to start.

[0060] The integrated activation conditions for the "Production Work Order Generation" node are as follows: When the following prerequisite business status combination conditions are met (the sales order processing stage has completed order summarization, the product inventory stage has provided real-time inventory data, the current business progress is in the production planning stage, and the relevant resource reserve status meets production requirements), and a text-based order summary report, image-based inventory report, structured production environment data, and voice-based authorization instructions (or text-based authorization instructions that meet the alternative conditions) that meet the format requirements are received within the specified receiving time window, the activation conditions for the "Production Work Order Generation" core control node are met.

[0061] Step S11510: Based on the complete condition rules for node activation, generate a node activation condition description. The node activation condition description clearly lists the various pre-service states, multimodal external input information, related combination logic, and receiving requirements required for the activation of the core control node.

[0062] The activation conditions for the "Production Work Order Generation" node are as follows: Activation of this node requires meeting the following prerequisite business conditions: 1. The sales order processing stage has completed the summary and confirmation of all pending orders, and the order summary report has been generated with complete and accurate information; 2. The product inventory stage has completed the latest inventory count, and the provided real-time inventory data error rate is less than 1%, with the last update time no more than 2 hours from the present; 3. The current business progress is in the production planning stage, product design has been completed, and technical documents are ready; 4. Production equipment is normal and available, production personnel have been assigned, and resource reserves meet production requirements. Simultaneously, the following multimodal external input information must be received within the specified time window: 1. Text-based order summary reports output by the sales order processing system (PDF or Word format, received within 24 hours); 2. Image-based inventory reports provided by the product inventory management system (JPG or PNG format charts, received within 12 hours); 3. Structured production environment data provided by the production management system and human resource management system (Excel spreadsheets or database tables, received within 6 hours); 4. Voice-based authorization instructions from the production department manager (MP3 format, received within 1 hour; in emergencies, text-based authorization instructions with digital signatures can be used instead). All prerequisite business states must be met simultaneously, and multimodal external input information must be received in accordance with regulations before the node can be activated.

[0063] Step S116: Analyze the interaction between different core control nodes, determine the triggering or inhibiting effect of a core control node on other core control nodes after its activation, and establish the linkage logic between core control nodes.

[0064] Taking the two core control nodes, "Production Work Order Generation" and "Raw Material Procurement Planning," as examples, this paper analyzes their interaction. When the "Production Work Order Generation" node is initiated and generates a production work order, it outputs work order information containing the types and quantities of raw materials required. This information triggers the "Raw Material Procurement Planning" node to start, as raw material procurement requires determining the types and quantities to be procured based on the production work order—this is a triggering effect. Conversely, if the "Raw Material Procurement Planning" node cannot complete its procurement plan on time due to factors such as tight raw material market supply, resulting in untimely raw material supply, it will have an inhibitory effect on the "Production Work Order Generation" node, potentially requiring adjustments to the production plan or delaying its start. By analyzing the above interaction relationships among all core control nodes, it clarifies which nodes exhibit triggering effects and which exhibit inhibitory effects, thereby establishing the linkage logic between the core control nodes.

[0065] Step S117: Based on the node activation condition description and the linkage logic between the core control nodes, formulate the linkage triggering conditions between the core control nodes. The linkage triggering conditions between the core control nodes define the order of triggering of the core control nodes and the associated constraints.

[0066] Based on the activation conditions of the "Production Work Order Generation" node and its linkage logic with the "Raw Material Procurement Planning" node, the linkage trigger conditions between them are defined. For example, the "Raw Material Procurement Planning" node must be triggered only after the "Production Work Order Generation" node is started and outputs a valid production work order; this is the order of triggering. Simultaneously, the activation of the "Raw Material Procurement Planning" node also requires meeting its own activation conditions, such as receiving raw material demand information output by the "Production Work Order Generation" node and having resources available from the procurement department; this is a correlation constraint. For nodes with a suppressive effect, such as the "Raw Material Procurement Planning" node suppressing the "Production Work Order Generation" node, the linkage trigger condition can be defined as follows: if the "Raw Material Procurement Planning" node fails to complete the procurement plan and cannot resolve the raw material supply problem within a preset time, it sends a suppression signal to the "Production Work Order Generation" node. Upon receiving this signal, the "Production Work Order Generation" node decides to suspend or adjust the production work order based on the degree of suppression. Through the above methods, clear linkage trigger conditions are defined for all core control nodes.

[0067] Step S118: Structure and integrate all core control nodes according to the business process sequence and linkage relationship to form a node list containing node identifiers, function descriptions, and activation conditions.

[0068] Following the sequence of the enterprise's business processes, such as core control nodes related to market demand analysis, product design, production planning, manufacturing, and sales and service, all core control nodes are ranked. Simultaneously, considering the interrelationships between nodes, directly related nodes are grouped together. Then, a unique node identifier is assigned to each core control node, such as "PP-001" representing the first core control node related to production planning. A detailed description of each node's function is provided, such as "PP-001: Production work order generation, creating production planning work orders based on sales orders and inventory data," along with a summary of the node's activation conditions. This information is then compiled into a table to form a node list.

[0069] Step S119: Associate and bind the node list with the linkage triggering conditions between the core control nodes to build the basic framework of the business operation intervention node set.

[0070] Each core control node in the node list is associated with the previously defined trigger conditions for linkage between core control nodes. For example, under the entry for the "Production Work Order Generation (PP-001)" node in the node list, a linkage trigger condition for its linkage with the "Raw Material Procurement Plan Formulation (PP-002)" node is added: "PP-001 starts and outputs a production work order, triggering PP-002 to start"; as well as linkage conditions with other related nodes. Through the above association and binding, the node list not only contains information about individual nodes, but also includes the interaction rules between nodes, thereby constructing the basic framework of the set of business operation intervention nodes.

[0071] Step S1110: Perform a logical integrity analysis on the basic framework of the business operation intervention node set, check for conflicts in the linkage relationship and triggering conditions of each core control node, correct the conflicting content, and form a business operation intervention node set.

[0072] A comprehensive logical check is performed on the basic framework of the business operation intervention node set. For example, it checks for cyclical triggering situations where node A triggers node B, and node B in turn triggers node A; or situations where the activation condition of a node contradicts its associated triggering condition. Suppose it is found that the activation condition of the "Production Work Order Generation (PP-001)" node requires raw material inventory to meet certain conditions, while its associated triggering condition for the "Raw Material Procurement Planning (PP-002)" node is insufficient raw material inventory. There is no conflict between these two. However, if it is found that the "Production Work Order Generation (PP-001)" node requires to start before time T, while its associated triggering node "Raw Material Procurement Planning (PP-002)" node can only be completed after time T, causing the production work order to fail to execute on time, then a time conflict exists. In this case, the associated triggering condition needs to be corrected, such as adjusting the start time of the "Raw Material Procurement Planning (PP-002)" node or optimizing its process to resolve the conflict. After the above logical integrity analysis and conflict correction, the final set of business operation intervention nodes is formed.

[0073] Step S120: Collect multi-dimensional and multi-modal information resources related to business operation. The multi-dimensional and multi-modal information resources include text-based business rule information, structured historical business execution information, real-time business status sensor data, image-based business credential information, voice-based business instruction information, and multi-dimensional data of the external environment.

[0074] In a multimodal AI-based intelligent enterprise management system, various types of information resources need to be collected to comprehensively understand the enterprise's business operations. These include: text-based business rule information (such as internal regulations, business operation process descriptions, and contract terms); structured historical business execution information (including sales order data, production output data, and financial income and expenditure data stored in a database); real-time business status sensor data (such as real-time monitoring data of production equipment temperature, pressure, and speed, collected in real-time by sensors); image-based business voucher information (such as product quality inspection reports, raw material receiving and acceptance forms, and customer signed receipts); voice-based business instruction information (such as management voice instructions, customer voice orders, and customer service voice recordings); and multi-dimensional external environmental data (including market data, competitor activities, policy and regulatory changes, and macroeconomic indicators that affect the enterprise's business). These data are collected through various methods, such as database queries, sensor access, image scanning, voice recording, and web crawling.

[0075] Step S130: Perform cross-modal coupling processing on the set of business operation intervention nodes and the multi-modal information resources through a multimodal large model to generate multiple sets of business intervention execution schemes, each set of business intervention execution schemes corresponding to different node triggering sequences.

[0076] The multimodal big data model possesses the capability to process and fuse information from multiple modalities. First, core control node information and inter-node linkage triggering conditions from the set of business operation intervention nodes are input into the multimodal big data model. Then, collected multi-modal information resources, including text-based business rules, structured historical data, real-time sensor data, image credentials, voice commands, and external environmental data, are also input into the multimodal big data model. The multimodal big data model performs deep coupling processing on the aforementioned cross-modal information, including information alignment, correlation, and fusion operations, to understand the intrinsic connections between different modalities and their impact on business intervention nodes. Based on the processed information, the multimodal big data model generates multiple different combinations of node triggering sequences according to the nodes and linkage conditions in the set of business operation intervention nodes. For each combination of node triggering sequences, combined with business rules, historical data, and real-time status from the multi-modal information resources, specific execution steps, resource allocation plans, and time arrangements are formulated, forming multiple business intervention execution plans, each corresponding to a specific node triggering sequence.

[0077] Step S131: Perform a structured transformation on the textual business rule information in the multi-modal information resources, transforming the unstructured rule description into a standardized rule expression that can be recognized by the multi-modal large model, thus forming standardized business rules.

[0078] Much of the text-based business rule information is unstructured, such as regulations in Word documents or operational procedures in PDF format. This text may contain complex sentence structures, technical jargon, and inconsistent expressions. It is necessary to perform a structured transformation on this unstructured text. For example, the rule description "When the product qualification rate is below 95%, the production department should immediately stop production and conduct equipment inspection" can be broken down into the condition "product qualification rate < 95%" and the action "the production department stops production and conducts equipment inspection," and expressed using standardized logical expressions and action instructions. Simultaneously, key entities (such as "product qualification rate" and "production department") and attributes (such as "95%)" in the rule are extracted and labeled to establish a structured representation framework for the rule. Through this transformation, unstructured text-based business rule information is converted into standardized rule expressions that can be understood and processed by a multimodal large-scale model, forming standardized business rules.

[0079] Step S132: Extract key operational data from the structured historical business execution information in the multi-modal information resources, analyze the triggering effect and operational rules of business intervention nodes under different historical scenarios, and form a historical intervention effect dataset.

[0080] From a database of structured historical business execution information, key operational data related to business intervention nodes are extracted, such as the trigger time, trigger frequency, business status data at the time of triggering, and changes in business indicators after triggering (e.g., percentage increase in production efficiency, amount of cost reduction, changes in customer satisfaction, etc.) of each core control node in different periods. Then, the data is categorized according to different historical scenarios, such as by season (peak season, off-season), by market demand (high demand, low demand), and by product type (new product promotion, mature product maintenance, etc.). For each historical scenario, the triggering effect of business intervention nodes is analyzed, such as which nodes significantly improve business performance and which have little or no effect or even a negative impact. The operational patterns of business intervention nodes are summarized, such as which nodes tend to trigger first in a certain scenario and the linkage patterns between nodes. The above analysis results and related key operational data are compiled into a dataset, forming a historical intervention effect dataset.

[0081] Step S133: Analyze the real-time service status sensing data in the multi-modal information resources, capture the operating parameters and execution progress of each link of the current service, and generate a real-time service status description.

[0082] Real-time business status sensor data comes from sensors distributed across various business processes within the enterprise, such as equipment sensors in the production workshop, temperature and humidity sensors in the warehouse, and GPS positioning sensors on logistics vehicles. The analysis of this real-time sensor data begins with data cleaning and preprocessing to remove noise and outliers. Then, the data is categorized according to business processes, such as equipment operating parameters (temperature, pressure, speed, current, etc.) in production, inventory quantities and storage environment parameters in warehousing, and transportation location and speed in logistics. Next, data fusion technology integrates multiple sensor data points from the same process to comprehensively capture its operational status. Simultaneously, the execution progress of each process is analyzed in conjunction with time nodes in the business process, such as the percentage of planned output achieved in production and the progress of transportation tasks in logistics. Finally, the above operating parameters and execution progress information are integrated in the form of natural language descriptions, structured tables, and key indicator values ​​to generate a real-time business status description that clearly reflects the actual operational status of each business process.

[0083] Step S134: Extract features from the image-type business voucher information in the multi-modal information resources, convert it into a standardized image feature vector, and form an image voucher feature set.

[0084] Image-based business voucher information includes scanned or electronic images of various paper vouchers, such as purchase invoices, product quality inspection reports, and warehouse receipts. Feature extraction is performed on these images. First, image preprocessing is conducted, including grayscale conversion, noise reduction, and edge detection to enhance image quality and feature saliency. Then, image feature extraction algorithms such as convolutional neural networks are used to extract deeper visual features from the preprocessed images, such as texture, shape, and color features. The extracted features are then quantized and standardized, transforming them into fixed-length vectors, i.e., standardized image feature vectors. For example, for a product quality inspection report image, a feature vector containing 512 elements is extracted after processing, with each element representing a specific image feature. The standardized image feature vectors corresponding to all image-based business voucher information are then aggregated to form an image voucher feature set.

[0085] Step S135: Perform speech-to-text and semantic parsing on the voice-type service command information in the multi-modal information resources to generate a structured voice command description and form a voice command dataset.

[0086] Voice-based business instructions include voice commands from managers, voice requests from customers, and voice recordings from meetings. First, speech recognition technology is used to convert the speech information into text, i.e., speech-to-text. During the conversion process, the speech signal is preprocessed (e.g., noise reduction, endpoint detection), and then the speech features are converted into corresponding text sequences using acoustic and language models. After obtaining the text, semantic analysis is performed using natural language processing techniques such as word segmentation, part-of-speech tagging, named entity recognition, syntactic analysis, and semantic role labeling to extract key information from the voice instructions, such as the issuer, receiver, action, object, time requirement, and constraints. This key information is then organized according to a pre-defined structured format, such as "Instruction issuer: Production Manager; Action: Adjust production plan; Object: Product A; Time requirement: Before the end of the workday; Constraint: Ensure sufficient supply of component B." All structured voice instruction descriptions are then integrated to form a voice instruction dataset.

[0087] Step S136: Identify the key influencing factors in the multidimensional data of the external environment in the multi-dimensional information resources, analyze the way and path of the key influencing factors in the identified multidimensional data of the external environment on business operations, and form a description of the impact of the external environment.

[0088] External environmental data encompasses multiple dimensions, including market demand data, competitor data, raw material price data, policy and regulatory data, and macroeconomic data. Key influencing factors are identified from this data, such as "demand growth rate" and "changes in consumer preferences" in market demand data; "competitor new product launches" and "pricing adjustment strategies" in competitor data; "fluctuations in major raw material prices" in raw material price data; and "tightening environmental policies" and "tax policy adjustments" in policy and regulatory data. Then, the impact of these key influencing factors on the company's business operations is analyzed. For example, an "increased demand growth rate" directly promotes increased orders in the sales process, thereby driving capacity expansion in the production process; "increased fluctuations in major raw material prices" affect cost control and inventory strategies in the procurement process. Simultaneously, the impact paths are traced, for example, "tightening environmental policies" → "requires increased investment in environmental protection equipment in the production process" → "increased production costs" → "product pricing adjustments" → "changes in market competitiveness in the sales process." The key influencing factors, their mechanisms of action, and their impact paths are then organized into a written description to form a description of the external environmental impact.

[0089] Step S137: Input the standardized business rules, the historical intervention effect dataset, the real-time business status description, the image credential feature set, the voice command dataset, and the external environment impact description into the multimodal information fusion module of the multimodal large model; In the multimodal information fusion module, the feature representations corresponding to the various types of input data are aligned, and the information of different modalities is mapped to a unified semantic or feature space. Then, cross-modal information association processing is performed to generate a multimodal fused information body.

[0090] The processed standardized business rules, historical intervention effect datasets, real-time business status descriptions, image credential feature sets, voice command datasets, and external environmental impact descriptions are input into the multimodal information fusion module of the multimodal large model. This module first performs feature representation on each type of data. For example, it converts text-based standardized business rules and real-time business status descriptions into text feature vectors, structured historical intervention effect datasets and voice command datasets into structured data feature vectors, and uses image credential feature sets as image feature vectors. Then, alignment processing is performed through temporal alignment (matching different modalities of data along the time dimension), spatial alignment (matching spatial locations for data with spatial information, such as images), and semantic alignment (mapping different modalities of data to similar semantic spaces through semantic similarity calculations). This ensures that the feature representations of different modalities are comparable within a unified semantic or feature space. Next, cross-modal information association processing is performed. Attention mechanisms, graph neural networks, and other methods are used to uncover potential correlations between different modalities. For example, the correlation between "product quality issues" in the text description and the features of a quality inspection report image in the image voucher feature set; the correlation between "production work order adjustment" in historical intervention effect data and "production progress lag" in the real-time business status description. Through these processes, information from multiple modalities is organically integrated to generate a multimodal fusion information body containing comprehensive features of multimodal information.

[0091] Step S1371: Perform word segmentation on the standardized business rules, extract the core keywords and constraint statements in the rules, and form a set of rule keywords.

[0092] Taking a standardized business rule, "When the cost of raw material procurement exceeds 10% of the budget, the procurement department should re-evaluate the supplier and submit alternative solutions," as an example, we perform word segmentation, breaking it down into words such as "when," "raw materials," "procurement," "cost," "exceeds," "budget," "of," "10%," "when," "procurement department," "should," "re-evaluate," "supplier," "and," "submit," and "alternative solutions." Then, we extract core keywords from these words, such as "raw materials," "procurement cost," "budget," "10%," "procurement department," "evaluate supplier," and "alternative solutions." Simultaneously, we extract the constraint statement "exceeds 10% of the budget." After processing all standardized business rules in the above way, we summarize the extracted core keywords and constraint statements to form a rule keyword set.

[0093] Step S1372: Extract features from the historical intervention effect dataset, identify key influencing factors and effect representation data of business interventions in different historical scenarios, and form a historical effect feature set.

[0094] From historical intervention effect datasets, we analyze changes in business metrics before and after the triggering of business intervention nodes for different historical scenarios (such as peak season production scenarios and new product promotion scenarios). For example, in the peak season production scenario, we extract "production work order adjustment frequency," "raw material procurement lead time," and "equipment utilization rate" as key influencing factors; and "percentage increase in on-time order delivery rate," "production efficiency improvement," and "unit product cost reduction" as effect characterization data. Through statistical analysis and correlation analysis, we determine the relationship between these key influencing factors and effect characterization data, and represent them in the form of feature vectors or structured descriptions. We then summarize the key influencing factors and effect characterization data from all historical scenarios to form a historical effect feature set.

[0095] Step S1373: Perform structured parsing on the real-time service status description, transforming the descriptive information into standardized status parameters and numerical representations to form a set of real-time status parameters.

[0096] Real-time business status descriptions may include descriptive information such as "Production line A is currently operating normally, output has reached 80% of the planned target, equipment temperature is within the normal range, and raw material inventory is sufficient." This information is then structured and parsed to extract status parameters, such as "Production line identifier: A," "Operating status: Normal," "Output completion rate: 80%," "Equipment temperature: [specific value range]," and "Raw material inventory status: Sufficient." Qualitative descriptions such as "Operating status" and "Raw material inventory status" are converted into standardized numerical codes, such as "Normal" corresponding to 1, "Abnormal" corresponding to 0; "Sufficient" corresponding to 1, "Tight" corresponding to 0.5, and "Shortage" corresponding to 0. These standardized status parameters and numerical representations are then organized into structured data tables or feature vectors to form a real-time status parameter set.

[0097] Step S1374: Perform dimensional normalization processing on the image voucher feature set to form a standardized image feature set.

[0098] Each image feature vector in the image voucher feature set may have different dimensions or numerical ranges; for example, some feature vectors may have a length of 256, while others may have a length of 512, or the value ranges of each feature may vary significantly. To facilitate the fusion of multimodal information, the image voucher feature set needs to undergo dimensionality normalization. For feature vectors with different dimensions, feature mapping or dimensionality reduction / upgrading methods are used to unify them to the same dimension, such as 512 dimensions. For feature values ​​with different numerical ranges, methods such as min-max normalization or z-score standardization are used to map the feature values ​​to the range [0,1] or a range with a mean of 0 and a standard deviation of 1. After the above processing, the resulting image feature vectors have a unified dimension and numerical range, forming a standardized image feature set.

[0099] Step S1375: Extract semantic features from the voice command dataset and convert it into semantic vectors that can be processed by a multimodal large model to form a voice command semantic set.

[0100] The structured voice command descriptions in the voice command dataset contain key information about the command. For each structured voice command description, semantic features are extracted using a pre-trained language model (such as BERT). The command description text is input into the language model, and a fixed-length semantic vector is obtained through the model's last layer output or a specific pooling operation (such as average pooling or max pooling). This vector represents the semantic meaning of the voice command. For example, the command "The production department should immediately increase the production quantity of product B" yields a 768-dimensional semantic vector after processing. The semantic vectors corresponding to all voice commands are then aggregated to form a voice command semantic set.

[0101] Step S1376: Extract elements from the description of external environmental impacts, define the types and specific contents of environmental elements that affect business operations, and form a set of environmental impact elements.

[0102] The description of external environmental impacts includes various external environmental information that affects business operations. Key environmental element types are extracted, such as "market demand," "competitors," "raw material prices," "policies and regulations," and "macroeconomic factors." For each element type, specific content is further defined. For example, the specific content of the "market demand" element might include "total demand," "demand growth rate," and "changes in demand structure"; the specific content of the "raw material prices" element might include "major raw material price indices," "price fluctuation range," and "price trend forecasts." These environmental element types and their corresponding specific content are organized into a structured list or feature representation, forming a set of environmental impact elements.

[0103] Step S1377: Input the set of rule keywords, the set of historical effect features, the set of real-time state parameters, the set of standardized image features, the set of voice command semantics, and the set of environmental impact factors into the information preprocessing module of the multimodal large model to perform multimodal data format unification processing, so that the expression form of different modal information remains consistent.

[0104] The following data are input into the information preprocessing module of the multimodal large model: a set of rule keywords (text keywords), a set of historical effect features (structured feature vectors / descriptions), a set of real-time state parameters (standardized state parameters and values), a set of standardized image features (normalized image feature vectors), a set of speech command semantics (semantic vectors), and a set of environmental impact factors (structured environmental factors). This module standardizes the format of the data from these different modalities. For example, it converts all textual information (text descriptions in the set of rule keywords and environmental impact factors) into word embedding vectors; it converts structured data (partial data from the set of historical effect features and the set of real-time state parameters) into numerical feature vectors; and it maintains the vector form of image feature vectors and speech semantic vectors, ensuring that all vectors have a unified dimension (e.g., 512 or 768 dimensions). Through this processing, information from different modalities is represented in a unified feature vector form, preparing for subsequent cross-modal correlation analysis.

[0105] Step S1378: Using a multimodal large model cross-modal association analysis algorithm, mine the matching relationship between the rule keyword set, the real-time state parameter set, the standardized image feature set, and the voice command semantic set, identify the business rule clauses that the current real-time state conforms to, and form a rule matching result.

[0106] The multimodal large-scale model cross-modal association analysis algorithm utilizes deep learning technology to calculate the similarity between keyword vectors in the rule keyword set and state parameter vectors in the real-time state parameter set, image feature vectors in the standardized image feature set, and semantic vectors in the voice command semantic set. For example, the similarity calculation is performed between the keyword vector of "raw material procurement cost" in the rule keyword set and the parameter vector of "current value of raw material procurement cost" in the real-time state parameter set. If the similarity is higher than a preset threshold, the current real-time state is considered to be related to the rule keyword. Further, considering the constraints in the rules, it is determined whether the current real-time state meets the triggering conditions of the rule. For example, for the rule "raw material procurement cost exceeds 10% of the budget," the "exceeding 10%" condition is determined by comparing the "raw material procurement cost" and the "budget value" in the real-time state parameters. All business rule clauses that meet the conditions are filtered out to form the rule matching results.

[0107] Step S1379: Analyze the similarity between the historical effect feature set, the real-time state parameter set, and the standardized image feature set, identify historical scenarios similar to the current business state and their corresponding intervention effect features, and form historical scenario matching results.

[0108] The comprehensive similarity is calculated between the feature vectors of each historical scene in the historical effect feature set and the feature vectors of the current real-time state parameter set and the standardized image feature set. The comprehensive similarity can be calculated using a weighted summation method; for example, the similarity weight between the historical scene feature vector and the real-time state parameter feature vector is set to 0.7, and the similarity weight with the standardized image feature vector is set to 0.3. A similarity threshold is set, and historical scenes with similarity higher than this threshold are filtered out, considered similar to the current business state. Simultaneously, the intervention effect features corresponding to these similar historical scenes are extracted, such as which business intervention nodes showed significant effects after being triggered in that scene, and by how much business metrics improved. The above similar historical scenes and their corresponding intervention effect features are then organized to form the historical scene matching results.

[0109] Step S13710: Calculate the correlation between the set of environmental impact factors, the set of rule keywords, the set of real-time status parameters, and the set of standardized image features, identify the environmental factors that significantly affect the execution of business rules and the current business status and their influence weights, and form an environmental correlation result.

[0110] For each environmental element in the set of environmental impact factors, calculate its semantic relevance with each keyword in the set of rule keywords, its correlation with each parameter in the set of real-time state parameters, and its correlation with each image feature in the set of standardized image features. Semantic relevance can be calculated using word vector similarity, and correlation can be calculated using methods such as Pearson correlation coefficient. Combine these correlation values ​​to obtain the overall correlation between each environmental element and the execution of business rules and the current business state. Set a correlation threshold, and identify environmental elements with an overall correlation higher than this threshold as significant impact factors. Then, assign influence weights to these significant impact factors according to the magnitude of the correlation; the higher the correlation, the greater the weight. Organize the significant impact factors and their influence weights to form the environmental correlation results.

[0111] Step S13711: Integrate the rule matching results, the historical scene matching results, and the environment association results to construct a cross-modal information association network. In the cross-modal information association network, each node represents a core element of a type of modal information, and the edges represent the association relationships between elements.

[0112] The network nodes are constructed from the business rule clauses in the rule matching results, the historical scenarios and intervention effect features in the historical scenario matching results, and the significant environmental elements and their weights in the environmental association results. For example, a rule node could be "re-evaluation of suppliers is required when raw material procurement costs exceed the budget by 10%", a historical scenario node could be "the peak season production scenario in Q3 2023", and an environmental element node could be "the fluctuation range of raw material prices". Then, edges are drawn based on the relationships between these elements. For example, there might be an edge between a rule node and a historical scenario node indicating that "the rule was triggered in that historical scenario", an edge between a rule node and an environmental element node indicating that "the environmental element affects the triggering condition of the rule", and an edge between a historical scenario node and an environmental element node indicating that "the environmental element is one of the features of that historical scenario". The weights of the edges can be set according to the strength of the association (e.g., similarity, association value). A cross-modal information association network is constructed using this method.

[0113] Step S13712: Based on the cross-modal information association network, extract the core association elements and interaction logic of various modal information to generate a multimodal fusion information body containing rule constraints, historical references, real-time status, image features, speech semantics, and environmental influences.

[0114] From the cross-modal information association network, the core association elements most closely related to the set of business intervention nodes are identified. These include business rule clauses that have the greatest impact on the triggering conditions of core control nodes, historical scenarios most similar to the current business state and their intervention effects, and environmental elements with the highest weight in influencing business operations. Simultaneously, the interaction logic between these core association elements is analyzed, such as how environmental elements affect the triggering of rules by influencing the real-time state, and how intervention experience in historical scenarios provides a reference for the execution of current rules. Integrating these core association elements and interaction logic forms a multimodal fusion information body that integrates rule constraints, historical references, real-time state, image features, speech semantics, environmental influences, and other information. This information body comprehensively reflects the current state of business operations and various influencing factors.

[0115] Step S138: Call the node trigger sequence generation module of the multimodal large model to generate multiple different combinations of node trigger sequences based on the multimodal fusion information body and the core control nodes in the business operation intervention node set.

[0116] The node trigger sequence generation module of the multimodal large model takes core control nodes from the multimodal fusion information body and the set of business operation intervention nodes as input. First, based on the rule constraints and real-time status in the multimodal fusion information body, the initial activation probability and priority of the core control nodes are determined. For example, if the multimodal fusion information body contains the real-time status "raw material procurement costs exceed the budget by 10%" and the corresponding rule constraint, the activation priority of the "re-evaluate suppliers" core control node will be higher. Then, the priority of the core control nodes is adjusted by combining historical trigger patterns in historical scenario matching results and the influence of environmental factors in environmental association results. Next, using heuristic search algorithms or reinforcement learning algorithms, under the premise of satisfying the inter-node linkage triggering conditions, multiple different combinations of core control node trigger sequences are generated. These combinations may differ in the triggering order, triggering interval, and whether certain nodes are included, to cover different business intervention strategies.

[0117] Step S1381: Extract the identification information and functional description of all core control nodes from the set of business operation intervention nodes to form a list of core control nodes.

[0118] Traverse the set of business operation intervention nodes, extract the identification information (such as "PP-001", "PP-002") and corresponding functional descriptions (such as "production work order generation" and "raw material procurement plan formulation") of all core control nodes, and arrange them in a certain order (such as business process order or node identification order) to form a core control node list.

[0119] Step S1382: Analyze the rule constraint content in the multimodal fusion information body, define the business rules and constraints that must be followed during the core control node triggering process, and form node triggering rule constraints.

[0120] Extract rule constraints related to the triggering of core control nodes from the multimodal fusion information body, such as "the production work order generation node must be triggered after the sales order summary is completed", "the production work order must be confirmed to have been generated before the raw material procurement plan formulation node is triggered", and "no node triggering shall violate environmental protection policy requirements". Organize and clarify the above rule constraints to form node triggering rule constraints, which serve as the criteria that must be followed when generating the sequence of node triggering.

[0121] Step S1383: Extract historical reference data from the multimodal fusion information body, analyze the triggering sequence pattern and corresponding intervention effect of core control nodes in historical scenarios, and form a reference historical triggering pattern.

[0122] Historical data is extracted from the historical scenario matching results of the multimodal fusion information body to analyze the triggering sequence patterns of core control nodes under different historical scenarios. For example, in historical peak season production scenarios, the common triggering sequence is "production work order generation → raw material procurement plan formulation → production equipment scheduling → production and processing start-up"; while in scenarios with tight raw material supply, the triggering sequence may be "raw material procurement plan formulation → supplier evaluation → production work order adjustment → production and processing start-up". Simultaneously, the intervention effects corresponding to these triggering sequence patterns are recorded, such as which patterns show more significant improvements in business indicators. The above triggering sequence patterns and their corresponding effects are compiled to form a reference historical triggering pattern.

[0123] Step S1384: Based on the real-time status description and multimodal feature data in the multimodal fusion information body, determine the initial activation priority of each core control node under the current business status. The initial activation priority is determined based on the business urgency and impact scope.

[0124] Based on real-time business status descriptions in the multimodal fusion information body, such as "Currently, there is a severe backlog of orders for product A, and the delivery date is approaching" or "The inventory of raw material B is below the safety threshold," the business urgency of each core control node is assessed. Nodes that can resolve urgent issues, such as "expedited processing of production work orders" and "urgent procurement of raw materials," are assigned a higher urgency level. Simultaneously, nodes are evaluated based on their impact scope, such as the number of affected business processes and the scale of resources involved; nodes with a wider impact scope have relatively higher priority. Combining the business urgency and impact scope, each core control node is assigned an initial activation priority score, forming an initial activation priority ranking.

[0125] Step S1385: Based on the environmental impact description in the multimodal fusion information body, adjust the initial activation priority of each core control node according to the predefined environmental factor and node priority mapping rules.

[0126] The environmental impact description in the multimodal fusion information body includes the impact of current external environmental factors on the business, such as "the recent continuous rise in raw material prices" and "competitors launching similar new products." Predefined mapping rules between environmental factors and node priorities specify the direction and degree of influence of different environmental factors on node priorities. For example, when "raw material prices rise," the priority of the "supplier evaluation and replacement" node should be increased; when "competitors launch new products," the priority of the "product differentiation design adjustment" node should be increased. Based on these mapping rules, the initial activation priorities of each core control node are adjusted to obtain the adjusted activation priorities.

[0127] Step S1386: The node trigger sequence generation module of the multimodal large model generates the initial node trigger sequence combination based on the core control node list, node trigger rule constraints, reference historical trigger modes and adjusted activation priorities, using a permutation and combination algorithm.

[0128] The node trigger sequence generation module first treats the nodes in the core control node list as elements to be sorted. Then, it determines the approximate sorting direction of the nodes based on the adjusted activation priority, while adhering to node triggering rules, such as certain nodes must trigger after other nodes. It references historical triggering patterns and draws upon historically effective triggering order structures. Using a permutation and combination algorithm, it generates a large number of initial node triggering order combinations while satisfying the above constraints and reference patterns. For example, for the three highest-priority nodes A, B, and C, it may generate multiple initial combinations such as A→B→C, A→C→B, and B→A→C, without violating the rule constraints.

[0129] Step S1387: Perform rule verification on the initial node triggering order combination, eliminate node triggering order combinations that violate node triggering rule constraints, and generate node triggering order combinations that meet the business rule requirements.

[0130] Each generated initial node triggering order combination is compared one by one with the node triggering rule constraints. For example, if the rule constraint stipulates that "node B must be triggered after node A", then it checks whether the position of node B in the combination is after node A. If there is a combination where node B is before node A, it is discarded. After rule verification, only the node triggering order combinations that meet all business rule requirements are retained.

[0131] Step S1388: Analyze the rationality of the node triggering order in the remaining initial node triggering order combinations, determine whether the order of the core control nodes in the combination conforms to the logical relationship and linkage requirements of the business process, and eliminate node triggering order combinations with logical contradictions.

[0132] For the node triggering sequence combinations that have passed rule validation, their logical rationality is further analyzed. For example, the "Raw Material Procurement Plan Formulation" node should logically follow the "Production Work Order Generation" node because the procurement plan needs to be formulated based on the production work order. If the "Raw Material Procurement Plan Formulation" node precedes the "Production Work Order Generation" node in a combination, there is a logical contradiction, and it should be removed. At the same time, it is checked whether the linkage requirements between nodes are met. For example, if the triggering of one node requires another node to trigger within a specified time, and the combination does not reflect the above linkage, it may also be judged as logically unreasonable and removed.

[0133] Step S1389: Match the logically reasonable node triggering sequence combination with the reference historical triggering mode, and select the node triggering sequence combination whose historical intervention effect reaches the preset business indicator threshold.

[0134] The similarity between logically sound node triggering sequences and historical triggering patterns is calculated. This includes comparing the node sequence structure and key nodes included in the combinations. For combinations with high similarity, further examination is conducted to determine if their corresponding historical intervention effects meet preset business metric thresholds (e.g., an increase of over 10% in on-time order delivery rate, or a cost reduction of over 5%). Node triggering sequence combinations that meet these thresholds are selected; these combinations have historical data support and are more likely to achieve better intervention effects in the current business context.

[0135] Step S13810: By randomly adjusting the triggering order of some core control nodes and increasing or decreasing the interval steps of core control node triggering, the remaining node triggering order combinations are diversified and expanded to generate multiple different node triggering order combinations.

[0136] To increase the diversity of node triggering order combinations and cover more possible intervention strategies, the retained node triggering order combinations are diversified and expanded. For example, for a combination A→B→C→D, it can be randomly adjusted to A→C→B→D (adjusting the order of some nodes), or A→B→[interval step]→C→D (adding an interval step), or A→C→D (removing a certain node). During the expansion process, it is ensured that the node triggering rule constraints and logical rationality are not violated. Through the above method, multiple different combinations of node triggering orders are generated.

[0137] Step S139: For each set of node trigger sequence combinations, the multimodal large model uses the built-in business process simulator to deduce the activation process of each core control node and the evolution of business operation status parameters under the node trigger sequence combination, and form a candidate preliminary intervention plan.

[0138] The multimodal large-scale model's built-in business process simulator contains mathematical models and logical rules for enterprise business processes, capable of simulating the operation of business links and the interaction of nodes. For each combination of node trigger sequences, these are input into the business process simulator. The simulator activates the corresponding core control nodes sequentially according to the trigger order of the nodes in the combination, simulating the operations performed after node activation (such as generating production work orders, adjusting procurement plans, etc.). Simultaneously, it calculates the evolution of business operation status parameters in real time, such as changes in production progress, inventory levels, cost consumption, and order delivery rate over time. The description of the node activation process and the data on the evolution of business operation status parameters are recorded to form a candidate preliminary intervention plan, which includes the expected process of business operation under a specific node trigger sequence.

[0139] Step S1310: Combining the historical intervention effect dataset and multimodal feature matching results, predict the expected operating effect of each candidate preliminary intervention plan, supplement the missing execution details and constraints in the candidate preliminary intervention plans, and improve them into business intervention execution plans.

[0140] By leveraging historical intervention effect data from similar historical intervention schemes in the historical intervention effect dataset, and key influencing factors identified in multimodal feature matching results, the expected operational effects of the candidate preliminary intervention scheme are predicted. For example, based on historical data, if a similar node trigger sequence can improve production efficiency by X% under certain business conditions, it is predicted that the current scheme may also achieve a similar effect. Simultaneously, the candidate preliminary intervention scheme is checked for any missing execution details, such as the specific time points of node triggers, the specific quantities of resources allocated, and the responsible parties, and these are supplemented based on business rules and historical experience. Furthermore, constraints during the scheme execution process are clarified, such as maximum cost limits and minimum execution time requirements. Through the above prediction, supplementation, and refinement, the candidate preliminary intervention scheme is transformed into a complete and clearly defined business intervention execution plan.

[0141] Step S1311: Uniquely identify each group of business intervention execution plans, define the node triggering order, applicable business scenarios and expected intervention objectives corresponding to the group of business intervention execution plans, and form plan identification information.

[0142] Assign a unique identifier to each business intervention execution plan, such as "Plan-20240520-001". The plan identifier information should clearly specify the triggering order of the corresponding nodes (e.g., A→B→C→D), the applicable business scenario (e.g., "rising raw material prices and order backlog"), and the expected intervention goal (e.g., "15% increase in on-time order delivery rate and 8% cost reduction"). Linking this information with the plan's unique identifier forms the plan identification information, facilitating plan management and retrieval.

[0143] Step S1312: Associate and store all business intervention execution plans with their corresponding plan identification information to form multiple sets of business intervention execution plans.

[0144] Establish a business intervention execution plan database, and associate and store the detailed content of each business intervention execution plan (including node triggering order, execution steps, resource allocation, constraints, etc.) with the corresponding plan identification information (unique identifier, node triggering order description, applicable scenario, expected goal, etc.). Plans can be classified according to applicable scenarios, which facilitates the quick selection of suitable plans based on the current business status.

[0145] Step S140: Based on real-time multimodal feedback data of business operation, dynamically adapt and adjust the multiple sets of business intervention execution plans, and select the target business intervention execution plan that matches the current business status.

[0146] During business operations, real-time multimodal feedback data is continuously collected through various monitoring methods, such as real-time operating data of production equipment, order status update data, customer feedback (voice or text data), and product quality inspection image data. This real-time feedback data is compared and analyzed with multiple business intervention execution plans to assess the degree of matching between each plan and the current business state. For plans with low matching degrees, dynamic adjustments are made to the node triggering sequence, execution parameters, and resource allocation based on changes in the real-time feedback data (such as anomalies in a certain link or new changes in the external environment). After adjustment, the expected effects of the plans are re-evaluated, and the plan that best matches the current business state and has the best expected effect is selected as the target business intervention execution plan.

[0147] Step S141: Through the real-time multimodal feedback data acquisition channel of business operation, continuously collect sensor data on state changes in each link of the business during operation, structured data of execution results, image data of abnormal scenes, and voice feedback information to form a real-time multimodal feedback dataset.

[0148] The real-time multimodal feedback data acquisition channel for business operations includes sensor networks, data acquisition interfaces, image acquisition devices, and voice recording devices distributed across various business processes. Status change sensor data includes real-time monitoring data of production equipment temperature, pressure, and vibration, as well as temperature and humidity data of the storage environment; structured execution result data includes production output, sales volume, and order completion rate stored in tabular form; abnormal scene image data includes images of product defects and equipment malfunctions during production; and voice feedback information includes customer complaints and employee work reports. All of this data is transmitted in real-time to the data processing center through the acquisition channel. After aggregation and preliminary processing, it forms a real-time multimodal feedback dataset.

[0149] Step S142: Classify the real-time multimodal feedback dataset by grouping the data according to business process, data modality type, and feedback time to generate classified real-time multimodal feedback data.

[0150] The real-time multimodal feedback dataset is categorized according to different dimensions. First, it is categorized by business process, such as production, sales, and procurement data, ensuring each business process's data is grouped separately. Then, within each business process group, it is further categorized by data modality type, such as text, structured data, image data, and audio data. Finally, for each modality, the data is sorted and grouped by feedback time, such as hourly, daily, or by stage of the business process. This categorization process generates classified real-time multimodal feedback data, facilitating subsequent targeted analysis of feedback data from different business processes, modalities, and time periods.

[0151] Step S143: Extract key status indicators from the classified real-time multimodal feedback data. The extracted key status indicators from the classified real-time multimodal feedback data can directly reflect the current operating efficiency, execution quality and overall situation of the business, forming a set of key status indicators.

[0152] From the categorized real-time multimodal feedback data, key status indicators that directly reflect the operational status of the business are extracted. For example, indicators such as "production qualification rate," "equipment utilization rate," and "production plan completion rate" are extracted from structured data in the production process; indicators such as "order conversion rate," "customer complaint rate," and "sales growth rate" are extracted from structured data in the sales process; the "product defect rate" indicator is extracted from abnormal scene image data through image analysis; and the "customer satisfaction" indicator is extracted from voice feedback information through semantic analysis. These key status indicators can quantify the operational efficiency (such as equipment utilization rate), execution quality (such as production qualification rate and product defect rate), and overall situation (such as sales growth rate and customer satisfaction). The extracted key status indicators are then summarized to form a key status indicator set.

[0153] Step S144: Compare the set of key status indicators with the preset adaptation conditions of each group of business intervention execution plans, and initially screen out candidate business intervention execution plans that meet the basic adaptation requirements.

[0154] Each set of business intervention execution plans has preset adaptation conditions, which define the applicable business status range of the plan, such as "production qualification rate below 90%", "order backlog exceeding 500 orders", and "raw material inventory turnover days greater than 15 days". The indicator values ​​in the key status indicator set are compared one by one with the preset adaptation conditions of each plan. If all preset adaptation conditions of a plan are met by the data in the key status indicator set, or if most of the key conditions are met, then the plan is considered to meet the basic adaptation requirements and is selected as a candidate business intervention execution plan. This preliminary screening reduces the number of plans that require detailed evaluation later.

[0155] Step S145: For each candidate business intervention execution plan, analyze the degree of difference between the data in the set of key status indicators and the preset parameters of the plan, and determine the key parameters and execution details that need to be adjusted for the plan.

[0156] For each candidate business intervention execution plan, the data in the key status indicator set is compared with the preset parameters in the plan. For example, the preset "production qualification rate target value" of the plan is 95%, while the current "production qualification rate" in the key status indicator is 92%, with a difference of 3%. The degree of this difference is analyzed to determine whether the plan needs to be adjusted. For numerical indicators, the percentage or absolute difference is calculated; for logical indicators, the reasons for the difference are analyzed. Based on the degree and cause of the difference, the key parameters in the plan that need to be adjusted are determined, such as the production target of the production work order, the operating parameters of the equipment, the allocation ratio of resources, and execution details, such as the specific time of node triggering and the arrangement of responsible persons.

[0157] Step S1451: Extract the set of preset parameters in each candidate business intervention execution plan, and define the preset values ​​of key parameters such as node trigger threshold, execution time span, and resource allocation ratio involved in each candidate business intervention execution plan.

[0158] Extract the set of preset parameters from the content of the candidate business intervention execution plans. For example, preset values ​​for key parameters in a plan, such as the trigger threshold for the "production work order generation" node (e.g., triggering when the order backlog reaches 300 orders), the execution time span (e.g., the completion time of the production work order is 48 hours), and the resource allocation ratio (e.g., allocating 30% of the production personnel to this work order). Compile these preset parameters and their preset values ​​to form a preset parameter list for each plan.

[0159] Step S1452: Compare each indicator data in the set of key status indicators with the corresponding preset parameters in the candidate business intervention execution plan one by one, and record the numerical differences and logical differences of each comparison.

[0160] Compare the "Order Backlog" indicator data in the key status indicator set with the preset trigger threshold value of the "Production Work Order Generation" node in the solution, and record the numerical difference between the two (e.g., if the current order backlog is 350 orders and the preset threshold is 300 orders, the difference is 50 orders). For logical indicators, such as "Equipment Operating Status," if the key status indicator is "Abnormal," but the solution's preset parameters require the equipment operating status to be "Normal," then record this as a logical difference. Perform the above comparison on all key status indicators and their corresponding preset parameters, and record the differences in detail.

[0161] Step S1453: For the difference between numerical indicator data and preset parameters, calculate the difference ratio and absolute difference amount to quantify the degree of deviation between the data and preset parameters.

[0162] For the difference between numerical indicator data and preset parameters, calculate the difference ratio and the absolute difference. The formula for calculating the difference ratio is (current indicator data - preset parameter value) / preset parameter value. 100%, the absolute difference is the current indicator data minus the preset parameter value. For example, if the current order backlog is 350 orders, the preset threshold is 300 orders, the absolute difference is 50 orders, and the difference ratio is (350-300) / 300. 100% ≈ 16.67%. These quantitative indicators are used to assess the degree of deviation between the data and the preset parameters.

[0163] Step S1454: Analyze the reasons for the differences between logical indicator data and preset parameters, and determine whether the differences are caused by changes in business status or by unreasonable preset parameters.

[0164] For discrepancies between logical indicator data and preset parameters, such as the "equipment operating status" indicator being "abnormal" while the preset parameter requires "normal," analyze the reasons for the discrepancy. This could be due to a change in business status, such as a sudden equipment malfunction causing abnormal operation; or it could be due to unreasonable preset parameters, such as parameters that do not consider the normal maintenance cycle of the equipment, leading to a misjudgment of abnormality during maintenance. Determine the specific cause of the discrepancy by investigating equipment operation logs, maintenance records, etc.

[0165] Step S1455: Based on the quantitative analysis of the degree of deviation and the reasons for the difference, the preset parameters corresponding to the key indicators whose degree of difference exceeds the reasonable range are selected, and the selected preset parameters are output as preliminary candidate parameters that need to be adjusted.

[0166] A reasonable range of variation is set, such as a variation ratio of ±5% for numerical indicators and no fundamental difference for logical indicators. For key indicators that deviate beyond this range, their corresponding preset parameters are selected as preliminary candidate parameters that need adjustment. For example, if the variation ratio of order backlog is 16.67%, exceeding the reasonable range of 5%, then the preset parameter of the trigger threshold for the "Production Work Order Generation" node is listed as a preliminary candidate parameter.

[0167] Step S1456: Combining the core objectives of business operations and the intervention logic of candidate business intervention execution plans, analyze the impact of adjusting the preliminary candidate parameters on the overall execution effect of the plan, and determine whether adjusting the preliminary candidate parameters can improve the adaptability of the plan to the current business status.

[0168] The core objectives of business operations may include improving order delivery rates and reducing production costs. The intervention logic of candidate business intervention execution plans achieves these objectives by adjusting the triggering order and parameters of nodes. The impact of preliminary candidate parameter adjustments on the overall execution effect of the plan is analyzed. For example, will adjusting the trigger threshold of the "Production Work Order Generation" node from 300 orders to 350 orders cause order processing delays and affect the delivery rate? Or will the adjustment more accurately respond to the current order backlog and improve production efficiency? Based on the analysis results, it is determined whether adjusting this parameter can improve the adaptability of the plan to the current business state.

[0169] Step S1457: For each parameter that needs to be adjusted, determine the adjustment direction. The adjustment direction may be to increase the parameter value, decrease the parameter value, or adjust the applicable range of the parameter. The adjustment direction is determined based on the business operation goals and real-time status requirements.

[0170] Based on business operation goals and real-time status requirements, determine the adjustment direction for each parameter that needs adjustment. For example, if the current order backlog is large and the business goal is to speed up order processing, the trigger threshold for the "Production Work Order Generation" node should be lowered to trigger production work order generation earlier; if raw material supply is tight and the business goal is to control costs, the "Raw Material Procurement Quantity" parameter should be lowered. Regarding the applicable scope of parameters, such as the applicable time period for the trigger conditions of the "Production Work Order Generation" node, it may need to be adjusted from "all day" to "weekdays 8:00-18:00".

[0171] Step S1458: Analyze the relationship between the parameter that needs to be adjusted and other parameters in the plan, and determine whether adjusting the parameter will affect the effectiveness of other parameters. If there is a correlation, determine the adjustment requirements of other related parameters simultaneously.

[0172] There may be correlations between parameters. For example, lowering the trigger threshold for the "Production Work Order Generation" node may lead to an increase in the number of production work orders, thereby affecting the effectiveness of parameters such as "Raw Material Procurement Quantity" and "Production Equipment Scheduling Quantity." It is necessary to analyze these correlations and assess the impact of parameter adjustments on other parameters. If the "Production Work Order Quantity" increases but the preset parameter for "Raw Material Procurement Quantity" does not increase accordingly, it will lead to a shortage of raw material supply; therefore, the "Raw Material Procurement Quantity" parameter needs to be adjusted simultaneously. Through the above analysis, the adjustment requirements for all relevant parameters are determined.

[0173] Step S1459: Based on the actual business operation scenario and parameter application specifications, define the specific numerical range or logical conditions of each parameter after adjustment.

[0174] Based on the actual business operation scenarios, such as current production capacity, raw material supply status, and market demand, as well as the company's internal parameter application standards (such as procurement process standards and production safety standards), define the specific numerical range or logical conditions after adjustment for each parameter that needs to be adjusted. For example, the adjusted value range of the trigger threshold for the "production work order generation" node may be 250-300 orders, with the specific value determined based on the order growth trend; the adjustment of the "raw material procurement quantity" parameter must meet inventory management standards to ensure that there is neither shortage nor overstocking.

[0175] Step S14510: Summarize all key parameters that need to be adjusted and their corresponding adjustment details to form an adjustment requirement list for each candidate business intervention execution plan. The adjustment requirement list defines the parameter name, current differences, adjustment direction, adjusted parameter requirements, and related parameter adjustment instructions.

[0176] The key parameters that need to be adjusted in each candidate business intervention execution plan, such as "production work order generation trigger threshold" and "raw material purchase quantity", as well as the corresponding current difference (e.g., difference ratio 16.67%), adjustment direction (e.g., reduction), parameter requirements after adjustment (e.g., 250-300 orders), and related parameter adjustment instructions (e.g., synchronous adjustment of raw material purchase quantity) are summarized to form an adjustment requirement list.

[0177] Step S146: Based on the results of the difference analysis, dynamically adjust the node triggering timing, triggering intensity and linkage logic in the candidate business intervention execution plan to generate an adjusted plan that adapts to the current business status.

[0178] Based on the adjustments to the requirements list, specific adjustments were made to the candidate business intervention execution plans. These adjustments included: adjusting the trigger timing of nodes, such as changing the trigger time for the "Production Work Order Generation" node from when the order backlog reached 300 orders to when it reached 250 orders; adjusting the trigger intensity, such as increasing the procurement quantity for the "Raw Material Procurement Planning" node from the original 1000 units to 1200 units; and adjusting the linkage logic, such as triggering not only the "Raw Material Procurement Planning" node but also the "Production Equipment Maintenance" node simultaneously when the "Production Work Order Generation" node is triggered, ensuring that equipment is in good condition before high-load production. Through these adjustments, a revised plan adapted to the current business status was generated.

[0179] Step S147: Run each adjusted plan in a multimodal large model simulation. Combine multimodal scenario extrapolation technology to extrapolate the operational changes and expected results of each business link after the implementation of the adjusted plan, and form the plan simulation effect data.

[0180] The multimodal large-scale model utilizes its built-in business process simulator and multimodal scenario extrapolation technology to simulate the operation of each adjusted solution. During the simulation, the current business status data and the parameters of the adjusted solution are input. The simulator dynamically extrapolates the operational changes of each link in the business according to the node triggering sequence and execution steps of the solution, such as changes in output in the production link, changes in inventory levels in the inventory link, and changes in order processing progress in the sales link. At the same time, it predicts the expected results after the solution is executed, such as the final order delivery rate, production costs, customer satisfaction and other indicators. The operational change data and expected result data during the simulation process are recorded to form the solution simulation effect data.

[0181] For example, step S1471: build a multimodal simulation environment for business operation, wherein the multimodal simulation environment for business operation includes the operation logic model of each link of the business, the multimodal information interaction interface model and the resource model.

[0182] Build a multimodal simulation environment capable of simulating enterprise business operations. This environment includes: operational logic models for each business stage, such as production scheduling logic in the production stage, order processing logic in the sales stage, and supplier selection logic in the procurement stage. These models are built based on the actual business processes and rules of the enterprise; multimodal information interaction interface models, used to simulate the transmission and interaction of different modal information between stages, such as the transmission of text commands, recognition of image data, and response to voice commands; and resource models, including models of resources such as manpower, equipment, raw materials, and funds, simulating the quantity, capacity, cost, and other attributes of resources, as well as their allocation and consumption in business operations.

[0183] Step S1472: Import the data content of node triggering order, triggering conditions, and execution parameters in the adjusted scheme into the business operation multimodal simulation environment, and set the initial conditions and boundary constraints of the business operation multimodal simulation environment.

[0184] Input the adjusted node triggering order (e.g., A→B→C→D), triggering conditions (e.g., node A is triggered when there are 250 backlogged orders), and execution parameters (e.g., node B's purchase quantity is 1200 units) into the multimodal business operation simulation environment. Simultaneously, set the initial conditions of the simulation environment, such as the current order quantity, inventory level, equipment status, and resource availability; and set boundary constraints, such as maximum production capacity limits, minimum inventory safety thresholds, and resource budget limits, to ensure that the simulation process conforms to the actual situation of the enterprise.

[0185] Step S1473: Start the simulation of the multimodal simulation environment for the business operation. According to the requirements of the adjusted plan, trigger the corresponding core control nodes in sequence to simulate the response behavior and changes in the operating status of each link of the business after the core control nodes are triggered.

[0186] The system launches a multimodal simulation environment for business operations. Based on the adjusted node triggering sequence and conditions, the simulation environment sequentially activates core control nodes when the conditions are met. For example, when the order backlog in the simulation environment reaches 250 orders, the "Production Work Order Generation" node is triggered. This node generates a production work order and transmits it to the production process. Upon receiving the work order, the production process adjusts the production plan according to the execution parameters, starts production equipment, assigns production personnel, and simulates the production process. Simultaneously, the system simulates the response behaviors of other business processes after node triggering, such as the procurement process adjusting its procurement plan based on the production work order, and the inventory process tracking changes in raw material and product inventory, recording the operational status changes of each process in real time.

[0187] Step S1474: During the simulation operation of the multimodal simulation environment of the business operation, the trigger time, execution duration and business impact data of each core control node are recorded in real time to form a node execution record.

[0188] During the simulation, the operation of each core control node is tracked in real time. The specific time the node is triggered (e.g., 2024-05-20 10:30:00), the time taken from triggering to completion (e.g., 45 minutes), and the business impact data generated after the node execution (e.g., after the "Production Work Order Generation" node is executed, the production plan increases by 500 products, and the expected raw material consumption increases by 1000 units). The above information is compiled into a node execution record.

[0189] Step S1475: Track the changes in operating parameters of each business process, including dynamic changes in execution efficiency, resource consumption, and output quality, to form dynamic data of process operation.

[0190] During the simulation, operational parameters for each stage of the business were continuously tracked, such as production efficiency (pieces / hour) and equipment utilization rate (%) in the production stage; procurement cost (RMB / unit) and procurement cycle (days) in the procurement stage; and order processing speed (orders / hour) and customer complaint rate (%) in the sales stage. The dynamic changes of these parameters over time were recorded. For example, production efficiency increased from 100 pieces / hour at the start of the simulation to 120 pieces / hour and then stabilized at that level; procurement cost fluctuated from RMB 10 / unit to RMB 10.5 / unit. This dynamic data was then compiled to form dynamic operational data for each stage.

[0191] Step S1476: Analyze the linkage effect between core control nodes during the simulation operation of the multimodal simulation environment of the business operation, record the activation or inhibition effect of core control nodes on other core control nodes after the core control nodes are triggered, and form node linkage effect data.

[0192] During the simulation, the impact of triggering core control nodes on other nodes was observed. For example, triggering the "Production Work Order Generation" node activated the "Raw Material Procurement Planning" and "Production Equipment Scheduling" nodes; the "Raw Material Procurement Planning" node triggered the "Supplier Evaluation" node due to rising raw material prices; and the "Production Equipment Scheduling" node suppressed the "Non-Emergency Production Work Order Generation" node due to excessive equipment load. The timing, involved nodes, and degree of impact of these activation or suppression effects were recorded to form node linkage effect data.

[0193] Step S1477: After the simulation of the multimodal business operation environment is completed, the overall business operation results data are statistically analyzed, including the final business indicators achieved, overall execution efficiency, and total resource consumption, to form overall effect statistics.

[0194] After the simulation runs, the overall operational results are statistically analyzed. The final business metrics achieved include on-time order delivery rate, product qualification rate, and customer satisfaction; overall execution efficiency includes total process time and average processing efficiency; total resource consumption includes total raw material consumption, total labor costs, and total equipment energy consumption. These statistics are then compiled to form overall performance statistics.

[0195] Step S1478: Compare the simulation results with the intervention goals preset in the plan, analyze the degree of goal achievement, identify the advantages and disadvantages in the implementation process of the plan, and form goal achievement analysis data.

[0196] The overall statistical data obtained from the simulation are compared with the intervention targets preset in the plan. For example, if the preset on-time delivery rate target for orders is 95%, and the simulation result is 92%, then the target achievement rate is 96.8%; if the preset cost reduction target is 8%, and the simulation result is 7.5%, then the target achievement rate is 93.75%. The reasons for not fully achieving the targets are analyzed, and the advantages (such as significant improvement in production efficiency) and disadvantages (such as poor control of raw material procurement costs) in the implementation process are identified, forming target achievement analysis data.

[0197] Step S1479: Integrate node execution records, dynamic data of process operation, node linkage effect data, overall effect statistics and target achievement analysis data to form scheme simulation effect data.

[0198] By integrating node execution records, dynamic data of process operation, node linkage effect data, overall effect statistics and target achievement analysis data, and organizing them according to data type and analysis dimension, a comprehensive simulation effect data reflecting the simulation operation effect of the adjusted scheme is formed.

[0199] Step S14710: Classify and store the simulation effect data of the scheme according to data type and analysis dimension, and set the data classification storage directory and retrieval identifier.

[0200] Establish categorized storage directories for the simulation effect data, such as subdirectories based on data types like "Node Execution Records," "Dynamic Data of Process Operation," "Node Linkage Effect Data," "Overall Effect Statistics," and "Target Achievement Analysis Data." Further subdivide each subdirectory according to analytical dimensions such as business processes and time. Assign search identifiers to each data type, such as keywords, timestamps, and scheme identifiers, to facilitate rapid querying and analysis of the simulation effect data.

[0201] Step S148: Based on the predefined evaluation function, the simulation effect data of all adjusted schemes are quantitatively scored in multiple dimensions such as business operation efficiency, execution cost, and result volatility. Based on the quantitative scoring results, the adjusted scheme with the best simulation effect is selected as the target business intervention execution scheme.

[0202] The predefined evaluation function comprehensively considers multiple dimensions, including operational efficiency, execution cost, and outcome volatility. For example, the operational efficiency dimension score can be based on indicators such as order processing speed and production efficiency; the execution cost dimension score can be based on indicators such as raw material costs and labor costs; and the outcome volatility dimension score can be based on the fluctuation range of business indicators (such as delivery rate and pass rate). The evaluation function standardizes the simulated performance data for each dimension, assigns different weights, and then calculates the overall score. For example, if the efficiency dimension has a weight of 0.4, the cost dimension has a weight of 0.3, and the volatility dimension has a weight of 0.3, the overall score would be 0.4. Efficiency score +0.3 Cost score +0.3 Volatility score. Based on the overall score, all adjusted solutions are ranked from highest to lowest, and the solution with the highest score is selected as the target business intervention implementation solution.

[0203] Step S149: Extract the core execution points and key constraints of the target business intervention execution plan to form an execution description of the plan.

[0204] Extract core execution points from the target business intervention execution plan, such as the triggering order and timing of key nodes, the setting values ​​of important execution parameters, and the allocation plan of major resources; clarify key constraints, such as cost limits, time constraints, and quality standards. Organize the above content into a clear and concise execution description to facilitate business personnel's understanding and implementation of the plan.

[0205] Step S150: Embed the target business intervention execution plan into the business operation process, drive each link of the business to adjust its operation mode according to the requirements of the target business intervention execution plan, and complete the optimized control of business operation.

[0206] The content of the target business intervention implementation plan is transformed into specific operational instructions and parameter settings for each stage of the business operation process. For example, production work order information from the plan is sent to the production management system to guide the production stage to adjust production plans; procurement plan information is sent to the procurement management system to guide the procurement stage to adjust purchase orders. Through the enterprise's business management system, the plan is embedded into the actual business operation process, driving each business stage to adjust its operating mode according to the plan's requirements, such as changing production scheduling methods, adjusting inventory management strategies, and optimizing sales processes. During the plan's execution, the business operation status and the plan's effectiveness are continuously monitored to ensure that business operations are optimized as expected, ultimately achieving optimized control over business operations.

[0207] Figure 2 The following is a schematic diagram of the hardware structure of the intelligent business optimization control system 100 based on a multimodal large model provided in an embodiment of the present invention, such as... Figure 2As shown, the intelligent business optimization control system 100 based on a multimodal large model may include a processor 110, a machine-readable storage medium 120, a bus 130, and a communication unit 140.

[0208] Machine-readable storage medium 120 can store data and / or instructions. In some embodiments, machine-readable storage medium 120 can store data acquired from an external terminal. In some embodiments, machine-readable storage medium 120 can store data and / or instructions used by the intelligent service optimization control system 100 based on a multimodal large model to execute or use in order to complete the exemplary methods described in this invention. In a specific implementation, one or more processors 110 execute the computer-executable instructions stored in machine-readable storage medium 120, enabling processor 110 to execute the intelligent service optimization control method based on a multimodal large model as described in the above method embodiments. Processor 110, machine-readable storage medium 120, and communication unit 140 are connected via bus 130, and processor 110 can be used to control the transmission and reception actions of communication unit 140. The specific implementation process of processor 110 can be found in the various method embodiments executed by the intelligent service optimization control system 100 based on a multimodal large model described above, and their implementation principles and technical effects are similar, so they will not be repeated here.

[0209] Furthermore, embodiments of the present invention also provide a readable storage medium containing computer-executable instructions. When the processor executes the computer-executable instructions, the above-mentioned intelligent business optimization control method based on a multimodal large model is implemented.

[0210] It should be noted that, in order to simplify the description of this invention and thus aid in the understanding of one or more embodiments, the foregoing description of the embodiments of this invention sometimes combines multiple features into a single embodiment, drawing, or description thereof. Similarly, it should be noted that, in order to simplify the description of this invention and thus aid in the understanding of one or more embodiments, the foregoing description of the embodiments of this invention sometimes combines multiple features into a single embodiment, drawing, or description thereof.

Claims

1. A smart service optimization control method based on a multimodal large model, characterized in that, The method includes: Based on the dynamic correlation of the entire business operation process, a set of business operation intervention nodes is generated. The set of business operation intervention nodes includes key control nodes of each business link and the linkage triggering conditions between nodes. Collect diverse and multimodal information resources related to business operations. These diverse and multimodal information resources include text-based business rule information, structured historical business execution information, real-time business status sensor data, image-based business credential information, voice-based business command information, and multi-dimensional data of the external environment. The multimodal large model performs cross-modal coupling processing on the set of business operation intervention nodes and the multi-dimensional multimodal information resources to generate multiple sets of business intervention execution schemes, each set of business intervention execution schemes corresponding to different node triggering sequences; Based on real-time multimodal feedback data of business operations, the multiple sets of business intervention execution plans are dynamically adapted and adjusted to select the target business intervention execution plan that matches the current business status. The target business intervention execution plan is embedded into the business operation process, driving each link of the business to adjust its operation mode according to the requirements of the target business intervention execution plan, thereby completing the optimized control of business operation.

2. The intelligent service optimization control method based on a multimodal large model according to claim 1, characterized in that, The set of business operation intervention nodes generated based on the dynamic correlation of the entire business operation process includes: Analyze the entire business operation process, traverse all execution stages from the start to the end of the business, define the core functions and operational requirements of each execution stage, and form a list of business process stages; Analyze the interaction between each execution stage and other execution stages, identify the multimodal information transmission paths and dependencies between execution stages, and construct a network of business stage connections; Based on the business process association network, operation points that meet predefined key criteria in each execution process are extracted, and the extracted operation points are used as candidate initial control nodes. Based on predefined business indicators, the candidate initial control nodes are prioritized by function to select core control nodes. The predefined business indicators include the number of downstream links affected by the node and the degree of interruption of the overall process due to node failure. For each core control node, analyze its activation conditions in business operation, define the prerequisite business states and multimodal external input information required for node activation, and form a node activation condition description. Analyze the interaction between different core control nodes, determine the triggering or inhibiting effect of one core control node on other core control nodes after its activation, and establish the linkage logic between core control nodes. Based on the node activation condition description and the linkage logic between the core control nodes, linkage triggering conditions between the core control nodes are formulated, and the linkage triggering conditions between the core control nodes define the order of triggering and the associated constraints of the core control nodes. All core control nodes are structurally integrated according to the business process sequence and linkage relationship to form a node list that includes node identifiers, function descriptions, and activation conditions; Associate and bind the node list with the linkage triggering conditions between the core control nodes to construct the basic framework of the business operation intervention node set; A logical integrity analysis is performed on the basic framework of the business operation intervention node set to identify conflicts in the linkage relationships and triggering conditions of each core control node, correct any conflicting content, and form the business operation intervention node set.

3. The intelligent service optimization control method based on a multimodal large model according to claim 1, characterized in that, The process involves cross-modal coupling of the set of business operation intervention nodes and the diverse multimodal information resources using a multimodal large model to generate multiple sets of business intervention execution plans, including: The textual business rule information in the multi-modal information resources is structurally transformed into a standardized rule expression that can be recognized by the multi-modal large model, thus forming standardized business rules. Key operational data are extracted from the structured historical business execution information in the multi-modal information resources. The triggering effects and operational patterns of business intervention nodes under different historical scenarios are analyzed to form a historical intervention effect dataset. The system analyzes the real-time business status sensing data from the multi-modal information resources, captures the operating parameters and execution progress of each stage of the current business, and generates a real-time business status description. Feature extraction is performed on the image-type business voucher information in the multi-modal information resources, which is then transformed into standardized image feature vectors to form an image voucher feature set; The voice-type business command information in the multi-modal information resources is converted from speech to text and semantically parsed to generate structured voice command descriptions, forming a voice command dataset; Identify key influencing factors in the multidimensional data of the external environment in the multi-dimensional information resources, analyze the way and path of the key influencing factors in the identified multidimensional data of the external environment on business operations, and form a description of the impact of the external environment. The standardized business rules, the historical intervention effect dataset, the real-time business status description, the image credential feature set, the voice command dataset, and the external environment impact description are input into the multimodal information fusion module of the multimodal large model. In the multimodal information fusion module, the feature representations corresponding to the various types of input data are aligned, and the information of different modalities is mapped to a unified semantic or feature space. Then, cross-modal information association processing is performed to generate a multimodal fused information body. The node trigger sequence generation module of the multimodal large model is invoked to generate multiple different combinations of node trigger sequences based on the multimodal fusion information body and the core control nodes in the business operation intervention node set. For each set of node trigger sequence combinations, the multimodal large model uses the built-in business process simulator to deduce the activation process of each core control node and the evolution of business operation status parameters under that node trigger sequence combination, and form candidate preliminary intervention schemes. Based on the historical intervention effect dataset and multimodal feature matching results, the expected operating effect of each candidate preliminary intervention plan is predicted, and the missing execution details and constraints in the candidate preliminary intervention plans are supplemented to improve them into business intervention execution plans. Each business intervention execution plan is uniquely identified, defining the node triggering sequence, applicable business scenarios, and expected intervention objectives corresponding to the business intervention execution plan, thus forming plan identification information; All business intervention execution plans are associated with and stored with their corresponding plan identification information to form multiple sets of business intervention execution plans.

4. The intelligent service optimization control method based on a multimodal large model according to claim 1, characterized in that, The real-time multimodal feedback data based on business operations is used to dynamically adapt and adjust the multiple sets of business intervention execution plans, and to select the target business intervention execution plan that matches the current business status, including: Through the real-time multimodal feedback data acquisition channel for business operations, we continuously collect sensor data on status changes in each business process, structured data on execution results, image data of abnormal scenarios, and voice feedback information to form a real-time multimodal feedback dataset. The real-time multimodal feedback dataset is classified and grouped according to business process, data modality type, and feedback time to generate classified real-time multimodal feedback data. Key status indicators are extracted from the classified real-time multimodal feedback data. The extracted key status indicators can directly reflect the current operating efficiency, execution quality and overall situation of the business, forming a set of key status indicators. The set of key status indicators is compared with the preset adaptation conditions of each group of business intervention execution plans to initially screen out candidate business intervention execution plans that meet the basic adaptation requirements. For each candidate business intervention execution plan, analyze the degree of difference between the data in the set of key status indicators and the preset parameters of the plan, and determine the key parameters and execution details that need to be adjusted in the plan. Based on the results of the difference analysis, the timing, intensity and linkage logic of node triggering in the candidate business intervention execution plan are dynamically adjusted to generate an adjusted plan that adapts to the current business status. The multimodal large model simulates each adjusted plan, and combined with multimodal scenario extrapolation technology, extrapolates the operational changes and expected results of each business link after the implementation of the adjusted plan, forming the plan simulation effect data; Based on a predefined evaluation function, the simulation effect data of all adjusted plans are quantitatively scored in multiple dimensions such as business operation efficiency, execution cost, and result volatility. Based on the quantitative scoring results, the adjusted plan with the best simulation effect is selected as the target business intervention execution plan. Extract the core execution points and key constraints of the target business intervention execution plan to form an execution description.

5. The intelligent service optimization control method based on a multimodal large model according to claim 2, characterized in that, The analysis of the interaction between each execution stage and other execution stages, the identification of multimodal information transmission paths and dependencies between stages, and the construction of a business stage association network include: For each execution stage, define the types of multimodal input information it needs to receive and the types of multimodal result information it needs to output during business operations, and form a multimodal information interaction list for each stage; Based on the business process definition and data flow logs, the upstream generation link corresponding to the multimodal input information of each execution link is determined, thereby determining the starting and ending links of multimodal information transmission and forming a multimodal information transmission path record; Identify the receiving stage of the multimodal information output at each execution stage, define the downstream stage of multimodal information transmission, supplement and improve the multimodal information transmission path record, and form a multimodal information transmission path diagram; Analyze whether the operation of each execution stage depends on the execution results of other stages. If there is a dependency, determine the specific stage and the multimodal result type of the dependency, and form a list of stage dependencies. The list of dependencies between the links is further refined, distinguishing between direct dependencies and indirect dependencies. Direct dependencies refer to the fact that the operation of one link directly requires the immediate result of another link, while indirect dependencies refer to dependencies formed through multiple intermediate links. Based on the multimodal information transmission path graph and the refined list of process dependencies, an initial framework for business process association is constructed. In the initial framework, nodes represent execution processes, and directed edges represent multimodal information transmission paths and dependencies. Add attribute identifiers to each directed edge to define the type of multimodal information or dependency type that the directed edge represents; Analyze whether there are circular dependencies or multimodal information transmission loops in the initial framework of business process association. If so, parse the relationship between the links in the multimodal information transmission loop, define the operation logic and triggering conditions of the multimodal information transmission loop, and correct unreasonable parts in the business process association network. The nodes and edges in the initial framework of business process associations are optimized and reorganized, nodes with duplicate functions are merged, and redundant multimodal information transmission paths are simplified to form a streamlined initial business process association network. Verify the completeness and accuracy of the simplified initial business process association network, check whether all execution steps are included in the business process association network, whether all multimodal information transmission paths and dependencies are accurately expressed, correct any missing or incorrect content, and form the business process association network.

6. The intelligent service optimization control method based on a multimodal large model according to claim 3, characterized in that, The standardized business rules, the historical intervention effect dataset, the real-time business status description, the image credential feature set, the voice command dataset, and the external environment impact description are input into the multimodal information fusion module of the multimodal large model; In the multimodal information fusion module, the feature representations corresponding to various input data are aligned, and information from different modalities is mapped to a unified semantic or feature space. Then, cross-modal information association processing is performed to generate a multimodal fused information body, including: The standardized business rules are segmented to extract core keywords and constraint statements, forming a set of rule keywords; Feature extraction is performed on the historical intervention effect dataset to identify key influencing factors and effect representation data of business interventions in different historical scenarios, forming a historical effect feature set. The real-time service status description is structured and parsed to transform the descriptive information into standardized status parameters and numerical representations, forming a set of real-time status parameters. The image credential feature set is subjected to dimensionality normalization to form a standardized image feature set; Semantic features are extracted from the voice command dataset and transformed into semantic vectors that can be processed by a multimodal large model, forming a voice command semantic set; Extract elements from the description of external environmental impacts, define the types and specific contents of environmental elements that affect business operations, and form a set of environmental impact elements. The set of rule keywords, the set of historical effect features, the set of real-time state parameters, the set of standardized image features, the set of voice command semantics, and the set of environmental impact factors are input into the information preprocessing module of the multimodal large model for multimodal data format unification processing, so that the expression form of different modal information remains consistent. By using a cross-modal association analysis algorithm based on a multimodal large model, the matching relationship between the rule keyword set, the real-time state parameter set, the standardized image feature set, and the voice command semantic set is mined, and the business rule clauses that the current real-time state conforms to are identified to form a rule matching result; Analyze the similarity between the historical effect feature set, the real-time state parameter set, and the standardized image feature set to identify historical scenarios similar to the current business state and their corresponding intervention effect features, thus forming historical scenario matching results; Calculate the correlation degree between the set of environmental impact factors, the set of rule keywords, the set of real-time status parameters, and the set of standardized image features, identify the environmental factors that significantly affect the execution of business rules and the current business status and their influence weights, and form an environmental correlation result; The rule matching results, historical scene matching results, and environmental association results are integrated to construct a cross-modal information association network. In the cross-modal information association network, each node represents a core element of a type of modal information, and the edges represent the association relationships between elements. Based on the cross-modal information association network, the core association elements and interaction logic of various modal information are extracted to generate a multimodal fusion information body that includes rule constraints, historical references, real-time status, image features, speech semantics, and environmental influences.

7. The intelligent service optimization control method based on a multimodal large model according to claim 4, characterized in that, For each candidate business intervention execution plan, the degree of difference between the data in the key status indicator set and the plan's preset parameters is analyzed to determine the key parameters and execution details that need to be adjusted, including: Extract the set of preset parameters from each candidate business intervention execution plan, and define the preset values ​​of key parameters involved in each candidate business intervention execution plan, such as node trigger threshold, execution time span, and resource allocation ratio; Each indicator data in the set of key status indicators is compared one by one with the corresponding preset parameters in the candidate business intervention execution plan, and the numerical differences and logical differences of each comparison are recorded. For the difference between numerical indicator data and preset parameters, calculate the difference ratio and absolute difference amount to quantify the degree of deviation between the data and the preset parameters; For discrepancies between logical indicator data and preset parameters, analyze the reasons for the discrepancies and determine whether they are caused by changes in business status or by unreasonable preset parameters in the solution. Based on the quantitative analysis of the degree of deviation and the reasons for the difference, the preset parameters corresponding to the key indicators whose degree of difference exceeds the reasonable range are selected, and the selected preset parameters are output as the preliminary candidate parameters that need to be adjusted. By combining the core objectives of business operations and the intervention logic of candidate business intervention execution plans, we analyze the impact of adjusting the initial candidate parameters on the overall execution effect of the plan, and determine whether adjusting the initial candidate parameters can improve the adaptability of the plan to the current business status. For each parameter that needs to be adjusted, determine the direction of adjustment. The direction of adjustment may be to increase the parameter value, decrease the parameter value, or adjust the applicable range of the parameter. The direction of adjustment is determined based on the business operation goals and real-time status requirements. Analyze the relationship between the parameter that needs to be adjusted and other parameters in the plan, and determine whether adjusting this parameter will affect the effectiveness of other parameters. If there is a correlation, determine the adjustment requirements of other related parameters simultaneously. Based on the actual business operation scenarios and parameter application specifications, define the specific numerical range or logical conditions of each parameter after adjustment. Summarize all key parameters that need to be adjusted and their corresponding adjustment details to form an adjustment requirement list for each candidate business intervention execution plan. The adjustment requirement list defines the parameter name, current differences, adjustment direction, post-adjustment parameter requirements, and related parameter adjustment instructions.

8. The intelligent service optimization control method based on a multimodal large model according to claim 2, characterized in that, For each core control node, the activation conditions during business operation are analyzed, defining the prerequisite business states and multimodal external input information required for node activation, forming a node activation condition description, including: Trace the position of each core control node in the business process, define the business stage and related links before and after the core control node, and determine the business scenario background of the core control node's operation. Analyze the core functions of the core control node, define the business operations that need to be completed and the expected business effects after the core control node is activated, and deduce the functional requirements for activating the core control node based on this. Based on the functional requirements of the core control node, determine the internal business state required for its activation. The internal business state includes the execution results of the preceding steps, the current business progress status, and the reserve status of relevant resources. The multimodal external input information on which the core control node relies during startup is determined. This multimodal external input information includes text-based instruction information, image-based verification credentials, voice-based authorization instructions, and structured environmental data. The identified preceding business status is described in detail, and the specific manifestation and judgment criteria of each preceding business status are defined. The extracted multimodal external input information is classified, and the source, format requirements and reception time window of each type of multimodal external input information are defined; Analyze the logical relationships between the preceding business states, determine whether the preceding business states are satisfied simultaneously or selectively, and define the combination conditions between the preceding business states. Analyze the relationships between multimodal external input information, determine the priority and complementarity of different multimodal external input information, and define the order of receiving multimodal external input information and the conditions for substitution. The combined conditions of the preceding business status are integrated with the requirements for receiving multimodal external input information to form a complete condition rule for node activation. The complete condition rule for node activation defines all the necessary conditions for the core control node to start. Based on the complete condition rules for node activation, a node activation condition description is generated. The node activation condition description clearly lists the various prerequisite business states, multimodal external input information, related combination logic, and receiving requirements required for the activation of the core control node.

9. The intelligent service optimization control method based on a multimodal large model according to claim 3, characterized in that, The node trigger sequence generation module for calling the multimodal large model generates multiple different combinations of node trigger sequences based on the multimodal fusion information body and the core control nodes in the business operation intervention node set, including: Extract the identification information and functional descriptions of all core control nodes from the set of business operation intervention nodes to form a list of core control nodes; The rule constraint content in the multimodal fusion information body is analyzed to define the business rules and constraints that must be followed during the triggering process of the core control node, thus forming node triggering rule constraints; Historical reference data is extracted from the multimodal fusion information body, and the triggering sequence pattern and corresponding intervention effect of the core control nodes in the historical scenario are analyzed to form a reference historical triggering pattern. Based on the real-time status description and multimodal feature data in the multimodal fusion information body, the initial activation priority of each core control node under the current business status is determined. The initial activation priority is determined based on the business urgency and the scope of impact. Based on the environmental impact description in the multimodal fusion information body, and according to the predefined environmental factor and node priority mapping rules, the initial activation priority of each core control node is adjusted. The node trigger sequence generation module of the multimodal large model generates the initial node trigger sequence combination based on the core control node list, node trigger rule constraints, reference historical trigger modes, and adjusted activation priorities, using a permutation and combination algorithm. Perform rule validation on the initial node triggering order combination, eliminate node triggering order combinations that violate node triggering rule constraints, and generate node triggering order combinations that meet the business rule requirements; Analyze the rationality of the node triggering order in the remaining initial node triggering order combinations, determine whether the order of the core control nodes in the combination conforms to the logical relationship and linkage requirements of the business process, and eliminate node triggering order combinations with logical contradictions. The logically reasonable node triggering sequence combination is matched with the reference historical triggering mode to filter out the node triggering sequence combination whose historical intervention effect has reached the preset business indicator threshold. By randomly adjusting the triggering order of some core control nodes and increasing or decreasing the interval steps of core control node triggering, the reserved node triggering order combinations are diversified and expanded to generate multiple different node triggering order combinations.

10. An intelligent business optimization control system based on a multimodal large model, characterized in that, The system includes a processor and a memory, the memory being connected to the processor. The memory is used to store programs, instructions, or code, and the processor is used to run the programs, instructions, or code in the memory to implement the intelligent business optimization control method based on a multimodal large model as described in any one of claims 1-9.

Images