Multi-modal large model multi-agent collaborative fishery adaptive control method and system

Abstract

The present application relates to the technical field of intelligent decision-making and equipment linkage control of facility fisheries, and in particular to a facility fisheries adaptive control method and system based on multi-modal fisheries large model and multi-agent collaborative reasoning, comprising: collecting multi-modal data of facility fisheries field; uniformly encoding the multi-modal data to obtain a uniform state vector; constructing a fisheries knowledge graph; establishing a multi-agent collaborative reasoning structure, each agent generating candidate actions, risk judgments and reason chains in its respective professional field; submitting the candidate actions generated by each agent to the collaborative decision-making layer to obtain a joint control scheme that meets the consistency threshold; performing safety checking on the joint control scheme; and converting the joint control scheme that passes the safety checking into a device-level control sequence and executing it. The present application forms a complete control closed loop from perception, decision-making, safety checking and execution through multi-modal uniform encoding, knowledge graph and multi-agent collaboration, overcoming the limitations of single-body decision-making.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent decision-making and equipment linkage control for facility fishery, and specifically relates to an adaptive control method and system for facility fishery based on multimodal fishery large model and multi-agent collaborative reasoning. Background Art

[0002] With the popularization of sensing equipment for facility fishery, various heterogeneous data such as video streams, water quality time series, equipment logs, operation and maintenance records, and professional literature have been gradually accumulated at aquaculture sites. However, how to convert the above multimodal data into executable control strategies remains a difficulty in current intelligent aquaculture systems. Most of the existing technologies stay at the stage of single-task recognition, rule base recommendation or single-model output suggestion, and have not formed an integrated reasoning and control closed loop oriented to multi-equipment linkage, causal verification and safe execution.

[0003] The prior art proposes an intelligent aquaculture management system, which adopts image recognition, cluster analysis and central control equipment for management. Such solutions can complete basic recognition and management instruction generation, but their decision-making logic is still dominated by single-point recognition and rule triggering, and lacks unified deep representation of multimodal information such as videos, water quality, operation logs and knowledge documents. Meanwhile, some technologies disclose a feeding decision-making method based on knowledge base and word vector analysis. This solution generates feeding suggestions through knowledge base retrieval and image analysis, which is valuable for specific subtasks, but its technical boundary is focused on single-domain feeding decision-making, does not cover multi-equipment linkage scenarios such as aeration, water circulation, disease treatment and energy consumption scheduling, nor establishes a cross-agent collaboration mechanism.

[0004] In addition, the prior art discloses a decision optimization system for aquaculture, which uses a learning engine and a main knowledge base to generate schemes for aquaculture activities. Compared with the traditional rule base, such systems further introduce the idea of learning and optimization, but their knowledge processing is still biased towards centralized strategy generation, does not form independent reasoning units for different professional roles, nor performs dual verification on the generated actions based on equipment constraints and counterfactual results.

[0005] Recently, an exploratory direction of inputting visual recognition results into a language model to generate management suggestions has emerged in the field, indicating that the industry has begun to pay attention to the application of large models in aquaculture. However, such routes usually transcribe the recognition results into text in one direction, and then output suggestions by a single language model, which has the following problems: first, the model lacks a unified representation of underlying sensing time series and equipment states; second, the output suggestions are difficult to be directly mapped to equipment-level control sequences; third, in the absence of cross-validation and execution constraints, it is easy to generate hallucinatory suggestions that are inconsistent with on-site working conditions.

[0006] Existing technologies suffer from at least the following technical shortcomings: First, the lack of a unified fisheries state representation that can simultaneously accommodate visual, water quality, equipment, and textual knowledge results in decision-making systems only seeing problems locally and struggling to perform global scheduling. Second, the lack of a reasoning structure with clearly defined division of labor among multiple professional roles forces a single model to both identify problems and formulate control strategies, easily overlooking cross-subsystem conflicts. Third, the lack of security constraints and causal verification for the output strategies of large models makes it difficult to directly translate textual suggestions into credible equipment actions. Fourth, existing systems typically only output suggestions, rather than executable control sequences, execution conditions, fallback strategies, and post-execution evaluation results, making it difficult to form a closed loop.

[0007] Therefore, it is necessary to propose an adaptive control scheme for facility fisheries that can construct a unified fisheries state representation, support multi-agent collaborative reasoning, and perform constraint verification and counterfactual verification before execution. Summary of the Invention

[0008] The purpose of this invention is to provide an adaptive control method and system for facility fisheries based on a multimodal fisheries big model and multi-agent collaborative reasoning, in order to solve the technical problems in existing facility fisheries control, such as difficulty in integrating multimodal heterogeneous data into a unified state representation, lack of a specialized collaborative reasoning mechanism oriented towards multiple control objectives, inability of the large language model generation results to meet equipment and process constraints, and inability to convert control suggestions into executable closed loops.

[0009] To achieve the above objectives, embodiments of the present invention provide a fisheries adaptive control method based on multimodal large model and multi-agent collaboration, comprising: Collect multimodal data from facility-based fisheries sites; The multimodal data is uniformly encoded to obtain a unified state vector; Constructing a fisheries knowledge graph; Establish a multi-agent collaborative reasoning structure, in which each agent generates candidate actions, risk judgments, and reasoning chains in its own professional field; Each agent submits its candidate actions to the collaborative decision-making layer to obtain a joint control scheme that meets the consensus threshold. The joint control scheme is subjected to security verification; The joint control scheme that has passed safety verification is converted into a device-level control sequence and executed.

[0010] Optionally, uniformly encoding the multimodal data to obtain a unified state vector includes: Visual features, dynamic state features, and knowledge semantic features are mapped to intermediate vector spaces of the same dimension through feature projection layers corresponding to their respective modalities, resulting in visual projection features, temporal projection features, and text projection features. Different modal features from the same time window and the same scene state are used as positive sample pairs, and modal features from different time windows or different scene states are used as negative sample pairs. By comparing and learning the loss function, the distance between positive sample pairs in the semantic space is shortened, and the distance between negative sample pairs is widened. The cross-modal attention module performs weighted fusion of visual projection features, temporal projection features, and text projection features at the same time to obtain a unified state vector, where the weights are dynamically determined based on the confidence or reliability of each modal feature.

[0011] Optionally, constructing a fisheries knowledge graph includes: The multi-source heterogeneous aquaculture knowledge text is structured and parsed. Entities are extracted using named entity recognition and relation extraction methods, and the relationships between entities are identified. The extracted entities and relations are organized into triples according to a predefined ontology schema; Extract the sequence of working conditions, actions, and results recorded in historical cases into case subgraphs, and establish associations between the action nodes in the case subgraphs and the existing equipment entities and action entities in the knowledge graph. The automatically extracted results are validated and supplemented using expert rules to obtain a searchable fisheries knowledge graph.

[0012] Optionally, a multi-agent collaborative reasoning structure can be established, where each agent generates candidate actions, risk assessments, and reasoning chains for its own professional domain, including: Each specialized intelligent agent extracts a subset of states relevant to its own field from the unified state vector and retrieves a knowledge subgraph relevant to its own field from the fisheries knowledge graph; Each specialized intelligent agent generates candidate actions based on the state subset and the knowledge subgraph, following the reasoning chain of phenomenon recognition, cause inference, action generation, and risk assessment. Each specialized intelligent agent organizes the basis for phenomenon identification, the basis for causal inference, the knowledge graph reference path, and the basis for action generation into a structured reason chain and outputs it.

[0013] Optionally, submitting candidate actions generated by each agent to the collaborative decision-making layer to obtain a joint control scheme that satisfies the consensus threshold includes: Conflict detection is performed on the candidate actions generated by each intelligent agent. The conflict detection includes judging whether there is a conflict from the dimensions of resource consumption, device timing and risk assessment. When any conflict in resource usage, device timing, or risk assessment is detected, cross-questioning is performed. The cross-questioning includes the collaborative decision-making layer initiating questions to the relevant agents in turn, and each agent returning supplementary explanations or correcting candidate actions based on the questions. A consistency score is performed on the candidate actions after questioning. The consistency score includes scoring the joint solution based on the consistency of knowledge graph references, the consistency of expected effect direction, and the feasibility of resource consumption. Determine whether the consistency score is greater than or equal to a preset consistency threshold; If the consensus score is greater than or equal to a preset consensus threshold, a joint control scheme is output. If the consensus score is determined to be less than the preset consensus threshold, the collaborative decision-making layer will feed back the conflict point to the relevant agent to request re-reasoning and return to the step of cross-questioning.

[0014] Optionally, consistency scoring of candidate actions after questioning includes: The knowledge graph reference consistency score is obtained according to formula (1). (1) in, This represents the number of key citations that are supported by multiple agents. This indicates the number of valid and conflict-free references. Indicates the number of contradictory or unclosed references. This indicates the total number of key citations used in the rating. , and Preset weights; According to formula (2), the consistency score of the expected effect direction is obtained. (2) in, This represents the weight of the k-th key indicator. The directional consistency coefficient of the kth key indicator; The resource occupancy feasibility score is obtained according to formula (3). (3) in, Indicates the percentage of resources exceeding the limit. Indicates the overlap ratio of time windows. This indicates the penalty for violating hard constraints. , and Preset weights.

[0015] Optionally, the safety verification of the joint control scheme includes equipment hard constraint verification, which includes: Based on the equipment capability boundary knowledge stored in the fishery knowledge graph, the action intensity of each action object in the joint control scheme is checked one by one to see if it exceeds the power limit, valve position boundary or start-stop frequency limit of the corresponding equipment, and whether the same equipment is unexecutable due to interlocking relationship or maintenance status restrictions within the verification time window. If any check fails, the joint control scheme is deemed to have failed the equipment hard constraint verification, and the specific actions and constraint types that do not meet the constraints are fed back to the collaborative decision-making layer.

[0016] Optionally, the safety verification of the joint control scheme includes process boundary verification, which includes: Verify whether the expected water exchange rate, feeding period, and dissolved oxygen level generated by the joint control scheme are within the process boundary range recorded in the fisheries knowledge graph; If the execution of the joint control scheme is expected to cause any process parameter to exceed its process boundary, then the process boundary check is deemed to have failed.

[0017] Optionally, the security verification of the joint control scheme includes counterfactual result verification, which includes: Call the control effect prediction model or digital twin simulation module, input the action sequence in the joint control scheme into the model or simulation module, and obtain the predicted trajectory of water quality indicators, fish status indicators and equipment load indicators within a future preset time window after the execution of the action sequence; The predicted trajectory is compared with the preset safety range. If any indicator exceeds its safety range within the preset time window, or if the prediction result shows that the current solution causes new secondary risks while eliminating the primary risks, it is determined that the counterfactual result verification has not been passed, and the specific indicator that the prediction exceeds the limit and the time of occurrence are fed back to the collaborative decision-making layer.

[0018] On the other hand, the present invention provides a fisheries adaptive control system based on a multimodal large model and multi-agent collaboration, implementing the fisheries adaptive control method based on a multimodal large model and multi-agent collaboration described in any of the above claims, wherein the system includes: A multimodal data access module is used to collect multimodal data from facility-based fisheries sites. A unified state representation module is used to uniformly encode the multimodal data to obtain a unified state vector; The knowledge graph construction and retrieval module is used to organize knowledge about equipment, processes, and disposal. A multi-agent cooperative reasoning module is used to generate candidate actions and coordinate conflicts to obtain a joint control scheme; A security verification module is used to perform security verification on the joint control scheme. The control orchestration module is used to convert the joint control scheme into a device-level control sequence.

[0019] The beneficial effects of this invention are: This invention overcomes the limitation of relying on a single perception link to form a global situational awareness by collecting multimodal data from aquaculture facilities and encoding it into a unified state vector. Simultaneously, it integrates heterogeneous information from multiple sources, including visual, water quality, equipment, and textual knowledge. Building upon this, it constructs a fisheries knowledge graph and establishes a multi-agent collaborative reasoning structure, breaking down different control objectives into specialized reasoning units. This enables each agent to generate candidate actions, risk assessments, and reasoning chains based on the unified state vector and knowledge graph, effectively reducing the risk of decision omissions in complex scenarios. By submitting the candidate actions generated by each agent to the collaborative decision-making layer and obtaining joint control schemes that meet the consistency threshold, conflict detection and cross-validation among multiple agents are completed before the scheme is issued, reducing the execution risk caused by unreliable single-generation results. Furthermore, by performing safety verification on the joint control scheme, it is ensured that the output control scheme meets the requirements of equipment constraints and process constraints. This distinguishes it from schemes that only output text suggestions. It can generate and execute executable, rollbackable, and auditable equipment-level control sequences, ultimately forming a complete control closed loop from multimodal perception, knowledge-driven reasoning, multi-agent collaborative decision-making to safety verification and execution.

[0020] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0021] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings: Figure 1 A flowchart of a fisheries adaptive control method based on multimodal large model and multi-agent collaboration according to an embodiment of the present invention; Figure 2 A flowchart illustrating a method for constructing a fisheries knowledge graph according to an embodiment of the present invention; Figure 3 To establish a multi-agent collaborative reasoning structure according to one embodiment of the present invention, each agent generates a flowchart of a candidate action, risk judgment, and reasoning chain method in its own professional field. Detailed Implementation

[0022] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the scope of the present invention.

[0023] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of this application all comply with relevant laws and regulations. In the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solution of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0024] like Figure 1 The diagram shows a flowchart of a fisheries adaptive control method based on a multimodal large model and multi-agent collaboration according to an embodiment of the present invention. Figure 1 The control method may include the following steps: In step S10, multimodal data of the facility-based fisheries site are collected; In step S11, the multimodal data is uniformly encoded to obtain a unified state vector; In step S12, a fisheries knowledge graph is constructed; In step S13, a multi-agent collaborative reasoning structure is established, and each agent generates candidate actions, risk judgments, and reasoning chains in its own professional field. In step S14, the candidate actions generated by each agent are submitted to the collaborative decision layer to obtain a joint control scheme that meets the consistency threshold. In step S15, a safety check is performed on the joint control scheme; In step S16, the joint control scheme that has passed the safety verification is converted into a device-level control sequence and executed.

[0025] In such Figure 1 In the illustrated multimodal large model and multi-agent collaborative adaptive control method for fisheries, step S10 is used to collect multimodal data from the facility fisheries site. In this embodiment, the multimodal data includes at least one or more of the following: fish school images or videos, water quality time series, equipment operation logs, feeding records, aquaculture batch information, manual inspection text, and equipment instruction documents.

[0026] Step S11 is used to uniformly encode multimodal data to form a basic state representation of fisheries. Specifically, image or video data is converted into visual features by a visual encoder, water quality time series and equipment logs are converted into dynamic state features by a time series encoder, and documents and inspection texts are converted into knowledge semantic features by a text encoder. A unified state vector is then obtained through contrastive learning or cross-modal alignment learning. Specifically, in this implementation, visual features, dynamic state features, and knowledge semantic features are first mapped to an intermediate vector space of the same dimension through their respective modal-specific feature projection layers, resulting in visual projection features, time series projection features, and text projection features. Then, different modal features from the same time window and the same field state are used as positive sample pairs, and modal features from different time windows or different field states are used as negative sample pairs. A contrastive learning loss function is used to shorten the distance between positive sample pairs in the semantic space and widen the distance between negative sample pairs, making different modal features (such as "fish schools clustering along the edge in images," "dissolved oxygen decreasing trend in water quality," and "abnormal aerator load in equipment logs") comparable and correlated in the unified semantic space. After the comparative learning training is completed, the visual projection features, temporal projection features and text projection features at the same time are further weighted and fused through the cross-modal attention module. The weighting weights are dynamically determined according to the confidence or reliability of each modality feature. The vector output after fusion is the unified state vector.

[0027] In this example, when the video recognition confidence is high but there are short-term gaps in the water quality time series data, the cross-modal attention module automatically reduces the weight of the time series projection features and increases the weight of the visual projection features, thereby obtaining a unified state vector that better reflects the current actual situation on site. When the feeding records and historical inspection texts in a certain aquaculture batch simultaneously indicate that the fish have shown abnormal feeding behavior, and the video image shows a decrease in the confidence of the visual features due to water turbidity, the cross-modal attention module correspondingly increases the weight of the text projection features to avoid distortion of the unified state vector due to a decrease in the quality of a single modality.

[0028] Step S12 is used to construct a fisheries knowledge graph. In this embodiment, the specific method for constructing the fisheries knowledge graph in step S12 can be of various forms known to those skilled in the art. In one example of the present invention, step S12 may include, for example... Figure 2 The steps shown are described. Figure 2 In this context, step S12 may include: In step S20, the multi-source heterogeneous aquaculture knowledge text is structured and parsed. Entities are extracted using named entity recognition and relation extraction methods, and the relationships between entities are identified. In step S21, the extracted entities and relations are organized into triples according to a predefined ontology schema; In step S22, the sequence of working conditions, handling actions, and handling results recorded in the historical handling cases is extracted into a case subgraph, and the handling action nodes in the case subgraph are associated with the existing equipment entities and action entities in the knowledge graph. In step S23, the automatically extracted results are verified and supplemented using expert rules to obtain a searchable fisheries knowledge graph.

[0029] In such Figure 2 In the method shown, step S20 is used to perform structured parsing of multi-source heterogeneous aquaculture knowledge text, extracting entities through named entity recognition and relation extraction methods, and identifying the relationships between entities. In this example, the multi-source heterogeneous aquaculture knowledge text includes one or more of aquaculture process documents, equipment manuals, operating procedures, historical treatment cases, and expert rules. Specifically, the aquaculture process documents, equipment manuals, operating procedures, historical treatment cases, and expert rules are first structuredly parsed. Named entity recognition and relation extraction methods are used to identify equipment entities, process parameter entities, risk type entities, and operational action entities from the text, and the subordinate, causal, precondition, and mutually exclusive relationships between entities are identified.

[0030] Step S21 organizes the extracted entities and relationships into triples according to a predefined ontology schema. In this example, the ontology schema includes at least four core relationship types: equipment-capacity boundary relationship, operating condition-risk relationship, action-precondition relationship, and action-prohibition relationship. Equipment capacity boundary is used to limit the maximum operating range of pumps, aerators, valves, and feeders; process causal knowledge is used to represent the changes in fish behavior that may be caused by a change in water quality; the order of actions is used to limit the order of action execution under specific risk scenarios; and prohibition knowledge is used to represent the combination of actions that cannot be executed under certain operating conditions.

[0031] Step S22 is used to extract the "operating condition phenomenon - handling action - handling result" sequence recorded in historical handling cases into a case subgraph, and establish the association between the handling action node in the case subgraph and the existing equipment entity and action entity in the knowledge graph to support the retrieval enhancement based on similar cases in the subsequent reasoning stage.

[0032] Step S23 is used to verify and supplement the automatically extracted results using expert rules to obtain a searchable fisheries knowledge graph. Specifically, in this example, erroneous or conflicting triples generated during the extraction process are removed, and domain experts supplement the constraints that are not explicitly expressed in the document but actually exist, forming a searchable fisheries knowledge graph. For example, for a certain type of aerator, the system parses triples from the equipment manual, such as "Aerator A - Capacity Boundary - Maximum Power" and "Aerator A - Interlock Relationship - Minimum Start-Stop Interval," and extracts triples from historical treatment cases, such as "Nighttime Low Oxygen Condition - Treatment Action - Increase Aerator A Output" and the corresponding treatment result record for that case. In the subsequent reasoning stage, the water quality agent can obtain the equipment's capacity boundary constraints and historical effective treatment path references by searching the subgraph that matches the current operating condition, without having to re-search the full text from the original document each time.

[0033] Step S13 is used to establish a multi-agent collaborative reasoning structure, whereby each agent generates candidate actions, risk judgments, and reasoning chains for its own professional domain. In this example, the multi-agent structure includes at least two or more of the following: a water quality agent, a feeding agent, an oxygenation agent, a disease risk agent, an energy consumption agent, and an execution supervision agent. Each agent uses a unified state vector and the corresponding subgraph in the fisheries knowledge graph as input to generate candidate actions, risk judgments, and reasoning chains for its own professional domain. In this embodiment, the specific method for each agent to generate candidate actions, risk judgments, and reasoning chains for its own professional domain in step S13 can be of various forms known to those skilled in the art. In one example of the present invention, step S13 may include, for example... Figure 3 The steps shown are described. Figure 3 In this context, step S13 may include: In step S30, each specialized intelligent agent extracts a subset of states relevant to its field from the unified state vector and retrieves a knowledge subgraph relevant to its field from the fisheries knowledge graph; In step S31, each specialized intelligent agent generates candidate actions based on the state subset and the knowledge subgraph, following the reasoning chain of phenomenon recognition, cause inference, action generation, and risk assessment. In step S32, each professional intelligent agent organizes the phenomenon identification basis, cause inference basis, knowledge graph reference path, and action generation basis into a structured reason chain and outputs it.

[0034] In such Figure 3In the illustrated method, step S30 is used to obtain a subset of states and a knowledge subgraph. Specifically, each specialized agent extracts a subset of states relevant to its own professional domain from a unified state vector, and retrieves a subgraph relevant to its domain from the fisheries knowledge graph. In this example, the knowledge subgraph may include the capability boundaries of operable equipment in the field, relevant process constraints, and historical disposal cases.

[0035] Step S31 is used to generate candidate actions based on the extracted state subset and the retrieved knowledge subgraph, following the reasoning chain of "phenomenon identification - cause inference - action generation - risk assessment". The phenomenon identification step is used to determine whether the current state deviates from the normal range of the field; the cause inference step is used to generate a list of possible causes by combining the process causal knowledge in the knowledge graph; the action generation step is used to retrieve candidate actions from the interconnected equipment relationships in the knowledge graph for the identified causes and to perform feasibility screening based on the current equipment capability boundaries; the risk assessment step is used to assess the risk level in the field that may be generated after the candidate action is executed.

[0036] Step S32 is used to organize the phenomenon identification basis, cause inference basis, knowledge graph reference path and action generation basis into a structured reason chain and output them together through professional intelligent agents, so that the collaborative decision-making layer and the security verification module can trace the generation basis of candidate actions.

[0037] In one embodiment of the present invention, the water quality agent identifies a continuous downward trend in dissolved oxygen in the unified state vector. Combining the process causal knowledge in the knowledge graph that "possible reasons for the decrease in dissolved oxygen include increased oxygen consumption load or decreased efficiency of aeration equipment", and further combining the fact that the aerator is currently under high load in the equipment log, it infers that the current cause is more likely to be increased oxygen consumption load rather than equipment failure. Thus, it generates two candidate actions: "increase aerator output" and "postpone the next round of feeding". The above identification, inference and retrieval basis are organized into a reason chain and output to the collaborative decision layer.

[0038] Step S14 is used to submit the candidate actions generated by each agent to the collaborative decision layer to obtain a joint control scheme that meets the consensus threshold. In this embodiment, the specific method for obtaining the joint control scheme that meets the consensus threshold in step S14 can be of various forms known to those skilled in the art. In one example of the present invention, step S14 may include: In step S40, conflict detection is performed on the candidate actions generated by each agent. Conflict detection includes determining whether there is a conflict from the dimensions of resource consumption, device timing and risk judgment. In step S41, when any type of conflict is detected in resource occupation, device timing, and risk judgment, cross-questioning is performed. Cross-questioning includes the collaborative decision-making layer initiating questions to the relevant intelligent agents in turn, and each intelligent agent returns supplementary explanations or corrects candidate actions based on the questions. In step S42, a consistency score is performed on the candidate actions after the questioning. The consistency score includes scoring the joint solution based on the consistency of knowledge graph references, the consistency of expected effect direction, and the feasibility of resource consumption. In step S43, it is determined whether the consistency score is greater than or equal to the preset consistency threshold; In step S44, if the consistency score is greater than or equal to the preset consistency threshold, the joint control scheme is output. If the consensus score is less than the preset consensus threshold, the collaborative decision-making layer will feed back the conflict point to the relevant agents to request re-reasoning and return to the execution of the cross-questioning step.

[0039] Step S40 is used to perform conflict detection on the candidate actions generated by each agent. Specifically, in this example, the collaborative decision-making layer compares the candidate actions submitted by each agent pairwise, judging whether there is a conflict from three dimensions: resource occupation, equipment timing, and risk assessment. Specifically, if two candidate actions involve the same equipment and their expected occupation time windows overlap, or if the sum of the expected occupation of the same type of resource by two candidate actions, such as total power or water exchange flow, exceeds the system's allowed limit, then a resource occupation conflict is determined to exist. If the knowledge graph records the order of actions, requiring one type of action to be executed before another, and the execution order in the candidate scheme contradicts this, then an equipment timing conflict is determined to exist. If different agents give inconsistent inferences about the cause or risk level of the same phenomenon, and this inconsistency will lead to disagreements in the selection of subsequent actions, then a risk assessment conflict is determined to exist.

[0040] Step S41 involves the collaborative decision-making layer generating a challenge task sheet for each conflict point when any type of conflict—resource occupancy, device timing, or risk assessment—is detected. The challenge task sheet includes at least the conflict type, conflict object, involved device or resource, expected execution time window, and the identifier of the challenged agent. The collaborative decision-making layer initiates challenges to relevant agents sequentially from high to low conflict correlation. The conflict correlation of the i-th conflict point... It can be represented as: ; in, Indicates the number of agents or actions involved in the conflict. This indicates the core-level weight of the resource corresponding to the conflict in the knowledge graph. This indicates the urgency of the time when the conflict is expected to occur relative to the current time. This indicates the potential risk level value corresponding to the unresolved conflict. , , and These are the weighting coefficients.

[0041] Request the challenged agent to return the set of preconditions for candidate actions. Knowledge graph reference paths corresponding to each prerequisite. Resource occupancy vector of candidate actions and the expected effect vector of candidate actions Among them, the resource occupancy vector It should include at least one or more of the following: device identifier, start time, duration, power consumption, and traffic consumption; Expected effect vector. It should include at least one or more of the following: target indicator, direction of change, target magnitude, and time required to reach the target. When the conflict type is a risk judgment conflict, the collaborative decision-making layer also requests the challenged agent to explain its basis for rejecting other candidate actions and the corresponding counter-evidence path. After receiving the challenge, each agent backtracks its original reasoning chain, supplements missing evidence, corrects preconditions, or adjusts the intensity, execution order, and duration of actions, and returns the supplemented or corrected candidate actions.

[0042] Step S42 is used to perform consistency scoring on the candidate actions after questioning. In this embodiment, the collaborative decision-making layer constructs a joint scheme based on the candidate action set supplemented by each agent, and calculates the knowledge graph reference consistency score for each. Consistency score of expected results and resource utilization feasibility score .

[0043] Specifically, in this example, the knowledge graph reference consistency score Based on the consistency of the key facts, constraints, and case paths referenced by each candidate action, it can be expressed as: ; in, This represents the number of key citations that are supported by multiple agents. This indicates the number of valid and conflict-free references. Indicates the number of contradictory or unclosed references. This indicates the total number of key citations used in the rating. , and Preset weights.

[0044] In this example, the expected outcome direction consistency score Based on the direction and timing of the influence of each candidate action on the same target indicator, it can be expressed as: ; in, This represents the weight of the k-th key indicator. Let be the directional consistency coefficient of the k-th key indicator. In this embodiment, The value is 1 when all agents act in the same direction on this indicator, 0.5 when compatibility can be achieved through time-series splitting, and 0 when the directions are opposite and cannot be coordinated.

[0045] In this example, the resource usage feasibility score Based on equipment capacity margin, resource overlap ratio, and hard constraint violation status, it can be expressed as: ; in, Indicates the percentage of resources exceeding the limit. Indicates the overlap ratio of time windows. This indicates the penalty for violating hard constraints. , and Preset weights.

[0046] Then according to the preset weights and Regarding the , and We perform a weighted aggregation to obtain the consistency score: ; in, .

[0047] Step S43 is used to determine whether the consistency score is greater than or equal to a preset consistency threshold. Step S44 is used to perform the next operation based on the determination result. Specifically, in this example, when the consistency score reaches the preset consistency threshold, a joint control scheme is output. When the consistency score does not reach the threshold, the collaborative decision-making layer feeds back the conflict point to the relevant agents, requiring them to re-reason, repeat the cross-questioning and scoring process, until the consistency threshold is reached or the preset maximum number of iterations is reached. If the consistency threshold is still not met after reaching the maximum number of iterations, the collaborative decision-making layer outputs a risk warning and transfers the process to manual decision-making.

[0048] Step S15 is used to perform a safety verification of the joint control scheme. In this embodiment, the safety verification includes at least equipment hard constraint verification, process boundary verification, and counterfactual result verification. Specifically, in this example, in the equipment hard constraint verification step, the safety verification module verifies, one by one, whether the action intensity of each action object in the joint control scheme exceeds the power limit, valve position boundary, or start / stop frequency limit of the corresponding equipment based on the equipment capability boundary knowledge stored in the knowledge graph, and verifies whether the same equipment is unexecutable due to interlocking relationships or maintenance status restrictions within the verification time window. If any verification fails, the joint control scheme is determined to have failed the equipment hard constraint verification, and the specific actions and constraint types that do not meet the constraints are fed back to the collaborative decision-making layer.

[0049] In this example, during the process boundary verification process, the safety verification module checks whether the expected water exchange rate, feeding period, and dissolved oxygen level generated by the joint control scheme fall within the process boundary range recorded in the knowledge graph. If the scheme is expected to cause the process parameters to exceed the boundary after execution, it is determined that the process boundary verification has failed.

[0050] In this example, during the counterfactual result verification phase, the safety verification module first uses the current unified state vector, the action sequence in the joint control scheme, the real-time state of the equipment, and the environmental boundary conditions as inputs to construct a baseline scenario where the joint control scheme is not executed and an intervention scenario where the joint control scheme is executed. Then, it calls the control effect prediction model or the digital twin simulation module to obtain the baseline trajectories of the two scenarios within a preset future time window T. and intervention trajectory The trajectory includes at least one or more of the following: dissolved oxygen, temperature, ammonia nitrogen, feeding activity, swimming stress index, equipment power, equipment temperature rise, or valve position load. Subsequently, for each indicator... Calculate safety margin and risk improvement : ; in, These are the lower and upper safety limits for the indicator, respectively. The improvement direction of the primary risk corresponding indicator in the intervention scenario relative to the baseline scenario is determined, and it is used to characterize whether the joint control scheme effectively mitigates the primary risk. If there exists any time t such that... If the value is less than 0, the indicator is determined to be outside the safe range at that time; if the indicator corresponding to the primary risk is less than 0, the indicator is considered to be outside the safe range at that time. If the preset improvement threshold is not met, the current solution is deemed ineffective in reducing the primary risk. Based on this, the safety verification module further calculates the secondary risk score according to the intervention trajectory. , can be represented as: ; in, This represents the width of the safe interval for the j-th indicator. This indicates whether the h-th type of secondary risk event has been triggered. and The preset weights are used. When If the risk exceeds the preset secondary risk threshold, or if the prediction results show that the joint control scheme, while mitigating the primary risk, will cause any of the following risks: dissolved oxygen fluctuations, increased fish stress, equipment overheating, excessive total power, or excessive water exchange disturbance, the scheme is deemed to have failed the counterfactual result verification. The exceeding indicators, the first exceeding time, the corresponding action segment, and the suggested correction direction are then fed back to the collaborative decision-making layer for the agent to re-reason. The scheme is considered to have passed the safety verification only if it simultaneously passes the equipment hard constraint verification, process boundary verification, and counterfactual result verification.

[0051] For example, for the candidate action of "increasing the output of the aerator", the counterfactual result verification step estimates the dissolved oxygen recovery trajectory and the aerator load change trajectory in the future period through the control effect prediction model. If the prediction result shows that the temperature rise of the aerator will approach its maintenance limit under continuous high load operation, the verification step determines that the plan has failed and prompts the collaborative decision-making level to change to increasing the aerator output in stages or reducing the feeding load at the same time.

[0052] Step S16 is used to convert the safety-verified joint control scheme into a device-level control sequence and execute it. In this embodiment, the device-level control sequence includes at least the action object, action sequence, action intensity, action duration, execution preconditions, and rollback conditions. Specifically, firstly, the control orchestration module parses the action objects of each candidate action included in the joint control scheme to determine the specific execution device identifier corresponding to the action. Then, based on the disposal sequence knowledge in the knowledge graph and the dependencies between each action, multiple actions are sorted to determine the action execution sequence. For each action, the specific action intensity value is determined based on the intensity range marked during the candidate action generation stage and the current device state, combined with the device capability boundary, and the action duration is determined based on the risk urgency and process requirements. At the same time, execution preconditions are set for each action, such as requiring the previous action to be completed or a specific sensor reading to reach a preset range before execution, and rollback conditions, such as stopping the current action and restoring to the pre-execution state if a key indicator exceeds the safety range during execution, or switching to a preset emergency action. The final generated device-level control sequence is output in the form of an action list. Each action record in the list includes six types of information: action object, action sequence number, action intensity, action duration, execution preconditions, and rollback conditions. The execution module calls the corresponding device interface in sequence to execute the action. Before calling, the execution module will also check whether the current actual state of the device meets the preconditions of the action. If not, the execution will be suspended and an abnormal status information will be reported.

[0053] Finally, the device-level control sequence is executed, and feedback data is collected after execution. The feedback data is used to update the unified state vector, correct the experience edges in the knowledge graph, and optimize the agent weights, thereby forming a closed-loop learning.

[0054] Example 1: Joint response to the risk of nighttime hypoxia in recirculating aquaculture systems First, the hardware is configured, and the system is connected to the workshop video stream, water quality sensor, aerator controller, circulating pump log, automatic feeder status and manual inspection terminal.

[0055] The software parameters include: a unified state representation module that continuously updates the field state vector; multiple agents including a water quality agent, an oxygenation agent, a feeding agent, and an execution supervision agent; and a safety verification module that calls a control effect prediction model to estimate changes in dissolved oxygen, fish aggregation, and equipment load over a future period.

[0056] Next, follow these steps: Step 1: The system identifies that the fish are clustered along the edge of the video and that the dissolved oxygen in the water quality time series shows a decreasing trend.

[0057] Specifically, the unified state representation module merges the visual feature of "fish clustering along the edge" output by the visual encoder at that moment with the dynamic state feature of "dissolved oxygen continuously decreasing" output by the temporal encoder after cross-modal alignment into a unified state vector; the execution supervisory agent continuously monitors the unified state vector, and when the unified state vector exceeds its respective historical normal fluctuation range in both the degree of clustering along the edge and the rate of decrease in dissolved oxygen, the subsequent multi-agent inference process is triggered.

[0058] Step 2: The water quality agent proposes candidate actions to increase oxygenation intensity, and the feeding agent proposes candidate actions to postpone the next round of feeding.

[0059] Specifically, the water quality agent extracts a subset of states related to dissolved oxygen from the unified state vector and retrieves the corresponding process causal subgraph in the knowledge graph for "dissolved oxygen decrease - possible cause - increased oxygen load or decreased efficiency of aeration equipment". Combining this with equipment logs, it determines that the aerator has not yet reached full load. Based on this, it generates a candidate action of "increasing aerator output" according to the "phenomenon identification - cause inference - action generation - risk assessment" chain, and labels the rationale chain as "dissolved oxygen decreasing trend + aerator still has spare capacity + effective handling of similar historical cases". Simultaneously, the feeding agent extracts a subset of states related to feeding and stress from the unified state vector and, combined with the process causal knowledge in the knowledge graph that "feeding under low oxygen conditions may exacerbate oxygen load", generates a candidate action of "postponing the next round of feeding" and labels the corresponding rationale chain. While generating candidate actions, both agents provide risk assessments for their respective domains: the water quality agent determines that if aeration is not timely, dissolved oxygen may fall below the safe lower limit within a preset time window; the feeding agent determines that maintaining the original feeding plan may further increase oxygen load.

[0060] Step 3: The collaborative decision-making layer conducts cross-questioning and consistency scoring on the two actions.

[0061] Specifically, the collaborative decision-making layer first performs conflict detection on the two candidate actions, determining that "increasing aerator output" and "delaying feeding" do not conflict in terms of resource consumption and equipment timing, but there is a potential inconsistency in risk assessment: the water quality agent believes that aeration should be prioritized, while the feeding agent believes that reducing oxygen consumption should be prioritized. Based on this, the collaborative decision-making layer initiates cross-questioning, requiring the water quality agent to explain whether increasing aeration output would exacerbate fish stress due to violent agitation of the water in the short term, and requiring the feeding agent to explain whether delaying feeding would offset the load reduction effect due to hunger stress. Both agents supplement evidence based on historical treatment cases in the knowledge graph, confirming that the combined execution of the two actions will not cancel each other out. Based on this, the collaborative decision-making layer scores the actions from three dimensions: consistency of knowledge graph references, consistency of expected effect direction, and feasibility of resource consumption. After the weighted summation of the three scores reaches the preset consistency threshold, "increasing aeration output first, then delaying feeding" is determined as the preliminary content of the joint control scheme and submitted to the safety verification module.

[0062] Step 4: The safety verification module verifies the aerator load, start-stop frequency, and water change boundary.

[0063] Specifically, the safety verification module sequentially performs equipment hard constraint verification to check whether the increased aeration output exceeds the power limit, start-stop frequency limit, and process boundary verification of the aerator model. It also verifies that the water exchange rate during the increased output period is still within the process boundary range of the pond. The module then calls the control effect prediction model to perform counterfactual result verification: the action sequence of "increasing aeration output first, then delaying feeding" is input into the model to obtain the predicted results of dissolved oxygen recovery trajectory, fish aggregation degree change trajectory, and aerator load change trajectory within the preset time window. After comparing the predicted results with the preset safety range, it is confirmed that dissolved oxygen can recover to a safe level within the preset time window, the aerator load is not close to the maintenance limit, and there will be no secondary stress risk caused by delayed feeding. If all three verifications are passed, the plan enters the distribution stage.

[0064] Step 5: Once the plan has passed verification, issue the control sequence of first increasing oxygen, then evaluating, and then resuming feeding.

[0065] Specifically, the control orchestration module converts the verified joint control scheme into an equipment-level control sequence: The first action is to increase the output of the aerator, with the target being the corresponding aerator. The precondition for execution is that the current dissolved oxygen is below a preset lower limit. The duration of the action is determined based on the predicted dissolved oxygen recovery trajectory in the counterfactual result verification. The rollback condition is that if the dissolved oxygen recovery rate is significantly lower than the predicted trajectory during execution, a manual prompt is triggered. The second action is to delay feeding, with the target being the corresponding automatic feeder. The precondition for execution is that the dissolved oxygen has recovered to a preset threshold after the first action is executed. The rollback condition is that if the delay time exceeds a preset upper limit, the original feeding plan is automatically restored and a manual attention is prompted. Due to the high reliability of this scheme and the urgency of the risk, the execution supervision module issues this control sequence according to the automatic execution level in the tiered execution strategy.

[0066] Step 6: After execution, continuously monitor the feedback results and adjust the agent weights based on the feedback.

[0067] Specifically, after the control sequence is issued, the execution monitoring module continuously compares the actual sensor feedback values ​​with the predicted trajectory during the counterfactual result verification stage: if the actual dissolved oxygen recovery rate matches the predicted trajectory, it indicates that the aerator output adjustment has achieved the expected effect; if the actual recovery rate is significantly lower than the predicted trajectory, the execution monitoring module records the deviation and feeds the deviation information back to the water quality intelligence agent, which then determines whether supplementary intervention needs to be triggered, such as further increasing the aeration output or notifying manual inspection of the aerator status. After the handling is completed and the risk is eliminated, the closed-loop feedback module writes the triggering characteristics of this event, namely the unified state vector snapshot, joint control scheme, safety verification results, and actual handling effects, including the deviation between the actual and predicted dissolved oxygen recovery trajectories and whether rollback conditions were triggered, into the historical handling case database, and updates the handling result field of the corresponding case subgraph in the knowledge graph. At the same time, the closed-loop feedback module updates the inference weights of each agent online based on the handling results: if the prediction of a candidate action of an agent in this handling is highly consistent with the actual result, the inference weight of that agent in subsequent similar scenarios is appropriately increased; if the deviation is large, the weight is appropriately reduced and a consistency check of the knowledge graph reference link of that agent is triggered for subsequent improvement.

[0068] Expected results: This embodiment helps to transform the on-site handling that originally relied on the experience of the duty officer into a traceable collaborative control process.

[0069] Compared to traditional methods that rely on a single rule base to trigger alarms or on the experience of on-duty personnel, the method described in this embodiment has at least the following differences in the same nighttime low-oxygen scenario: First, traditional methods typically trigger alarms only based on the single indicator of dissolved oxygen exceeding the limit, while this embodiment incorporates visual cues of fish gathering along the edge in the video and the trend of decreasing dissolved oxygen by using a unified state vector, enabling earlier identification of potential risks before dissolved oxygen exceeds the alarm threshold; Second, in traditional methods, oxygenation and feeding adjustments are usually decided manually, easily overlooking the mutual influence between the two, while this embodiment completes cross-domain verification before issuing commands through a multi-agent cross-questioning mechanism; Third, traditional methods usually lack prior estimation of the execution consequences before issuing commands, while this embodiment predicts the equipment load and water quality trajectory after execution before issuing commands through counterfactual result verification, reducing secondary risks.

[0070] Example 2: Conflict scheduling of multiple devices in an integrated aquaculture workshop First, the hardware configuration is carried out, and the system is connected to multiple pool video, water quality sensors, feeders, aerators, water supply valves, sewage discharge equipment and energy consumption monitoring modules.

[0071] The software parameters include: in addition to water quality and feeding agents, the multi-agent system also includes energy consumption agents and disease risk agents; the knowledge graph stores nighttime power limit rules, equipment interlock relationships, and special operating condition prohibition conditions.

[0072] Next, follow these steps: Step 1: The disease risk agent should reduce stress based on abnormal behavior and inspection text prompts.

[0073] Specifically, the disease risk agent extracts subsets related to behavioral phenotypes and textual descriptions from the unified state vector: visual features detect localized reddening areas on the fish body surface and some individuals swimming at significantly lower speeds than the group baseline; simultaneously, the text encoder extracts semantic features from the most recent inspection record, "some fish have scratches on their bodies and decreased feeding activity," indicating a risk signal of parasitic or bacterial stress. The disease risk agent retrieves the causal subgraph of "abnormal body surface + decreased feeding → increased fish stress risk" from the knowledge graph, combines it with historical treatment cases showing that "large-scale water changes or high-noise equipment startup during stress will increase mortality," generates candidate actions such as "it is recommended to reduce external stimuli during the current breeding cycle, avoid drastic water changes and long-term operation of high-power equipment," and outputs the corresponding reason chain to submit to the collaborative decision-making layer.

[0074] Step 2: The feeding intelligence suggests reducing the amount of feed, the water quality intelligence suggests changing the water in stages, and the energy consumption intelligence suggests using high-power equipment during off-peak hours.

[0075] Specifically, the three agents independently complete domain-specific reasoning and simultaneously submit candidate actions and justification chains: the feeding agent combines the signal of decreased feeding activity in the unified state vector and the process knowledge of "overfeeding during stress will increase metabolic burden" in the knowledge graph to generate the candidate action "reduce the current feeding amount to an appropriate proportion of the normal amount" and annotates the justification chain, which consists of abnormal behavior signals, process causal basis and historical cases; the water quality agent combines the current slightly high ammonia nitrogen proxy characteristics and the fish stress background, and searches the knowledge graph for the process experience of "staged water change can reduce the impact of a single water change on fish stress" to generate the candidate action "break the current water change plan into several small-batch water changes and disperse them during the daytime period of the day"; the energy consumption agent combines the nighttime power limit rules in the knowledge graph and the current power load status to search for the restriction rule "off-peak use of high-power equipment can avoid exceeding the nighttime total power limit" to generate the candidate action "stagger the start time of high-power circulation pump and ultraviolet sterilizer and adjust it to the off-peak period of the day". The three sets of candidate actions, along with their respective justification chains, are submitted simultaneously to the collaborative decision-making layer.

[0076] Step 3: The collaborative decision-making layer identifies conflicts between water exchange and certain equipment cleaning periods, requiring the relevant agents to re-reason.

[0077] Specifically, the collaborative decision-making layer performs timing conflict detection on the phased water exchange action proposed by the water quality agent and the staggered activation of high-power equipment proposed by the energy consumption agent. It finds an overlap between the sewage equipment cleaning window recorded in the knowledge graph and the expected time window for the water exchange action, indicating a timing conflict. Based on this, the collaborative decision-making layer initiates cross-questioning with both the water quality agent and the energy consumption agent, requiring them to explain the adjustable range of their respective actions. The water quality agent provides a shiftable time window range based on the water exchange process boundary in the knowledge graph, while the energy consumption agent provides a schedulable time window range for high-power equipment based on the nighttime power limitation rules in the knowledge graph. The collaborative decision-making layer then coordinates the two agents to execute the actions within non-overlapping time windows, re-evaluates the consistency score, and submits the adjusted plan to the security verification module after confirming that it meets the preset consistency threshold.

[0078] Step 4: After the security verification module passes, a time-segmented device control sequence is generated, and whether to execute it automatically is determined based on the reliability.

[0079] Specifically, the safety verification module sequentially performs equipment hard constraint verification, process boundary verification, and counterfactual result verification on the joint control scheme output by the collaborative decision-making layer: Equipment hard constraint verification confirms that the power, valve position, and start / stop frequency of each equipment action are within the equipment capacity boundaries recorded in the knowledge graph, and that the total power of high-power equipment in the peak-shifting plan does not exceed the nighttime limit; Process boundary verification confirms that the single water change rate of the phased water change is within the process boundary range corresponding to the current fish status, and that the amount of reduced feeding is not lower than the minimum daily feed ratio required to maintain the fish's basal metabolism; Counterfactual result verification uses the control effect prediction model to simulate the predicted trajectory of fish stress indicators, ammonia nitrogen proxy characteristics, and equipment load within a future preset time window after the scheme is executed, in order to confirm that it will not cause secondary risks; After all three verifications pass, the control orchestration module converts the joint scheme into a time-segmented equipment control sequence, in which each action includes the execution time, action object, action intensity, execution preconditions, and rollback conditions. In this embodiment, since the judgment basis for disease stress risk includes subjective inspection text, the system determines that the reasoning path of reducing feeding related to disease risk in this plan has medium credibility, and adopts the "push for manual confirmation and then execute" mode for this sub-plan; the reasoning chain of the water change and staggered scheduling sub-plans is complete and the knowledge graph reference is clear, with a high consistency score, and adopts the "automatic execution" mode, and pushes the operation notification to the aquaculture management terminal at the same time.

[0080] Step 5: After execution, write the results back to the closed-loop feedback module.

[0081] Specifically, after the execution of this multi-equipment conflict scheduling scheme, the execution monitoring module continuously collects feedback data during and after execution: recording the actual execution time and actual water exchange volume of each sub-batch of water exchange, comparing it with the time and water exchange volume agreed in the control sequence to confirm whether the rollback condition has been triggered; recording the actual power curve of high-power equipment after peak-shaving start-up to confirm whether the nighttime power limit constraint has been met; and recording the execution status of reduced feeding and the confirmation operation time of the aquaculture personnel. All the above execution records, along with the triggering scenarios of this scheme, including a unified state vector snapshot of disease stress signals, ammonia nitrogen proxy characteristics, and equipment status, the reasoning paths of each agent, the conflict resolution process of the collaborative decision-making layer, the safety verification results, and the actual execution effect, including the deviation between the actual change trajectory and the predicted trajectory of each key indicator, are written into the historical treatment case database. The closed-loop feedback module updates the weights of each agent online based on the execution results: if the predictions of the energy consumption agent and the water quality agent are highly consistent with the actual results, their weights are appropriately increased; if the reduced feeding suggestions of the disease risk agent are confirmed by the aquaculture personnel as reasonable afterward, the contribution weight of the text semantic features to the risk judgment in that scenario is increased accordingly.

[0082] Expected results: This embodiment is beneficial for balancing water quality safety, fish stress control, and energy consumption constraints under complex operating conditions.

[0083] On the other hand, this invention also provides a fisheries adaptive control system based on a multimodal large model and multi-agent collaboration, implementing the method described in any one of the fisheries adaptive control methods based on a multimodal large model and multi-agent collaboration. The system includes a multimodal data access module, a unified state representation module, a knowledge graph construction and retrieval module, a multi-agent collaborative reasoning module, a safety verification module, and a control orchestration module. The multimodal data access module is used to collect multimodal data from the facility fisheries site. The unified state representation module is used to uniformly encode the multimodal data to obtain a unified state vector. The knowledge graph construction and retrieval module is used to organize equipment, process, and disposal knowledge. The multi-agent collaborative reasoning module is used to generate candidate actions and coordinate conflicts to obtain a joint control scheme. The safety verification module is used to perform safety verification on the joint control scheme. The control orchestration module is used to convert the joint control scheme into an equipment-level control sequence.

[0084] The beneficial effects of this invention are: This invention employs a unified multimodal state representation, simultaneously fusing heterogeneous information from multiple sources, including visual, water quality, equipment, and textual knowledge, overcoming the limitation of a single perception link in forming a global situational awareness. Through a multi-agent collaborative reasoning structure, this invention decomposes different control objectives such as water quality, feeding, aeration, disease control, and energy consumption into specialized reasoning units. Each agent independently generates candidate solutions, which are then converged at the collaborative decision-making layer, effectively reducing the risk of decision omissions in complex scenarios. The conflict detection, cross-questioning, and consistency scoring mechanisms introduced at the collaborative decision-making layer enable internal cross-validation before control schemes are issued, reducing the execution risk caused by unreliable single-generation results from individual models. Through a safety verification mechanism comprised of equipment hard constraint verification, process boundary verification, and counterfactual result verification, this invention ensures that the output control scheme meets equipment capability limitations, process parameter ranges, and future state safety requirements. Unlike schemes that only output textual suggestions, this invention can generate executable, rollbackable, and auditable equipment-level control sequences. The feedback data collected after execution is further used to update the fisheries knowledge graph and the decision weights of various professional intelligent agents, forming a complete closed loop from perception, decision-making, execution to learning, which has the ability to continuously optimize compared to single recommendation solutions.

[0085] In summary, compared with existing centralized optimization schemes, the core innovation of this invention lies in constructing a combined control architecture that integrates unified multimodal representation, multi-agent collaborative reasoning, pre-execution dual verification, and post-execution closed-loop learning.

[0086] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0087] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0088] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0089] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0090] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0091] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0092] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0093] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0094] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A fisheries adaptive control method based on multimodal large model and multi-agent collaboration, characterized in that, The control method includes: Collect multimodal data from facility-based fisheries sites; The multimodal data is uniformly encoded to obtain a unified state vector; Constructing a fisheries knowledge graph; Establish a multi-agent collaborative reasoning structure, in which each agent generates candidate actions, risk judgments, and reasoning chains in its own professional field; Each agent submits its candidate actions to the collaborative decision-making layer to obtain a joint control scheme that meets the consensus threshold. The joint control scheme is subjected to security verification; The joint control scheme that has passed safety verification is converted into a device-level control sequence and executed.

2. The control method according to claim 1, characterized in that, Unified encoding of the multimodal data to obtain a unified state vector includes: Visual features, dynamic state features, and knowledge semantic features are mapped to intermediate vector spaces of the same dimension through feature projection layers corresponding to their respective modalities, resulting in visual projection features, temporal projection features, and text projection features. Different modal features from the same time window and the same scene state are used as positive sample pairs, and modal features from different time windows or different scene states are used as negative sample pairs. By comparing and learning the loss function, the distance between positive sample pairs in the semantic space is shortened, and the distance between negative sample pairs is widened. The cross-modal attention module performs weighted fusion of visual projection features, temporal projection features, and text projection features at the same time to obtain a unified state vector, where the weights are dynamically determined based on the confidence or reliability of each modal feature.

3. The control method according to claim 1, characterized in that, Constructing a fisheries knowledge graph includes: The multi-source heterogeneous aquaculture knowledge text is structured and parsed. Entities are extracted using named entity recognition and relation extraction methods, and the relationships between entities are identified. The extracted entities and relations are organized into triples according to a predefined ontology schema; Extract the sequence of working conditions, actions, and results recorded in historical cases into case subgraphs, and establish associations between the action nodes in the case subgraphs and the existing equipment entities and action entities in the knowledge graph. The automatically extracted results are validated and supplemented using expert rules to obtain a searchable fisheries knowledge graph.

4. The control method according to claim 1, characterized in that, Establish a multi-agent collaborative reasoning structure, where each agent generates candidate actions, risk assessments, and reasoning chains specific to its domain, including: Each specialized intelligent agent extracts a subset of states relevant to its own field from the unified state vector and retrieves a knowledge subgraph relevant to its own field from the fisheries knowledge graph; Each specialized intelligent agent generates candidate actions based on the state subset and the knowledge subgraph, following the reasoning chain of phenomenon recognition, cause inference, action generation, and risk assessment. Each specialized intelligent agent organizes the basis for phenomenon identification, the basis for causal inference, the knowledge graph reference path, and the basis for action generation into a structured reason chain and outputs it.

5. The control method according to claim 1, characterized in that, The candidate actions generated by each agent are submitted to the collaborative decision-making layer to obtain a joint control scheme that meets the consensus threshold, including: Conflict detection is performed on the candidate actions generated by each intelligent agent. The conflict detection includes judging whether there is a conflict from the dimensions of resource consumption, device timing and risk assessment. When any conflict in resource usage, device timing, or risk assessment is detected, cross-questioning is performed. The cross-questioning includes the collaborative decision-making layer initiating questions to the relevant agents in turn, and each agent returning supplementary explanations or correcting candidate actions based on the questions. A consistency score is performed on the candidate actions after questioning. The consistency score includes scoring the joint solution based on the consistency of knowledge graph references, the consistency of expected effect direction, and the feasibility of resource consumption. Determine whether the consistency score is greater than or equal to a preset consistency threshold; If the consensus score is greater than or equal to a preset consensus threshold, a joint control scheme is output. If the consensus score is determined to be less than the preset consensus threshold, the collaborative decision-making layer will feed back the conflict point to the relevant agent to request re-reasoning and return to the step of cross-questioning.

6. The control method according to claim 5, characterized in that, Consistency scoring of candidate actions after questioning includes: The knowledge graph reference consistency score is obtained according to formula (1). ，（1） in, This represents the number of key citations that are supported by multiple agents. This indicates the number of valid and conflict-free references. Indicates the number of contradictory or unclosed references. This indicates the total number of key citations used in the rating. , and Preset weights; According to formula (2), the consistency score of the expected effect direction is obtained. ，（2） in, This represents the weight of the k-th key indicator. The directional consistency coefficient of the kth key indicator; The resource occupancy feasibility score is obtained according to formula (3). ，（3） in, Indicates the percentage of resources exceeding the limit. Indicates the overlap ratio of time windows. This indicates the penalty for violating hard constraints. , and Preset weights.

7. The control method according to claim 1, characterized in that, The safety verification of the joint control scheme includes equipment hard constraint verification, which includes: Based on the equipment capability boundary knowledge stored in the fishery knowledge graph, the action intensity of each action object in the joint control scheme is checked one by one to see if it exceeds the power limit, valve position boundary or start-stop frequency limit of the corresponding equipment, and whether the same equipment is unexecutable due to interlocking relationship or maintenance status restrictions within the verification time window. If any check fails, the joint control scheme is deemed to have failed the equipment hard constraint verification, and the specific actions and constraint types that do not meet the constraints are fed back to the collaborative decision-making layer.

8. The control method according to claim 1, characterized in that, The safety verification of the joint control scheme includes process boundary verification, which includes: Verify whether the expected water exchange rate, feeding period, and dissolved oxygen level generated by the joint control scheme are within the process boundary range recorded in the fisheries knowledge graph; If the execution of the joint control scheme is expected to cause any process parameter to exceed its process boundary, then the process boundary check is deemed to have failed.

9. The control method according to claim 1, characterized in that, The security verification of the joint control scheme includes counterfactual result verification, which includes: Call the control effect prediction model or digital twin simulation module, input the action sequence in the joint control scheme into the model or simulation module, and obtain the predicted trajectory of water quality indicators, fish status indicators and equipment load indicators within a future preset time window after the execution of the action sequence; The predicted trajectory is compared with the preset safety range. If any indicator exceeds its safety range within the preset time window, or if the prediction result shows that the current solution causes new secondary risks while eliminating the primary risks, it is determined that the counterfactual result verification has not been passed, and the specific indicator that the prediction exceeds the limit and the time of occurrence are fed back to the collaborative decision-making layer.

10. A fisheries adaptive control system based on multimodal large model and multi-agent collaboration, implementing the fisheries adaptive control method based on multimodal large model and multi-agent collaboration as described in any one of claims 1-9, characterized in that, The system includes: A multimodal data access module is used to collect multimodal data from facility-based fisheries sites. A unified state representation module is used to uniformly encode the multimodal data to obtain a unified state vector; The knowledge graph construction and retrieval module is used to organize knowledge about equipment, processes, and disposal. A multi-agent cooperative reasoning module is used to generate candidate actions and coordinate conflicts to obtain a joint control scheme; A security verification module is used to perform security verification on the joint control scheme. The control orchestration module is used to convert the joint control scheme into a device-level control sequence.

Images