Quality evaluation driven offshore target multi-modal information collection agent system and method

Abstract

The embodiment of the application provides a kind of quality evaluation driven offshore target multi-modal information acquisition agent system and method, belong to multi-modal information acquisition field.The system includes: task interaction and constraint module, central arrangement agent module, for generating collection execution plan and carrying out task arrangement scheduling;Multi-modal information acquisition execution module, multi-modal analysis and alignment module, for data analysis, structured processing and cross-modal alignment;Execution stable control module, for the quality change of structured multi-modal result is converted into the control signal of collection execution process, and difference type adjustment is carried out to collection execution parameter vector under constraint condition;Data storage and traceability module.By introducing the execution stable control module based on quality evaluation driven, the quality change of collection result is used as endogenous control signal, and combining attribution mapping, parameter subspace constraint and difference type execution adjustment mechanism, the long-term stable operation of multi-modal information acquisition process is realized.

Description

Technical Field

[0001] This invention relates to the field of multimodal information acquisition technology, and more specifically to a quality assessment-driven intelligent agent system and method for acquiring multimodal information about maritime targets. Background Technology

[0002] With the development of applications such as maritime situational awareness, vessel target identification, target behavior analysis, and intelligent model training, the demand for information acquisition of maritime targets such as vessels is constantly increasing. To meet these needs, various multimodal information acquisition systems have emerged in the existing technology. These systems typically configure multiple heterogeneous data sources, such as text information sources and image information sources, to achieve the periodic collection and processing of target-related information.

[0003] Existing multimodal information acquisition systems of this type generally adopt a system architecture that combines a task scheduling module and a data acquisition and execution module. Based on a preset acquisition process or rules, they initiate query or capture requests to different information sources and perform preliminary integration of the acquired data. Such systems can complete basic information acquisition tasks in scenarios with short-term operation or manual configuration.

[0004] However, maritime targets are characterized by severe homonym issues, dispersed information sources, a long historical evolution span, and complex image acquisition conditions. In long-term, continuous, and unattended operational scenarios targeting ships and other maritime targets, the aforementioned existing technologies have gradually revealed the following technical problems: (1) Existing systems typically control the acquisition process based only on the task completion status. Changes in the quality of the acquisition results are not effectively introduced into the acquisition execution control process, which can easily lead to the accumulation of low-quality or semantically ambiguous data during long-term operation. (2) When the system detects abnormal acquisition or quality degradation, existing technologies often respond by re-executing the complete acquisition process or synchronously adjusting multiple acquisition parameters, which can easily lead to large fluctuations in the acquisition execution process and affect the stable operation of the system. (3) In the multimodal information acquisition scenario, the existing system lacks a stable control mechanism that can be constrained and adjusted only for execution parameters that are deterministically related to quality anomalies without changing the overall acquisition process structure. It is difficult to balance acquisition quality and execution stability in the long-term operation.

[0005] Therefore, it is necessary to propose a new technical solution to address the problem that existing multimodal information acquisition systems struggle to balance quality control and operational stability in long-term operation scenarios targeting maritime targets. Summary of the Invention

[0006] The purpose of this invention is to provide a quality assessment-driven intelligent agent system and method for acquiring multimodal information about maritime targets. This system can effectively control the quality of multimodal information acquisition without changing the overall acquisition process structure, and improve the stability of the acquisition process under long-term operating conditions.

[0007] To achieve the above objectives, embodiments of the present invention provide a quality assessment-driven intelligent agent system for acquiring multimodal information about maritime targets, deployed on a computer device or distributed computing platform, comprising: The task interaction and constraint module is used to receive multimodal information acquisition tasks for maritime targets and transform the tasks into a set of structured task constraints. The central orchestration intelligent agent module is used to generate a collection and execution plan based on the structured task constraint set, and to perform task orchestration and scheduling. The multimodal information acquisition and execution module is used to acquire information data of different modalities based on the current acquisition and execution parameter vector; The multimodal parsing and alignment module is used to parse, structure, and align the collected information data from different modalities, generating structured multimodal results for quality assessment. An execution stability control module is used to convert the quality changes of the structured multimodal results into control signals for the acquisition and execution process, and to perform differential adjustment on the acquisition and execution parameter vector under constraints to update the acquisition and execution parameter vector; The data storage and traceability module is used to store collected data, structured results, quality assessment results, parameter adjustment records, and execution trajectory information.

[0008] Optionally, the multimodal parsing and alignment module includes a text parsing unit, an image parsing unit, and a cross-modal alignment unit; The text parsing unit is used to perform semantic parsing, field extraction, and structuring processing on the text information contained in the collected information data of different modalities. The image analysis unit is used to perform quality screening and target region identification on the image information contained in the collected information data of different modalities. The cross-modal alignment unit is used to map structured text information and image features to a unified feature space and perform similarity calculation and alignment.

[0009] Optionally, the text parsing unit is used to perform semantic parsing, field extraction, and structuring processing on the text information contained in the collected information data of different modalities, including: For the original text set Sentence segmentation, word segmentation, and noise filtering are performed to obtain a set of text semantic units. ; Based on the target entity constraint, text fragments related to the target entity are selected from the set of text semantic units. ; Based on predefined field templates or entity attribute sets For the text fragment Perform field extraction to generate a set of field-value pairs:

[0010] For the field-value pair set The field values ​​are formatted, converted in units, and semantically disambiguated to generate a structured text representation.

[0011] in, This represents the constraint information of the target entity. Match(·) is the matching determination function between the text fragment and the target entity. Constraint information with target entity Return when matching Otherwise return ; This is a field extraction function. For text fragments Middle field The value of ; For text structuring mapping functions, Represents a structured text information unit.

[0012] Optionally, the image analysis unit is used to perform quality screening and target region identification on the image information contained in the acquired information data of different modalities, including: The acquired image set The first in j Zhang Image According to the formula Calculate image quality score: Filtering to meet preset quality thresholds Candidate image set ; In the candidate image set The process performs object detection to determine the set of image regions associated with the target entity. ; in, A quality scoring function that comprehensively evaluates resolution, repeatability, and noise level; Representing an image The target region identified in the middle, Represents the j-th image in the j-th image. One target area; The set of all image target recognition regions. For image collection The CCP p Zhang image.

[0013] Optionally, the cross-modal alignment unit is used to map structured text information and image features to a unified feature space and perform similarity calculation and alignment, including: Structured text information Mapped to text feature vectors ; and the target area output by the image parsing unit. Mapped to image feature vectors ; Based on text feature vectors With image feature vectors According to the formula Calculate cross-modal similarity; Based on cross-modal similarity, a cross-modal correspondence between textual and image information is established, generating structured multimodal results for quality assessment. ; in, This is a text feature mapping function used to map structured text information. Mapped to fixed-dimensional text feature vectors ; For image feature vectors, This is a mapping function from image features to semantic space, used to map image features to semantic space. I Target recognition area R Mapped to fixed-dimensional image feature vectors Sim(·) is the similarity calculation function; Generate a function for the result.

[0014] Optionally, the execution stability control module includes: a quality assessment unit, an attribution mapping unit, a differential execution control unit, and a stability determination unit; The quality assessment unit is used to perform structured quality assessment on the structured multimodal results, and use the assessment results as an endogenous control signal for executing stable control. It calculates the quality assessment vector for executing control, calculates the change between the current quality assessment vector and the quality assessment vector of the previous adjustment cycle, and determines whether the preset abnormal triggering conditions are met. The attribution mapping unit is used to calculate the correlation strength between quality changes and acquisition execution parameters based on the time change characteristics of the quality assessment vector within a predefined attribution space, and generate parameter subset constraint results based on the correlation strength. The differential execution control unit is used to calculate differential execution control quantities for the set of allowed adjustable parameters, provided that the parameter constraint mask is satisfied. The differential execution control quantities are incremental control quantities relative to the current state of the acquired execution parameters. For the acquired execution parameters in the set of prohibited adjustable parameters, their corresponding control quantities are explicitly constrained to zero and do not participate in the control quantity calculation. The stability determination unit is used to determine whether the multimodal information acquisition and execution process meets the preset stability convergence condition based on the change characteristics of the quality assessment vector in multiple consecutive adjustment cycles and the change trend of the differential execution control quantity. If it meets the condition, the acquisition continues according to the current parameters; if it does not meet the condition, the attribution mapping parameters are updated.

[0015] Optionally, the quality assessment unit includes: a basic quality index calculation subunit, an index normalization and weight combination subunit, and a quality change calculation subunit; The basic quality index calculation subunit is used to calculate basic quality sub-indicators from different quality dimensions respectively; The index normalization and weight combination unit is used to normalize the basic quality sub-indicators with different dimensions, and to perform weighted combination according to preset or adaptively learned weight parameters, thereby generating a quality assessment vector. Or overall quality score ; The quality change calculation subunit is used to calculate the change between the current quality assessment vector and the quality assessment vector of the previous adjustment cycle within an adjacent adjustment cycle, determine whether the preset abnormal triggering conditions are met, and start the attribution analysis process when the preset abnormal triggering conditions are met.

[0016] Optionally, the attribution analysis process is as follows: In each execution adjustment cycle The system maintains the following state variables: Current quality assessment vector Quality assessment vector of the previous adjustment cycle ; Attribution mapping unit calculates quality change vector And determine whether to trigger attribution analysis based on at least one of the following conditions: Any quality sub-indicator The fluctuations exceeded the preset range, and the direction of quality change was inconsistent with the historical convergence direction; among them, This is the threshold for the magnitude of change.

[0017] Optionally, the attribution mapping unit includes an association strength calculation subunit and a parameter subset partitioning subunit; The correlation strength calculation subunit is used to calculate the correlation strength between the acquisition execution parameter vector and the quality change for each acquisition execution parameter vector in the attribution space, based on the amount of change of the parameter vector in the historical adjustment period, the direction of change of the quality assessment vector and the magnitude of change. The parameter subset partitioning subunit is used to divide the acquisition and execution parameter vectors whose correlation strength meets the preset threshold condition into a set of adjustable parameters, divide the remaining acquisition and execution parameter vectors into a set of prohibited adjustable parameters, and generate a parameter constraint mask for execution control.

[0018] Secondly, the present invention also provides a quality assessment-driven method for acquiring multimodal information about maritime targets, comprising: Receive a multimodal information acquisition task targeting maritime targets, construct a structured task constraint set, and based on the structured task constraint set, generate a multimodal information acquisition execution plan and initialize the acquisition execution parameter vector; Based on the current collection execution parameter vector, text and image information related to the target entity are collected from at least two heterogeneous information sources. The collected text and image information is then parsed, structured, and cross-modal aligned to generate structured multimodal results for quality assessment. Based on the structured multimodal results, a quality assessment vector is calculated. Multiple basic quality sub-indicators are calculated, and the basic quality sub-indicators are normalized and weighted to generate a quality assessment vector corresponding to the current acquisition cycle. The quality assessment vector is compared with the quality assessment vector corresponding to the previous acquisition cycle to calculate the quality change. When the quality change meets the preset abnormal triggering conditions, it is determined that a quality abnormality has occurred in the current acquisition cycle. Then, the set of allowed adjustment parameters and the set of prohibited adjustment parameters are determined through attribution mapping, and parameter subspace constraints for the current adjustment cycle are generated. Under the constraints of the parameter subspace, differential parameter adjustment is performed on the set of allowable adjustable parameters to update the acquisition and execution parameter vector; The system determines whether the stability convergence condition is met based on the trend of quality assessment vector changes over multiple consecutive acquisition cycles, and then proceeds to the next acquisition cycle.

[0019] By introducing a quality assessment-driven execution stability control module, the quality changes of the acquired results are used as an endogenous control signal. Combined with attribution mapping, parameter subspace constraints, and differential execution adjustment mechanisms, the long-term stable operation of the multimodal information acquisition process can be achieved.

[0020] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0021] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram of the structure of a quality assessment-driven intelligent agent system for acquiring multimodal information about maritime targets, provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of an execution stability control module provided in an embodiment of the present invention; Figure 3 This is a flowchart of a quality assessment-driven multimodal information acquisition method for maritime targets provided in an embodiment of the present invention; Figure 4 This is a detailed implementation flowchart of a quality assessment-driven multimodal information acquisition method for maritime targets provided in an embodiment of the present invention. Detailed Implementation

[0022] Various embodiments of this disclosure will be described more fully in the following detailed description. This disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of this disclosure to the specific embodiments disclosed herein, but rather this disclosure should be understood to cover all adjustments, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of this disclosure.

[0023] In the following, the terms “comprising” or “may include”, which may be used in various embodiments of this disclosure, indicate the presence of the disclosed functions or operations and do not limit the addition of one or more functions or operations. Furthermore, as used in various embodiments of this disclosure, the terms “comprising,” “having,” and their cognates are intended only to indicate a specific feature, number, step, operation, or combination of the foregoing and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, or combinations of the foregoing, or the possibility of adding one or more features, numbers, steps, operations, or combinations of the foregoing.

[0024] In various embodiments of this disclosure, the expression "or" or "at least one of A and / or B" includes any combination or all combinations of the words listed simultaneously. For example, the expression "A or B" or "at least one of A and / or B" may include A, may include B, or may include both A and B.

[0025] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0026] See Figure 1 The diagram shown is a structural schematic of a quality assessment-driven intelligent agent system for acquiring multimodal information about maritime targets, provided in an embodiment of the present invention. Deployed on a computer device or distributed computing platform, it includes: The task interaction and constraint module is used to receive multimodal information acquisition tasks for maritime targets and transform the tasks into a set of structured task constraints. The central orchestration agent module is used to generate a collection and execution plan based on the structured task constraint set and to perform task orchestration and scheduling. The multimodal information acquisition and execution module is used to acquire information data of different modalities based on the current acquisition and execution parameter vector; The multimodal parsing and alignment module is used to parse, structure, and align the collected information data from different modalities to generate structured multimodal results for quality assessment. An execution stability control module is used to convert the quality changes of the structured multimodal results into control signals for the acquisition and execution process, and to perform differential adjustment on the acquisition and execution parameter vector under constraints to update the acquisition and execution parameter vector; The data storage and traceability module is used to store collected data, structured results, quality assessment results, parameter adjustment records, and execution trajectory information.

[0027] For example, the task constraints include at least: target entity constraints (at least one of ship name, hull number, model, IMO, MMSI), information requirement constraints, data source constraints, and quality threshold constraints. The data acquisition execution parameter vector includes at least query keyword parameters, data source priority parameters, capture depth parameters, and image quality parameters.

[0028] In one specific embodiment, the multimodal parsing and alignment module includes a text parsing unit, an image parsing unit, and a cross-modal alignment unit; wherein, the text parsing unit is used to perform semantic parsing, field extraction, and structuring processing on the text information contained in the collected information data of different modalities; the image parsing unit is used to perform quality screening and target region identification on the image information contained in the collected information data of different modalities; and the cross-modal alignment unit is used to map the structured text information and image features to a unified feature space and perform similarity calculation and alignment.

[0029] For example, the text parsing unit is used to perform semantic parsing, field extraction, and structuring processing on the text information contained in the collected information data of different modalities, including: For the original text set Sentence segmentation, word segmentation, and noise filtering are performed to obtain a set of text semantic units. ; Based on the target entity constraint, text fragments related to the target entity are selected from the set of text semantic units. ; Based on predefined field templates or entity attribute sets For the text fragment Perform field extraction to generate a set of field-value pairs:

[0030] For the field-value pair set The field values ​​are formatted, converted in units, and semantically disambiguated to generate a structured text representation.

[0031] in, This represents the constraint information of the target entity. Match(·) is the matching determination function between the text fragment and the target entity. Constraint information with target entity Return when matching Otherwise return ; This is a field extraction function. For text fragments Middle field The value of ; For text structuring mapping functions, Represents a structured text information unit.

[0032] For example, the image parsing unit is used to perform quality screening and target region identification on the image information contained in the acquired information data of different modalities, including: The acquired image set The first in j Zhang Image According to the formula Calculate image quality score: Filtering to meet preset quality thresholds Candidate image set ; In the candidate image set The process performs object detection to determine the set of image regions associated with the target entity. ; in, A quality scoring function that comprehensively evaluates resolution, repeatability, and noise level; Representing an image The target region identified in the middle, Represents the j-th image in the j-th image. One target area; The set of all image target recognition regions. For image collection The CCP p Zhang image.

[0033] In one specific embodiment, the cross-modal alignment unit is used to map structured text information and image features to a unified feature space and perform similarity calculation and alignment, including: Structured text information Mapped to text feature vectors ; and the target area output by the image parsing unit. Mapped to image feature vectors ; Based on text feature vectors With image feature vectors According to the formula Calculate cross-modal similarity; Cross-modal correspondences between text and image information are established based on cross-modal similarity, generating structured multimodal results for quality assessment. ; in, This is a text feature mapping function used to map structured text information. Mapped to fixed-dimensional text feature vectors ; For image feature vectors, This is a mapping function from image features to semantic space, used to map image features to semantic space. I Target recognition area R Mapped to fixed-dimensional image feature vectors Sim(·) is the similarity calculation function; Generate a function for the result.

[0034] In one specific implementation, see [reference] Figure 2 As shown, the execution stability control module includes: a quality assessment unit, an attribution mapping unit, a differential execution control unit, and a stability determination unit; The quality assessment unit is used to perform structured quality assessment on the structured multimodal results, and use the assessment results as an endogenous control signal for executing stable control. It calculates the quality assessment vector for executing control, calculates the change between the current quality assessment vector and the quality assessment vector of the previous adjustment cycle, and determines whether the preset abnormal triggering conditions are met. The attribution mapping unit is used to calculate the correlation strength between quality changes and acquisition execution parameters based on the time change characteristics of the quality assessment vector within a predefined attribution space, and generate parameter subset constraint results based on the correlation strength. The differential execution control unit is used to calculate differential execution control quantities for the set of allowed adjustable parameters, provided that the parameter constraint mask is satisfied. The differential execution control quantities are incremental control quantities relative to the current state of the acquired execution parameters. For the acquired execution parameters in the set of prohibited adjustable parameters, their corresponding control quantities are explicitly constrained to zero and do not participate in the control quantity calculation. The stability determination unit is used to determine whether the multimodal information acquisition and execution process meets the preset stability convergence condition based on the change characteristics of the quality assessment vector in multiple consecutive adjustment cycles and the change trend of the differential execution control quantity. If it meets the condition, the acquisition continues according to the current parameters; if it does not meet the condition, the attribution mapping parameters are updated.

[0035] By introducing a quality assessment-driven execution stability control mechanism into the multimodal information acquisition system, the system can adjust the acquisition execution parameters under constrained conditions without restarting or reconstructing the overall acquisition process, thereby improving the quality stability of the acquisition results and the overall stability of the system operation under long-term, continuous and unattended operation.

[0036] In one specific embodiment, the quality assessment unit includes: a basic quality index calculation subunit, an index normalization and weight combination subunit, and a quality change calculation subunit; wherein, the basic quality index calculation subunit is used to calculate basic quality sub-indicators from different quality dimensions respectively; the index normalization and weight combination subunit is used to normalize the basic quality sub-indicators of different dimensions and perform weighted combination according to preset or adaptively learned weight parameters, thereby generating a quality assessment vector. Or overall quality score The quality change calculation subunit is used to calculate the change between the current quality assessment vector and the quality assessment vector of the previous adjustment cycle within an adjacent adjustment cycle, determine whether the preset abnormal triggering conditions are met, and start the attribution analysis process when the preset abnormal triggering conditions are met.

[0037] For example, the basic quality sub-indicators include at least two or more of the following categories: information completeness sub-indicator: based on a predefined set of key fields, the coverage ratio of key fields in the collected results is calculated; multimodal consistency sub-indicator: derived from structured multimodal results. Cross-modal similarity in The multimodal consistency sub-index measures the semantic matching degree between text and image information. The source credibility sub-index is generated based on the historical reliability of the information source, cross-validation consistency, or anomaly frequency statistics. The structural integrity sub-index characterizes whether there are missing, abnormal, or inconsistent format issues in the collected results at the structural level. Each basic quality sub-index is represented as a comparable scalar value or vector component.

[0038] For example, the attribution analysis process is as follows: In each execution adjustment cycle The system maintains the following state variables: Current quality assessment vector Quality assessment vector of the previous adjustment cycle ; Attribution mapping unit calculates quality change vector And determine whether to trigger attribution analysis based on at least one of the following conditions: Any quality sub-indicator The fluctuations exceeded the preset range, and the direction of quality change was inconsistent with the historical convergence direction; among them, This is the threshold for the magnitude of change.

[0039] In one specific embodiment, the attribution mapping unit includes a correlation strength calculation subunit and a parameter subset partitioning subunit; wherein, the correlation strength calculation subunit is used to calculate the correlation strength between the acquisition execution parameter vector and the quality change for each acquisition execution parameter vector in the attribution space, based on the change amount of the parameter vector within the historical adjustment period, the change direction of the quality assessment vector, and the change magnitude of the change; the parameter subset partitioning subunit is used to partition the acquisition execution parameter vectors whose correlation strength meets the preset threshold condition into an allowed adjustment parameter set, partition the remaining acquisition execution parameter vectors into a prohibited adjustment parameter set, and generate a parameter constraint mask for execution control.

[0040] Specifically, the correlation strength calculation unit has the following specific process: Current collection execution parameter vector Previous execution parameter vector ; parameter Changes in the previous adjustment period:

[0041] For each current data collection and execution parameter Calculate the strength of its correlation with mass change:

[0042] in: The quality sensitivity weight corresponding to this parameter; : A predefined correlation calculation function used to characterize the correlation strength between changes in the data collection execution parameters and changes in the quality assessment vector, including at least one of the following: correlation coefficient of changes within a time window; consistency determination of the direction of parameter changes and quality changes; parameter influence scoring function based on preset rules or based on historical adjustment data.

[0043] Specifically, the parameter subset partitioning subunit is used to divide the acquisition and execution parameters whose correlation strength meets the preset threshold condition into a set of adjustable parameters, and divide the remaining acquisition and execution parameters into a set of prohibited adjustable parameters, and generate a parameter constraint mask for execution control. The specific process is as follows: The set of correlation strengths output by the correlation strength calculation subunit ; The parameter subset is divided into sub-units, and parameter constraint mask vectors are generated according to the following rules. : For each acquisition execution parameter in the attribution space ,definition:

[0044] in, Adjustment is permitted; Adjustment is prohibited; The threshold value is the correlation strength threshold, which can be a preset threshold or a threshold that can be adjusted based on historical operating status. This yields the parameter constraint mask vector:

[0045] like Then the parameters Included in the set of adjustable parameters: Otherwise, the parameters Included in the set of parameters that cannot be adjusted: .

[0046] In one specific implementation, the differential execution control unit is used to calculate differential execution control quantities only for the set of allowed adjustable parameters, provided that the parameter constraint mask is satisfied. The differential execution control quantity is an incremental control quantity relative to the current state of the acquired execution parameters; for acquired execution parameters in the set of prohibited adjustable parameters, their corresponding control quantities are explicitly constrained to zero and do not participate in the control quantity calculation.

[0047] The differential execution control unit incrementally updates the acquired execution parameters based on the differential execution control quantity, and does not trigger a restart, reconstruction, or full parameter synchronization modification of the overall acquisition process within the current adjustment cycle. The execution control module calculates the control quantity only for the set of allowed adjustment parameters.

[0048] in: : Control quantity generation function, used to generate incremental adjustment of acquisition execution parameters based on the direction and magnitude of change of the current quality assessment vector, under the premise of satisfying preset magnitude constraints and stability constraints. Its implementation methods include proportional adjustment, step adjustment or rule-based incremental mapping. Element-wise multiplication; for The parameters are: .

[0049] The rules for updating the collection and execution parameters are as follows: And satisfy the following strong constraints: Do not trigger a complete restart of the data collection process; No The parameters are subject to any form of synchronous, inherited, or historically accumulated adjustment; if the parameters are not re-attributed and confirmed within an adjacent adjustment period, their state remains unchanged.

[0050] In a continuous adjustment period: if a certain parameter In the cycle China belongs to Then it is in the period The initial state must not change due to historical control variables; whether a parameter participates in regulation is entirely determined by the attribution mapping result of the current period, and does not inherit the state of "having been regulated" in the past.

[0051] In one specific embodiment, this embodiment is applied to a long-term multimodal information acquisition scenario targeting maritime vessels. The vessel targets are identified by their names and models. The system needs to operate continuously without human intervention, periodically acquiring text and image information related to the targets, and ensuring the stability of the acquisition results over long-term operation. In this embodiment, the multimodal information acquisition intelligent agent system is deployed on a server device, which includes: at least one processor; a memory; and a network communication interface. The server establishes communication connections with multiple heterogeneous information sources via a network, including: a first information source: a text information source based on a web interface, used to acquire news text, encyclopedic information, or announcements related to the vessels; and a second information source: an image information source based on an image interface, used to acquire image data related to the vessels. Each functional module of the system is stored in the memory as a software program and is executed by the processor.

[0052] See Figure 3 The diagram shows a flowchart of a quality assessment-driven multimodal information acquisition method for maritime targets in a specific embodiment. The method is applied to long-term, continuous, and unattended multimodal information acquisition scenarios and includes the following execution steps: Step 300: Receive a multimodal information acquisition task for maritime targets, construct a structured task constraint set, generate a multimodal information acquisition execution plan based on the structured task constraint set, and initialize the acquisition execution parameter vector.

[0053] Specifically, the structured task constraint set includes at least: target entity constraints, used to limit the ship entity identifier corresponding to the collection object; information requirement constraints, used to limit the types of text and image information to be collected; data source constraints, used to limit the range of accessible heterogeneous information sources; and quality threshold constraints, used to limit the minimum quality requirements that the multimodal collection results must meet.

[0054] Step 301: Based on the current acquisition execution parameter vector, acquire text and image information related to the target entity from at least two heterogeneous information sources, and parse, structure, and align the acquired text and image information across modalities to generate structured multimodal results for quality assessment.

[0055] Specifically, the acquisition execution parameter vector includes at least query keyword parameters, data source priority parameters, capture depth parameters, and image quality parameters, which are used to control the specific execution method of subsequent multimodal information acquisition.

[0056] In this step, the acquisition execution parameters are used to constrain the query conditions, acquisition range, and image filtering conditions to form the raw multimodal data for the current acquisition cycle.

[0057] Specifically, the computer device first preprocesses the acquired text information, including at least text segmentation, keyword recognition, and filtering of irrelevant content. Based on the preprocessing results, it extracts fields from the text content according to a predefined set of target entity fields, including at least a target entity identifier field, an attribute description field, and a time or source field. The extracted fields are then organized according to a unified data structure to generate structured text information. Simultaneously, the acquired image information undergoes quality inspection, filtering images that meet preset clarity or resolution conditions. Target region recognition processing is then performed on the images that meet the conditions to determine image regions related to the target entity. Based on this, the structured text information and the target recognition regions are mapped to a unified feature representation space. Based on the similarity or matching relationship between the mapped text features and image features, a cross-modal correspondence between the text information and image information is established, thereby generating structured multimodal results for quality assessment.

[0058] Step 302: Calculate the quality assessment vector based on the structured multimodal results, calculate multiple basic quality sub-indicators respectively, and normalize and weight the basic quality sub-indicators to generate the quality assessment vector corresponding to the current acquisition cycle; compare the quality assessment vector with the quality assessment vector corresponding to the previous acquisition cycle to calculate the quality change.

[0059] Specifically, based on the structured multimodal results, the computer device calculates multiple basic quality sub-indicators according to a preset quality assessment model, and performs normalization and weighted combination on the basic quality sub-indicators to generate a quality assessment vector corresponding to the current acquisition cycle. The quality assessment vector is compared with the quality assessment vector corresponding to the previous acquisition cycle to calculate the quality change. When the quality change meets the preset abnormality triggering condition, it is determined that a quality abnormality has occurred in the current acquisition cycle, and the subsequent attribution mapping step is initiated.

[0060] Step 303: When the quality change meets the preset abnormal triggering conditions, it is determined that a quality abnormality has occurred in the current acquisition cycle. Then, the set of allowed adjustment parameters and the set of prohibited adjustment parameters are determined through attribution mapping, and parameter subspace constraints for the current adjustment cycle are generated.

[0061] Specifically, when a quality anomaly is determined to occur, the computer equipment calculates the correlation strength between each acquisition execution parameter and the quality change based on the quality change and historical acquisition execution parameter change records in the predefined acquisition execution parameter set. Acquisition execution parameters whose correlation strength meets the preset threshold condition are divided into a set of allowable adjustment parameters, and the remaining acquisition execution parameters are divided into a set of prohibited adjustment parameters. Based on the above division results, a parameter subspace constraint for the current adjustment cycle is generated to limit the scope of subsequent parameter adjustments.

[0062] Step 304: Under the constraints of the parameter subspace, perform differential parameter adjustment on the set of allowed adjustable parameters and update the acquisition execution parameter vector.

[0063] Specifically, under the constraints of the parameter subspace, the computer device calculates the corresponding differential parameter adjustment amount based on the direction and magnitude of the quality change, only for the set of adjustable parameters. For the acquisition execution parameters that are classified as prohibited adjustment parameters, their corresponding adjustment amount is limited to zero and does not participate in the parameter update of this adjustment cycle. The differential parameter adjustment amount is applied to the current acquisition execution parameter vector to obtain the updated acquisition execution parameter vector, and the overall restart or reconstruction of the acquisition process is not triggered during this process.

[0064] Step 305: Determine whether the stability convergence condition is met based on the trend of quality assessment vector changes over multiple consecutive acquisition cycles, and proceed to the next acquisition cycle.

[0065] Specifically, the computer device determines whether the current multimodal information acquisition process meets a preset stability convergence condition based on the changing trend of the quality assessment vector and the changes in the adjustment of the acquisition execution parameters over multiple consecutive acquisition cycles. If the stability convergence condition is met, the process proceeds to the next acquisition cycle and repeats steps 301 to 305 based on the updated acquisition execution parameter vector. If the stability convergence condition is not met, the current acquisition execution parameter state is maintained, and steps 303 to 305 are re-executed based on the current control strategy parameters until the stability convergence condition is met.

[0066] See Figure 4The diagram shows a detailed implementation flowchart of a quality assessment-driven multimodal information acquisition method for maritime targets provided by an embodiment of the present invention. The method includes: task reception and structured task constraint construction; acquisition execution plan generation and parameter initialization; multimodal information acquisition execution; multimodal parsing, structuring, and cross-modal alignment; quality assessment and anomaly trigger determination; if anomaly occurs, attribution mapping and parameter subspace constraint generation are performed; differential acquisition execution parameter adjustment is implemented; further stability determination and cycle switching are performed; if stable, multimodal information acquisition execution and subsequent steps are repeated; if unstable, attribution mapping and parameter subspace constraint generation and subsequent steps are performed; if normal, multimodal information acquisition execution and subsequent steps are repeated.

[0067] This application has the following beneficial technical effects: (1) For application scenarios with high ambiguity of maritime target entities and dispersed information sources, by using the quality assessment results of multimodal acquisition results as the endogenous control signal of acquisition execution control, the acquisition execution process can be adaptively adjusted based on the quality changes of acquisition results, avoiding the long-term accumulation of low-quality or semantically ambiguous data caused by controlling only based on the task completion status in the existing technology, thereby improving the overall reliability of multimodal acquisition results.

[0068] (2) By introducing an attribution mapping mechanism, when a quality anomaly occurs, only the acquisition execution parameters that are deterministically related to the quality change are generated as an allowable set of adjustable parameters, and the remaining acquisition execution parameters are subject to prohibition of adjustment constraints, thus explicitly limiting the scope of parameter adjustment. This avoids the irrelevant parameter disturbances introduced by the full parameter synchronous adjustment or overall re-execution method in the prior art, and effectively suppresses the oscillation phenomenon in the acquisition execution process.

[0069] (3) By implementing differential acquisition and execution parameter adjustment under parameter subspace constraints, incremental adjustment is applied only to the allowed adjustment parameters, and synchronous modification or historical inheritance adjustment of the prohibited adjustment parameters is explicitly prohibited. This allows the acquisition and execution parameters to gradually converge within the continuous adjustment cycle without restarting or reconstructing the overall acquisition process, thereby ensuring the continuity and stability of system operation while guaranteeing the improvement of acquisition quality.

[0070] (4) By organically combining quality assessment, attribution mapping, parameter subspace constraints and differential execution control, a quality-driven closed-loop control mechanism for multimodal information acquisition of maritime targets is constructed, enabling the system to achieve coordinated optimization of acquisition quality and execution stability under long-term, continuous and unattended operation conditions without changing the overall acquisition process structure. It is suitable for multimodal information acquisition applications in complex maritime target scenarios.

[0071] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0072] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A quality assessment-driven intelligent agent system for acquiring multimodal information about maritime targets, deployed on computer equipment or a distributed computing platform, characterized in that, include: The task interaction and constraint module is used to receive multimodal information acquisition tasks for maritime targets and transform the tasks into a set of structured task constraints. The central orchestration intelligent agent module is used to generate a collection and execution plan based on the structured task constraint set, and to perform task orchestration and scheduling. The multimodal information acquisition and execution module is used to acquire information data of different modalities based on the current acquisition and execution parameter vector; The multimodal parsing and alignment module is used to parse, structure, and align the collected information data from different modalities, generating structured multimodal results for quality assessment. An execution stability control module is used to convert the quality changes of the structured multimodal results into control signals for the acquisition and execution process, and to perform differential adjustment on the acquisition and execution parameter vector under constraints to update the acquisition and execution parameter vector; The data storage and traceability module is used to store collected data, structured results, quality assessment results, parameter adjustment records, and execution trajectory information; The execution stability control module includes: a quality assessment unit, an attribution mapping unit, a differential execution control unit, and a stability determination unit. The quality assessment unit is used to perform structured quality assessment on the structured multimodal results, and use the assessment results as an endogenous control signal for executing stable control. It calculates the quality assessment vector for executing control, calculates the change between the current quality assessment vector and the quality assessment vector of the previous adjustment cycle, and determines whether the preset abnormal triggering conditions are met. The attribution mapping unit is used to calculate the correlation strength between quality changes and acquisition execution parameters based on the time change characteristics of the quality assessment vector within a predefined attribution space, and generate parameter subset constraint results based on the correlation strength. The differential execution control unit is used to calculate differential execution control quantities for the set of allowed adjustable parameters, provided that the parameter constraint mask is satisfied. The differential execution control quantities are incremental control quantities relative to the current state of the acquired execution parameters. For the acquired execution parameters in the set of prohibited adjustable parameters, their corresponding control quantities are explicitly constrained to zero and do not participate in the control quantity calculation. The stability determination unit is used to determine whether the multimodal information acquisition and execution process meets the preset stability convergence condition based on the change characteristics of the quality assessment vector in multiple consecutive adjustment cycles and the change trend of the differential execution control quantity. If it meets the condition, the acquisition continues according to the current parameters; if it does not meet the condition, the update of the attribution mapping parameters is triggered. The attribution mapping unit includes an association strength calculation subunit and a parameter subset partitioning subunit; The correlation strength calculation subunit is used to calculate the correlation strength between the acquisition execution parameter vector and the quality change for each acquisition execution parameter vector in the attribution space, based on the amount of change of the parameter vector in the historical adjustment period, the direction of change of the quality assessment vector and the magnitude of change. The parameter subset partitioning subunit is used to divide the acquisition and execution parameter vectors whose correlation strength meets the preset threshold condition into a set of adjustable parameters, divide the remaining acquisition and execution parameter vectors into a set of prohibited adjustable parameters, and generate a parameter constraint mask for execution control.

2. The intelligent agent system for quality assessment-driven multimodal information acquisition of maritime targets according to claim 1, characterized in that, The multimodal parsing and alignment module includes a text parsing unit, an image parsing unit, and a cross-modal alignment unit; The text parsing unit is used to perform semantic parsing, field extraction, and structuring processing on the text information contained in the collected information data of different modalities. The image analysis unit is used to perform quality screening and target region identification on the image information contained in the collected information data of different modalities. The cross-modal alignment unit is used to map structured text information and image features to a unified feature space and perform similarity calculation and alignment.

3. The intelligent agent system for quality assessment-driven multimodal information acquisition of maritime targets according to claim 2, characterized in that, The text parsing unit is used to perform semantic parsing, field extraction, and structuring processing on the text information contained in the collected information data of different modalities, including: For the original text set Sentence segmentation, word segmentation, and noise filtering are performed to obtain a set of text semantic units. ; Based on the target entity constraint, text fragments related to the target entity are selected from the set of text semantic units. ; Based on predefined field templates or entity attribute sets For the text fragment Perform field extraction to generate a set of field-value pairs: For the field-value pair set The field values ​​are formatted, converted in units, and semantically disambiguated to generate a structured text representation. in, This represents the constraint information of the target entity. Match(·) is the matching determination function between the text fragment and the target entity. Constraint information with target entity Return when matching Otherwise return ; This is a field extraction function. For text fragments Middle field The possible values ​​of ; For text structuring mapping functions, Represents structured text information.

4. The intelligent agent system for quality assessment-driven multimodal information acquisition of maritime targets according to claim 3, characterized in that, The image analysis unit is used to perform quality screening and target region identification on the image information contained in the acquired information data of different modalities, including: The acquired image set The first in j Zhang Image According to the formula Calculate image quality score: Filtering to meet preset quality thresholds Candidate image set ; In the candidate image set The process performs object detection to determine the set of image regions associated with the target entity. ; in, A quality scoring function that comprehensively evaluates resolution, repeatability, and noise level; Representing an image The target region identified in the middle, Represents the j-th image in the j-th image. One target area; The set of all image target recognition regions. For image collection The CCP p Zhang image.

5. The intelligent agent system for acquiring multimodal information about maritime targets driven by quality assessment according to claim 4, characterized in that, The cross-modal alignment unit is used to map structured text information and image features to a unified feature space and perform similarity calculation and alignment, including: Structured text information Mapped to text feature vectors ; and the target area output by the image parsing unit. Mapped to image feature vectors ; Based on text feature vectors With image feature vectors According to the formula Calculate cross-modal similarity; Cross-modal correspondences between text and image information are established based on cross-modal similarity, generating structured multimodal results for quality assessment. ; in, This is a text feature mapping function used to map structured text information. Mapped to fixed-dimensional text feature vectors ; For image feature vectors, This is a mapping function from image features to semantic space, used to map image features to semantic space. I Target recognition area R Mapped to fixed-dimensional image feature vectors Sim(·) is the similarity calculation function; Generate a function for the result.

6. The intelligent agent system for acquiring multimodal information about maritime targets driven by quality assessment according to claim 1, characterized in that, The quality assessment unit includes: a basic quality index calculation subunit, an index normalization and weight combination subunit, and a quality change calculation subunit; The basic quality index calculation subunit is used to calculate basic quality sub-indicators from different quality dimensions respectively; The index normalization and weight combination unit is used to normalize the basic quality sub-indicators of different dimensions, and to perform weighted combination according to preset or adaptively learned weight parameters, thereby generating a quality assessment vector. Or overall quality score ; The quality change calculation subunit is used to calculate the change between the current quality assessment vector and the quality assessment vector of the previous adjustment cycle within an adjacent adjustment cycle, determine whether the preset abnormal triggering conditions are met, and start the attribution analysis process when the preset abnormal triggering conditions are met.

7. The intelligent agent system for quality assessment-driven multimodal information acquisition of maritime targets according to claim 6, characterized in that, The attribution analysis process is as follows: In each execution adjustment cycle The system maintains the following state variables: Current quality assessment vector Quality assessment vector of the previous adjustment cycle ; Attribution mapping unit calculates quality change vector And determine whether to trigger attribution analysis based on at least one of the following conditions: Any quality sub-indicator The fluctuations exceeded the preset range, and the direction of quality change was inconsistent with the historical convergence direction; among them, This is the threshold for the magnitude of change.

8. A method for acquiring multimodal information of maritime targets driven by quality assessment, applied to a quality assessment-driven intelligent agent system for acquiring multimodal information of maritime targets as described in any one of claims 1-7, characterized in that, include: Receive a multimodal information acquisition task targeting maritime targets, construct a structured task constraint set, and based on the structured task constraint set, generate a multimodal information acquisition execution plan and initialize the acquisition execution parameter vector; Based on the current collection execution parameter vector, text and image information related to the target entity are collected from at least two heterogeneous information sources. The collected text and image information is then parsed, structured, and cross-modal aligned to generate structured multimodal results for quality assessment. Based on the structured multimodal results, a quality assessment vector is calculated. Multiple basic quality sub-indicators are calculated, and the basic quality sub-indicators are normalized and weighted to generate a quality assessment vector corresponding to the current acquisition cycle. The quality assessment vector is compared with the quality assessment vector corresponding to the previous acquisition cycle to calculate the quality change. When the quality change meets the preset abnormal triggering conditions, it is determined that a quality abnormality has occurred in the current acquisition cycle. Then, the set of allowed adjustment parameters and the set of prohibited adjustment parameters are determined through attribution mapping, and parameter subspace constraints for the current adjustment cycle are generated. Under the constraints of the parameter subspace, differential parameter adjustment is performed on the set of allowable adjustable parameters to update the acquisition and execution parameter vector; The system determines whether the stability convergence condition is met based on the trend of quality assessment vector changes over multiple consecutive acquisition cycles, and then proceeds to the next acquisition cycle.

Images