Handwritten signature verification system, method and equipment based on multi-modal data fusion and multi-agent collaboration and medium

Abstract

The invention discloses a handwriting verification system based on multi-modal data and multi-agent cooperation. A multi-modal data acquisition module synchronously acquires a handwritten signature handwriting static image and corresponding online dynamic time sequence data, records a writing environment and a writing state, and marks writing level and stability labels in a classified manner; the multi-agent comprehensive processing module carries out training, processing and classification based on electronic signature handwriting static and dynamic data in combination with writing level and stability labels, and outputs a handwriting static image feature coding vector, a dynamic time sequence feature coding vector and writing environment information features; and the collaborative decision-making agent module continuously optimizes a handwriting authenticity verification strategy through a reinforcement learning mechanism, dynamically integrates the output of the multi-agent comprehensive processing module, and realizes the accurate judgment of the authenticity of the signature handwriting. The system also adopts a self-adaptive optimization strategy based on reinforcement learning, automatically adjusts a threshold value and a weight through an intelligent agent, further provides the stability and generalization of verification and collapse, and can maintain better recognition precision under different quality conditions and complex scenes.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a handwritten signature verification system based on multimodal data fusion and multi-agent collaborative optimization. Background Technology

[0002] Handwritten signatures can be categorized into two types: offline static signature images and online dynamic signature sequences. Offline static signature images reflect spatial information such as shape, thickness, and texture, while online dynamic signature sequences reflect information such as the time, pressure, and speed of the corresponding signature. In recent years, handwritten signatures have also faced various types of attacks and threats, including forgery, replay attacks, and tampering. Among these, skilled forgers can highly imitate the handwriting of the target's signature, posing a significant security risk.

[0003] Most existing handwritten signature verification technologies still rely on manually designed features based on prior knowledge of a single type of handwriting and deep learning-based feature extraction. These extracted features are then used for further comparison and verification. While these methods can solve most problems in normal scenarios to some extent, they may fail when the quality of the individual handwriting is poor or the style varies significantly. For example, signature verification based solely on offline static signatures only obtains the image data after the signature is signed, making it difficult to obtain dynamic information such as pressure, azimuth, acceleration, and velocity at the time of signing. This makes offline static signature verification easier to imitate, compromising security. While there are methods that integrate two types of handwritten signatures for multimodal handwritten signature verification, these methods lack the ability to effectively fuse and analyze multimodal data, failing to fully utilize the multidimensional features of the signature and external environmental characteristics, thus hindering in-depth personalized analysis of users.

[0004] For example, CN114360071A describes a method for offline handwritten signature verification based on artificial intelligence. This method proposes an AI-based feature extraction and verification algorithm, which can be effectively applied to signature scenarios where dynamic handwriting information cannot be obtained, such as traditional paper documents and scanned images. However, it lacks dynamic information, has limited anti-counterfeiting capabilities, and its discrimination ability is easily affected by the quality of the signature image.

[0005] For example, CN119027964A describes a multimodal fusion method for verifying handwritten electronic signatures. This method attempts to organically fuse multimodal features for handwritten signature verification. However, multimodal acquisition requires data synchronization and high precision, and is easily affected by the acquisition environment (such as lighting interference, handwriting posture, device information, etc.). Data distortion or missing data will seriously reduce the model's discrimination accuracy and limit its generalization ability in complex scenarios.

[0006] To address these issues, and considering the dynamic nature of handwritten signatures and the consistency of multimodal data, this system develops a handwritten signature verification system that can comprehensively consider the multidimensional feature information of multimodal handwritten signatures and adaptively optimize decisions, thereby further improving the accuracy of handwritten signature verification. Summary of the Invention

[0007] To address the aforementioned issues, this invention proposes a handwritten signature verification system based on multimodal data fusion and multi-agent collaborative optimization. The handwritten signature verification system comprises: a multimodal data acquisition module, a multi-agent integrated processing module, and a collaborative decision-making agent module. The multimodal data acquisition module synchronously acquires static images of handwritten signatures and corresponding online dynamic time-series data, records the writing environment and writing state, and categorizes and labels writing level and stability. The multi-agent integrated processing module trains, processes, and classifies the static and dynamic data of electronic signatures based on the writing level and stability labels, outputting static image feature encoding vectors, dynamic time-series feature encoding vectors, and writing environment information features. The collaborative decision-making agent module continuously optimizes the handwriting authenticity verification strategy through a reinforcement learning mechanism, dynamically integrating the output of the multi-agent integrated processing module to achieve accurate determination of the authenticity of the signature.

[0008] Further preferred, the continuously optimized handwriting authenticity verification strategy includes the collaborative decision-making agent repeatedly sampling writing state-action pairs and performing state-action iterative optimization based on the Bellman residual minimization criterion, so that the collaborative decision-making agent reaches the optimal verification strategy.

[0009] Further optimization and iterative refinement to the optimal strategy include: recording the state, actions, and immediate rewards at all writing moments to construct a data sample and store it in the experience pool. From the experience pool Small and medium batches of sampled data are used to optimize the target network parameters based on the data samples at any time i. When the global loss function converges and the difference between the Q values ​​before and after updating time i is less than the stability coefficient, that is, the average cumulative reward corresponding to the policy is stable, it is considered to have reached the optimal policy.

[0010] Further optimization involves using reinforcement learning and deep Q-networks to utilize the state-action value function. Measure in state Next action Expected cumulative return Q value According to the formula: , Calculate the Q value, where, This represents the expectation of all possible trajectories under this strategy. For a moment Discount factor, For a moment The instant reward, Q-value updates follow the Bellman optimality equation to calculate the optimal reward value. : State-action value function Approximate the expected cumulative return function and determine the loss function. for: , in, , in, To select an action in the next state The maximum return that can be obtained; For the target network parameters, The value function is the action that yields the maximum reward by selecting the target network parameters in the next state.

[0011] Further optimization involves the policy evaluation submodule within the collaborative decision-making agent, which determines whether the current policy converges and possesses generalization ability through statistical analysis, confidence estimation, and robustness assessment. This is followed by threshold tuning and model updates. Specifically, based on a signature dataset containing different user samples and various environmental perturbation conditions, the policy evaluation submodule receives the policy function output by the collaborative decision-making agent module. Each sample consists of a mini-batch of sampled data, which is composed of a (state-action-reward-next state) quadruple. Simultaneously, it integrates offline static features, dynamic sequence features, and environmental features, and uses the Monte Carlo simulation method to randomly sample and evaluate the policy function of the evaluation submodule in multiple signature scenarios.

[0012] Further preferably, the strategy function of the random sampling evaluation verification evaluation submodule includes: calculating the discounted sum of the return sequence from time t to the end, as an empirical return value for the strategy evaluation at time t. , Where T is the termination time step, and k is the offset relative to the current time t; for each batch test, the evaluation module calculates and calculates and statistics: false positive rate (FPR), false negative rate (FNR), accuracy (ACC), and expected cumulative return. The merits of the strategy functions of the integrated submodules are calculated by weighting the fusion weights.

[0013] Further optimization involves evaluating the score based on the current state after batch testing is completed. Calculate the new fusion weights and decision threshold adjustment coefficients, and calculate the average score across multiple batches. The threshold for fine-tuning the feature fusion weights of multiple agents is obtained. And the threshold for fine-tuning the authenticity judgment , in, and As the adaptive learning rate parameter, fine-tuning thresholds are used to fine-tune the multi-agent feature fusion weights and the high and low thresholds for true / false judgments, forming a feedback loop to achieve adaptive performance optimization. The score at time t satisfies: When the current strategy is the optimal strategy, then the current strategy is the optimal strategy.

[0014] Further optimization involves training a convolutional neural network architecture in the multi-agent integrated processing module to obtain an offline static image feature extractor. This extracts static features, including signature texture and morphology, from the offline static features of handwritten signatures, and linearly maps the static signatures into static feature encoding vectors. An online dynamic sequence feature extraction agent is constructed using a recurrent neural network (RNN) or Transformer model. Based on the online dynamic signature sequence data, dynamic features such as signature stroke order, speed changes, and pressure fluctuations are obtained, and the dynamic features of the handwriting sequence are output. Finally, a lightweight convolutional neural network is used to train an environmental information feature extraction agent using writing level and writing stability features and classification labels. This results in the output of environmental information feature vectors and classification results for writing level and writing stability.

[0015] Further preferably, the step of converting the static signature into a static feature encoding vector includes: an offline static image feature extractor uses a residual network to enhance the expression of static signature features, and simultaneously uses an attention mechanism to highlight the key areas of the static signature, thereby extracting a static signature feature encoding vector; the step of outputting a handwriting sequence dynamic feature encoding vector includes: obtaining the real-time trajectory point time-series information of the signature corresponding to the offline paper document image of the handwritten signature, constructing a multi-channel input sequence, performing deep learning modeling on the time-series information of each channel, training the model through a Transformer network architecture, obtaining the long-term dependencies between handwriting sequence data through a self-attention mechanism, obtaining a handwriting sequence feature extractor, which acts as an online dynamic sequence feature extraction agent, processes the signature trajectory point signal, and outputs a handwriting sequence dynamic feature encoding vector describing the writing rhythm and style of the signature sequence.

[0016] Based on the second aspect of this application, a handwritten signature verification method based on multimodal data and multi-agent collaboration is proposed. This method simultaneously acquires static images of handwritten signatures and corresponding online dynamic temporal data, records the writing environment and writing state, and categorizes and labels writing level and stability. Based on the static and dynamic data of electronic signature handwriting, combined with writing level and stability labels, the method is trained, processed, and classified to output static image features, dynamic temporal features, and writing environment information features. Through a reinforcement learning mechanism, the method continuously optimizes the handwriting authenticity verification strategy and dynamically integrates the output of the multi-agent comprehensive processing module to achieve accurate determination of the authenticity of the signature handwriting.

[0017] Further preferred, the continuously optimized handwriting authenticity verification strategy includes a collaborative decision-making agent repeatedly sampling writing state-action pairs and iteratively optimizing the state-action pairs to the optimal verification strategy based on the Bellman residual minimization criterion.

[0018] Further optimization and iterative refinement to the optimal strategy include: recording the state, actions, and immediate rewards at all writing moments to construct a data sample and store it in the experience pool. From the experience pool Small and medium batches of sampled data are used to optimize the target network parameters based on the data samples at any time i. When the global loss function converges and the difference between the Q values ​​before and after updating time i is less than the stability coefficient, that is, the average cumulative reward corresponding to the policy is stable, it is considered to have reached the optimal policy.

[0019] Further optimization is achieved by utilizing the state-action value function. Measure in state Next action Expected cumulative return Q value According to the formula: , Calculate the Q value, where, This represents the expectation of all possible trajectories under this strategy. For a moment Discount factor, For a moment The instant reward, Q-value updates follow the Bellman optimality equation to calculate the optimal reward value. : To select an action in the next state The maximum reward that can be obtained; in terms of state-action value function Approximate the expected cumulative return function and determine the loss function. for: , in, , For the target network parameters, The value function is the action that yields the maximum reward by selecting the target network parameters in the next state.

[0020] Further optimization involves determining whether the current policy converges and has generalization ability through statistical analysis, confidence estimation, and robustness assessment. Threshold tuning and model updates are then performed. Specifically, this includes constructing a policy function for the policy evaluation submodule to receive the output of the collaborative decision-making agent module, based on a signature dataset containing different user samples and multiple environmental perturbation conditions. Each sample consists of a mini-batch of sampled data, which is composed of a (state-action-reward-next state) quadruple. Simultaneously, it integrates offline static features, dynamic sequence features, and environmental features, and uses the Monte Carlo simulation method to randomly sample and evaluate the policy function of the evaluation submodule in multiple signature scenarios.

[0021] Further preferably, the strategy function of the random sampling evaluation verification evaluation submodule includes: calculating the discounted sum of the return sequence from time t to the end, as an empirical return value for the strategy evaluation at time t. , Where T is the termination time step, and k is the offset relative to the current time t; for each batch test, the evaluation module calculates and calculates and statistics: false positive rate (FPR), false negative rate (FNR), accuracy (ACC), and expected cumulative return. The submodule's policy is comprehensively evaluated using a weighted scoring function. Threshold tuning and model updates are performed through statistical analysis, confidence estimation, and robustness assessment. Specifically, this includes constructing a policy function for the policy evaluation submodule to receive the output from the collaborative decision-making agent module, based on a signature dataset containing different user samples and multiple environmental perturbation conditions. Each sample consists of a mini-batch of sampled data, which is composed of a (state-action-reward-next state) quadruple. Simultaneously integrating offline static features, dynamic sequence features, and environmental features, the strategy function of the evaluation submodule is evaluated and verified through random sampling in multiple signature scenarios using the Monte Carlo simulation method. The strategy function of the random sampling evaluation and verification submodule includes: calculating the discounted sum of the return sequence from time t to the end, as an empirical return value for evaluating the strategy at time t. , Where T is the termination time step, and k is the offset relative to the current time t; for each batch test, the evaluation module calculates and calculates and statistics: false positive rate (FPR), false negative rate (FNR), accuracy (ACC), and expected cumulative return. The merits of the submodule strategy functions are comprehensively evaluated by weighted fusion.

[0022] According to a third aspect of this application, an electronic device is provided, comprising: a processor; and a memory storing a program, wherein the program includes instructions that, when executed by the processor, cause the processor to perform the method according to any of the preceding claims.

[0023] According to a fourth aspect of this application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to perform the method described in any of the preceding claims.

[0024] This invention employs a multi-agent integrated processing mechanism, combining offline static image features, online dynamic sequence features, and environmental information features of handwritten signatures to extract and analyze multi-dimensional features of handwritten signatures. This overcomes the limitations of existing conventional technologies that rely solely on single-dimensional or type features, significantly improving the accuracy and reliability of signature verification. By adopting an adaptive reinforcement learning optimization verification strategy, the verification threshold and weights are adjusted by the agents themselves, dynamically optimizing the recognition model and improving the system's generalization ability. This allows the system to maintain high recognition accuracy even under complex conditions such as different signature qualities and environmental changes.

[0025] This invention captures static, dynamic, and environmental features of handwritten signatures through a multi-agent integrated processing module. Combined with a reinforcement learning-driven collaborative decision-making agent, it achieves flexible, adaptive, and highly accurate handwriting verification. Key technologies include multimodal feature fusion and adaptive threshold adjustment based on multi-agent coordinated optimization, supporting diverse and dynamic signature verification needs in real-world business scenarios.

[0026] This system not only effectively improves the overall performance of handwritten signature verification, but also enhances anti-counterfeiting capabilities and system stability, meeting the high standards of identity security required in fields such as finance, justice, and e-government. It has broad application prospects and promotional value. Attached Figure Description

[0027] Figure 1 A schematic diagram of the module structure of the handwritten signature verification system based on multimodal data fusion and multi-agent collaborative optimization in this embodiment of the invention.

[0028] Figure 2 The diagram shown is a structural block diagram of an exemplary electronic device that can be used to implement embodiments of this application. Detailed Implementation

[0029] Embodiments of this application will now be described in more detail with reference to the accompanying drawings. While some embodiments of this application are shown in the drawings, it should be understood that this application can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this application. It should be understood that the drawings and embodiments of this application are for illustrative purposes only and are not intended to limit the scope of protection of this application.

[0030] It should be understood that the steps described in the method embodiments of this application may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this application is not limited in this respect.

[0031] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the following description. It should be noted that the concepts of "first", "second", etc., mentioned in this application are used only to distinguish different devices, modules, or units, and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies.

[0032] The terms “a” and “a plurality” used in this application are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as “one or more”.

[0033] The names of the messages or information exchanged between multiple devices in the embodiments of this application are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0034] This invention provides a handwritten signature verification system based on multimodal data fusion and multi-agent collaborative optimization, which mainly includes a multi-agent integrated processing module and a collaborative decision-making agent module.

[0035] The multi-agent integrated processing module includes: an offline static image feature extraction agent that extracts texture and morphological features from handwritten signature images based on convolutional neural networks (CNNs); an online dynamic sequence feature extraction agent that uses recurrent neural networks (RNNs) or Transformer architectures to acquire and analyze time series data such as stroke order, speed, and pressure of handwritten signatures; an environmental information feature extraction agent that combines external information such as the writer's handwriting posture, paper interference, and pen condition to comprehensively analyze the impact of the environment on the signature; and a collaborative decision-making agent module that integrates the output of the multi-agent integrated processing module and dynamically adjusts handwriting authenticity verification decisions based on reinforcement learning strategies. This includes a value function calculation submodule and a policy evaluation submodule. The value function calculation submodule calculates the expected reward of the current handwriting state based on the system state and the stroke trajectory action space, while the policy evaluation submodule simulates the effects of different policy executions.

[0036] Through a multi-agent collaborative processing mechanism, combining agents for offline handwritten signature static image feature extraction, online handwritten signature dynamic sequence feature extraction, and environmental information feature extraction, the system acquires offline static features of handwritten signatures and corresponding online dynamic sequences. It records and labels the handwritten signature state and environmental information, extracts static features and corresponding temporal features, and obtains handwriting level and stability features, as well as classification labels. The system continuously optimizes the verification strategy through reinforcement learning, dynamically integrating the output of the multi-agent integrated processing module to extract and analyze multi-dimensional features of handwritten signatures, achieving accurate determination of signature authenticity and further improving the accuracy of handwriting verification. Simultaneously, the system employs an adaptive optimization strategy based on reinforcement learning, automatically adjusting thresholds and weights through agents, further enhancing the stability and generalization of the verification process, maintaining good recognition accuracy under different quality conditions and complex scenarios.

[0037] Figure 1The diagram illustrates the module structure of a handwritten signature verification system based on multimodal data fusion and multi-agent collaborative optimization in an exemplary embodiment of this application. The handwritten signature verification system includes: a multimodal data acquisition module, a multi-agent integrated processing module, and a collaborative decision-making agent module. In the multimodal data acquisition module, a paper-pen synchronous signature collector acquires offline high-dimensional static image features and online dynamic temporal features of handwritten handwriting / signatures, recording and labeling the handwritten signature's writing level, writing stability features, and writing environment information. In the multi-agent integrated processing module, an offline static image feature extraction agent extracts static features of the handwritten signature, and an online dynamic sequence feature extraction agent extracts dynamic series features of the handwritten signature. An environment information feature extraction agent acquires the handwritten signature's writing level, writing stability features, and classification results. The collaborative decision-making agent module includes a value function calculation submodule and a decision evaluation submodule, which continuously optimizes the verification strategy through a reinforcement learning mechanism, dynamically integrating the output of the multi-agent integrated processing module to achieve accurate determination of signature authenticity.

[0038] (1) Multimodal data acquisition module, which synchronously acquires offline paper document images of handwriting (handwritten signature) and corresponding online signature dynamic sequence data, and records and marks relevant information such as writing posture, level and writing environment.

[0039] This study collects offline static image features and online dynamic sequence features of handwritten handwriting. Based on a paper-and-pen synchronized writing tool, it collects static images of the signer's handwriting and dynamic writing trajectory data (such as X / Y coordinates of the handwriting points, pressure, writing speed, and acceleration). The simultaneously acquired offline document images of handwritten signatures and their corresponding online dynamic sequence data are also obtained. Alternatively, offline static image data of handwritten signatures can be obtained by scanning paper documents, and corresponding dynamic sequence data of the writing trajectory can be collected using the synchronized writing tool. Specifically, the offline static images of handwritten signatures are annotated with corresponding detection boxes using the classic detection box annotation tool LabelMe, including (x... min y min x max y max Then, the corresponding signature area handwriting image data is obtained by cropping. Based on the signing tool, the online dynamic sequence data of the handwritten signature is obtained, which specifically includes the time series of each corresponding handwriting point in the handwriting image, such as the horizontal coordinate x, vertical coordinate y, pressure p, time t, pen lifting state s, writing direction, and angle.

[0040] Information on the handwritten signature environment is collected. During the simultaneous paper-pen data collection, the data collection personnel specifically record and detect relevant information about the collection environment. They need to collect data multiple times under various complex conditions. The main dimensions include: the paper interference in the signature area, the pen-related conditions, the signer's posture when signing, the writing level of the handwritten signature, and the writing stability of the handwritten signature.

[0041] A. Paper interference in the signature area: Marked according to the degree of interference: no interference, slight interference, moderate interference, severe interference. The degree of interference can be determined by referring to the table below. B. Information about the writing pen: Record and label the type of writing pen, such as regular writing pen, thick marker pen, ultra-fine writing pen or other types; C. Signature posture: The signing posture of the signer is recorded in real time during the collection process, including sitting posture, standing posture and casual posture. Under certain circumstances, the signing posture will affect the signer's writing habits or style. D. Handwritten signature writing level: The handwritten signature writing level is pre-marked by the data collection personnel in real time during the data collection process. This part is relatively subjective, so it needs to be marked and corrected later. The writing level can be divided into high, average or low. E. Stability of Handwritten Signatures: During the data collection process, the stability of handwritten signatures is marked in real time. Multiple signatures by the same person in different scenarios are tested, and the signature is marked as stable or unstable based on differences in form, structure, and layout. Multiple signatures using the same font and structure are marked as stable, while multiple signatures using different fonts, such as regular script, cursive script, simplified characters, traditional characters, or inconsistent strokes, are marked as unstable.

[0042] (2) Multi-agent integrated processing module This includes an offline static image feature extraction agent, an online dynamic sequence feature extraction agent, and an environmental information feature extraction agent. Electronic signature handwriting data is trained, processed, and classified using labels based on handwriting level and stability to extract static image features, dynamic temporal features, and writing environment information features.

[0043] 1) An offline static image feature extraction agent is trained using a convolutional neural network (CNN) architecture to obtain an offline static image feature extractor. This extractor extracts signature texture and morphological features from collected handwritten signature images, including stroke thickness, handwriting continuity, and image edge features. Preprocessing is performed on the collected offline static signature data, primarily including image denoising, binarization, and normalization to ensure input data consistency. Deep learning networks, such as ResNet, are used to enhance the expressive power of static signature features. Attention mechanisms are also used to extract static signature features such as visual content and style features, highlighting the key areas of the static signature. After linear mapping, the static signature features extracted by the deep learning network are transformed into high-dimensional vectors and sent to the collaborative decision-making agent. The high-dimensional vector output corresponding to the static signature image features can have a dimension of 512*1.

[0044] 2) Online Dynamic Sequence Feature Extraction Agent. An online dynamic sequence feature extraction agent is constructed using a recurrent neural network (RNN) or Transformer model. Based on online dynamic signature sequence data corresponding to offline paper document images with handwritten signatures, it acquires high-dimensional, high-level dynamic features such as the stroke order, speed changes, and pressure fluctuations of the signature, outputting a dynamic feature encoding vector for the handwriting sequence. Specifically, this includes: The sensor built into the paper-pen synchronization device can be used to obtain the real-time trajectory point signal information of the signature corresponding to the offline paper document image of the handwritten signature, such as basic information (x, y, p, t, s), to construct a multi-channel input sequence. Deep learning modeling can be performed on the temporal information. The model can be trained through a network architecture similar to Transformer. Transformer obtains the long-term dependencies between handwriting sequence data through a self-attention mechanism. After training, a handwriting sequence feature extractor is obtained. The handwriting sequence feature extractor is an online dynamic sequence feature extraction agent. This agent processes the signature trajectory point signals acquired by the sensor and outputs a dynamic feature encoding vector of the handwriting sequence. This dynamic feature encoding vector describes the writing rhythm and style of the signature sequence. The feature output dimension can be 512*1.

[0045] 3) Environmental Information Feature Extraction Agent. During the paper-pen synchronous signature data collection process, the environmental information collection unit can synchronously classify and label the collected signing environment information and signature status, such as obtaining the paper interference level of the signature area (none, low, medium, high), collecting the pen number (normal pen, bold marker, or others), signing posture (sitting, standing, or casual posture), signature writing level (high, average, or low), and signature writing stability (stable, unstable). The environmental information-related features are mainly used as supplementary records of features beyond the actual signature handwriting. These features can, to a certain extent, reflect whether the signer's signature writing level and stability are affected by different environmental changes during the signing process.

[0046] 4) Data Preprocessing. The environmental information feature extraction agent quantifies and labels the features of the above-mentioned hierarchical annotation. Since the feature records of this part cannot be represented by continuous values, discrete values ​​are used for labeling, such as the degree of paper interference (none, low, medium, high can be labeled as 0, 1, 2, 3), the pen number collected (normal pen, marker pen or other pen can be labeled as 0, 1, 2), the signing posture (sitting posture, standing posture or casual posture can be labeled as 0, 1, 2), the signature writing level (high, average or low can be labeled as 0, 1, 2), and the signature writing stability (stable, unstable can be labeled as 0, 1).

[0047] The interference level of discrete numerical labels, pen number, and signature posture are used as input to a deep learning network. The corresponding signature writing level and stability are used as multi-classification labels. A lightweight convolutional neural network (ShuffleNet) is employed for training the multi-classification model. The aim is to confirm the signer's writing level and stability based on the collected information. This multi-classification model acts as an agent for extracting environmental information features, ultimately outputting the environmental information feature vector extracted by the backbone network, along with the classification results for writing level and stability. The environmental information feature vector can have a dimension of 512*2, while the classification result has a one-dimensional dimension, each corresponding to a classification label.

[0048] (3) The collaborative decision-making agent module includes a value function calculation submodule and a policy evaluation submodule. As the core of the system's decision-making, it dynamically integrates the output of the multi-agent integrated processing module to accurately determine the authenticity of signatures. The verification strategy is continuously optimized through a reinforcement learning mechanism to adapt to different user signature habits and environmental changes. Specifically: 1) Value Function Calculation Submodule This exemplary embodiment is described by way of example in conjunction with reinforcement learning and deep Q-network (DQN) algorithms.

[0049] By integrating the current state vector and fully fusing various features from the multi-agent feature extractor, a multi-dimensional "state" representation reflecting the current signature verification environment is formed. For example, The static visual content and style features output by the offline static image feature extraction agent are: ; The dynamic sequence writing rhythm and style features output by the online dynamic sequence feature extraction agent are: ; The environmental information features extracted by the intelligent agent are the features of the collected environment information. .

[0050] The above features are fused using adaptive trainable weights to obtain the output features of the multi-agent system, which are then synthesized into a fused state representation vector. If they can be merged into: in, , To enable adaptive and trainable weights, the weights can be adjusted in real time based on the distribution of signature feature data and task feedback, making the fused features more suitable for the final task. Generally, random initialization is used to assign values ​​during training, and the weights are adaptively adjusted and optimized to the best state during the training process.

[0051] State representation vector The state space input for signature verification at time t fully reflects the multidimensional attributes of the current acquisition environment and signature behavior.

[0052] Define actions and the basis for judgment and adjustment. Based on state representation vectors. The action space is defined by all possible judgment or adjustment behaviors that can be taken. In the reinforcement learning framework of the handwritten signature verification system, action definition and judgment criteria design are the core links in realizing intelligent decision-making, directly affecting the accuracy and stability of the system. Based on the aforementioned state representation vector... The action space and decision-making mechanism can be designed as follows: a. Definition of Action Space action Represents the state expression vector of the system The following are all possible judgment or adjustment actions that can be taken, covering the judgment results and control strategies, such as: - The signature was determined to be forged. - The signature was determined to be genuine. - Request manual review or trigger data collection / algorithm parameter adjustment.

[0053] This action space state allows the system to adaptively adjust or be manually intervened in cases of low confidence or abnormal samples, thereby improving system security.

[0054] b. Action Decision Business Logic Action states are defined based on confidence threshold discrimination and the probability settings for the authenticity of handwriting judgment results. A fusion state representation vector is then used. After the discrimination model outputs the probability of the judgment result being true or false, if the probability exceeds the high confidence threshold, then execution is performed. 1; If it is below the low threshold, then execute. 0; trigger if between the thresholds. The manual review process, including the setting of high and low thresholds, is achieved through model validation statistics using a validation machine. This part is divided into three regions (accept, review, and rejection) based on the low and high thresholds to balance system performance. The high threshold... Low false connection rate, ensuring an extremely low misclassification rate for samples identified as genuine; low threshold. Low false rejection rate, ensuring the false rejection limit for spurious samples; manual verification area between two thresholds to reduce the risk of false rejection.

[0055] Parameter adaptive adjustment: when an action is triggered This includes not only manual review, but also adjusting the feature fusion weights. , , This is to improve the quality of subsequent data.

[0056] Calculate expected returns (profits and risks) based on the DQN algorithm. Integrating multimodal fusion state into the handwritten signature verification system The expected reward for performing an action is calculated using a deep Q-network (DQN), which guides the optimization of the intelligent discrimination strategy. The specific steps and technical details are as follows: a. Definition of the State-Action Value Function DQN utilizes the state-action value function Measure in state Next action Expected cumulative return value The Q-value (expected cumulative return) is calculated as follows: , in, The symbol represents the expectation of all possible trajectories under this strategy, used to balance immediate and long-term returns. As a discount factor, For a moment Discount factor, For a moment Instant rewards and They represent state and action, respectively. The initial state of the agent is denoted as . , The action chosen by the agent at the initial moment is: .

[0057] The Q-value is updated following the Bellman optimality equation to calculate the optimal return value. : Indicates the next state The next possible action to choose; Choose the action that will yield the greatest reward in the next state.

[0058] In the DQN network, the neural network state-action value function is used. Approximating the above expected cumulative return function, the determined loss function is: , in, , In the state Next action Get instant rewards Expectations in the next state The target network parameters are periodically updated, whereby... The action that yields the greatest reward by selecting the target network parameters in the next state.

[0059] Instant reward function The definition, combining the accuracy of signature authenticity determination with system robustness, can be set as follows: The core problem of reinforcement learning is that when determining truth or falsehood, the agent receives an immediate reward after choosing an action. For example: if the judgment is correct (a genuine signature is judged as genuine, a fake signature as fake), a positive reward will be given (such as a reward). ); A negative reward will be given for incorrect judgment (misjudging a fake signature as genuine or a genuine signature as fake). -1); if manual review is triggered, a negative reward will be given ( (-0.5), the increase and decrease reward values ​​can be set to any real number.

[0060] The collaborative decision-making agent iterates through state-action pairs by repeatedly sampling them and performing parameter iteration based on the Bellman residual minimization criterion. The state-action iterative optimization is then performed to reach the optimal policy. This exemplary embodiment can adopt the following optimization process: Get the action in the current state : Record all moments, actions, and instant rewards to build a data sample and store it in the experience pool. From the experience pool Small to medium batch sampling, based on data samples at any time i. , ( The optimization aims to achieve the target network parameters. The optimization objective of this process is: , in, This refers to the parameters of the current Q-network, which need to be continuously updated iteratively to improve the accuracy of Q-value estimation. These represent the target Q-network parameters, which are auxiliary network parameters used to improve stability. Indicates the parameters of the main network Finding the gradient, the actual optimization action is adjusting... Instead .

[0061] When the global loss function converges, the differences in continuous updates are small, and the difference in Q-values ​​before and after the update is less than the stability coefficient. That is, satisfying: in, The learning rate for network training. The stability coefficient indicates that the average cumulative reward corresponding to the strategy is stable, which is considered to be the optimal strategy.

[0062] 2) Strategy Evaluation Submodule The policy evaluation submodule is used to perform batch testing and feedback on the decision performance of reinforcement learning agents (DQN decision policies) in the signature verification system. Its core function is to determine whether the current policy has convergence and generalization ability through statistical analysis, confidence estimation, and robustness assessment, and to guide threshold tuning and model updates based on the results.

[0063] Input and evaluation sample generation. Using a signature dataset containing a large number of different user samples and various environmental perturbation conditions as input, the policy evaluation submodule receives the policy function output by the collaborative decision-making agent module as follows: Here, argmax represents the action that takes the maximum value. Each sample consists of a mini-batch sampled data consisting of a (state-action-reward-next state) quadruple. Simultaneously, offline static features, dynamic sequence features, and environmental features are integrated to ensure the representativeness of the sample distribution. Using Monte Carlo simulation, the policy function of the evaluation sub-module is evaluated and verified through random sampling in multiple signature scenarios. Specifically, the evaluation includes calculating the discounted sum of the return sequence from time t to the end, which serves as an empirical return value for evaluating the policy at time t. , Where T is the termination time step, and k is the offset relative to the current time t.

[0064] By implementing the strategy function Export actions, collect corresponding experience trajectories, and then statistically analyze each step sample. As the real benefit of this step, it enables an objective evaluation of the performance of the policy or Q-function. In the process of policy evaluation, training, and policy improvement, the combination of the two promotes the agent's learning and iterative development of better decision-making methods.

[0065] Performance metrics calculation and weighted scoring: For each batch test, the evaluation module calculates and calculates the following performance metrics: * False Positive Rate (FPR): The percentage of a fake signature that is mistaken for a genuine signature; * False Negative Rate (FNR): The percentage of genuine signatures that are falsely rejected; * Accuracy (ACC): The overall correct judgment rate of the system; * Expected cumulative return: .

[0066] The effectiveness of a strategy is comprehensively evaluated using a weighted scoring function. , in, , , , These are the experience-weighted coefficients for different indicators, which can be adjusted according to task priority. The higher the score, the better the strategy.

[0067] The weight threshold is dynamically updated. After the batch test is completed, the system updates the weight threshold based on the evaluation score. Calculate the new fusion weights and decision threshold adjustment coefficients, and calculate the average score across multiple batches. The fine-tuned values ​​of the multi-agent feature fusion weights are obtained. And the threshold for fine-tuning the authenticity judgment , in, and The adaptive learning rate parameter, calculated according to the above update formula, is used to fine-tune the multi-agent feature fusion weights and the high / low thresholds for true / false judgments, forming a feedback loop to achieve adaptive performance optimization. The update trigger condition is when the score at time t satisfies: At this point, the current strategy can be considered globally optimal, and the system enters the long-term deployment phase.

[0068] The collaborative decision-making agent module effectively constructs a self-learning and self-optimizing intelligent decision-making system through value function calculation and policy evaluation sub-modules. It not only integrates multi-source information processed by multiple agents but also utilizes reinforcement learning techniques to continuously improve judgment criteria, achieving flexible, efficient, and accurate determination of the authenticity of handwritten signatures, making it suitable for complex and ever-changing real-world application scenarios.

[0069] In summary, this invention utilizes a multi-agent collaborative processing mechanism to acquire offline static features of handwritten signatures and corresponding online dynamic sequences of handwritten signatures. It records and labels the state and environmental information of handwritten signatures, extracts static features and corresponding temporal features, and obtains handwriting level and stability features, as well as classification labels. Through a reinforcement learning mechanism, it continuously optimizes the verification strategy, dynamically integrates the output of the multi-agent integrated processing module, and extracts and analyzes multi-dimensional features of handwritten signatures to accurately determine signature authenticity, further improving the accuracy of handwritten signature verification. Simultaneously, it employs an adaptive optimization strategy based on reinforcement learning, using agents to automatically adjust thresholds and weights, further enhancing the stability and generalization of verification, maintaining good recognition accuracy under different quality conditions and complex scenarios.

[0070] like Figure 2 The diagram shows a structural block diagram of an exemplary electronic device that can be used to implement embodiments of this application. The electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 302 or a computer program loaded from a storage unit 308 into a random access memory (RAM) 303. The RAM 303 may also store various programs and data required for the operation of the device 300. The computing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0071] Multiple components in electronic device 300 are connected to I / O interface 305, including: input unit 306, output unit 307, storage unit 308, and communication unit 309. Input unit 306 can be any type of device capable of inputting information to electronic device 300. Input unit 306 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device. Output unit 307 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 308 may include, but is not limited to, disk and optical disk. Communication unit 309 allows electronic device 300 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers, and / or chipsets, such as Bluetooth devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0072] The computing unit 301 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 301 performs the various methods and processes described above. For example, in some embodiments, the reconstruction and decomposition of the muscle movement trajectory based on the original trajectory of the signature stroke, and the decomposition of its logarithmic velocity curve, can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as storage unit 308. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 300 via ROM 302 and / or communication unit 309. In some embodiments, the computing unit 301 can be configured in any other suitable manner to perform signature handwriting comparison and verification methods.

[0073] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0074] In the context of this application, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0075] As used in this application, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or apparatus (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receive machine instructions as machine-readable signals. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0076] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0077] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0078] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other.

Claims

1. A handwriting verification system based on multimodal data and multi-agent collaboration, characterized in that, The handwriting verification system includes a multimodal data acquisition module, a multi-agent integrated processing module, and a collaborative decision-making agent module. The multimodal data acquisition module synchronously acquires static images of handwritten signatures and corresponding online dynamic time-series data, records the writing environment and writing state, and classifies and labels writing level and stability. The multi-agent integrated processing module trains, processes, and classifies based on the static and dynamic data of electronic signatures, combined with writing level and stability labels, and outputs static image feature encoding vectors, dynamic time-series feature encoding vectors, and writing environment information features. The collaborative decision-making agent module continuously optimizes the handwriting authenticity verification strategy through a reinforcement learning mechanism, dynamically integrates the output of the multi-agent integrated processing module, and achieves accurate determination of the authenticity of the signature handwriting.

2. The system according to claim 1, characterized in that, The continuously optimized handwriting authenticity verification strategy includes a collaborative decision-making agent that repeatedly samples writing state-action pairs and performs iterative optimization of state-action based on the Bellman residual minimization criterion, so that the collaborative decision-making agent can reach the optimal verification strategy.

3. The system according to claim 2, characterized in that, The iterative optimization strategy to reach the optimal state includes: recording the state, actions, and immediate rewards at all writing moments to construct a data sample and store it in the experience pool. From the experience pool Small and medium batches of sampled data are used to optimize the target network parameters based on the data samples at any time i. When the global loss function converges and the difference between the Q values ​​before and after updating time i is less than the stability coefficient, that is, the average cumulative reward corresponding to the policy is stable, it is considered to have reached the optimal policy.

4. The system according to claim 3, characterized in that, Reinforcement learning and deep Q-networks utilize the state-action value function Measure in state Next action Expected cumulative return Q value According to the formula: ， Calculate the Q value, where, This represents the expectation of all possible trajectories under this strategy. For a moment Discount factor, For a moment The instant reward, Q-value updates follow the Bellman optimality equation to calculate the optimal reward value. : State-action value function Approximate the expected cumulative return function and determine the loss function. for: ， in, , in, To select an action in the next state The maximum return that can be obtained; For the target network parameters, The value function is the action that yields the maximum reward by selecting the target network parameters in the next state.

5. The system according to any one of claims 2-4, characterized in that, In the collaborative decision-making agent, the policy evaluation submodule determines whether the current policy converges and has generalization ability through statistical analysis, confidence estimation, and robustness assessment. It then performs threshold tuning and model updates. Specifically, this includes constructing a policy function that the policy evaluation submodule receives from the collaborative decision-making agent module, based on a signature dataset containing different user samples and multiple environmental perturbation conditions. Each sample consists of a mini-batch of sampled data, which is composed of a (state-action-reward-next state) quadruple. Simultaneously, it integrates offline static features, dynamic sequence features, and environmental features, and uses the Monte Carlo simulation method to randomly sample and evaluate the policy function of the evaluation submodule in multiple signature scenarios.

6. The system according to claim 5, characterized in that, The strategy function of the random sampling evaluation verification submodule includes: calculating the discounted sum of the return sequence from time t to the end, as the empirical return value for evaluating the strategy at time t. , Where T is the termination time step, and k is the offset relative to the current time t; for each batch test, the evaluation module calculates and calculates and statistics: false positive rate (FPR), false negative rate (FNR), accuracy (ACC), and expected cumulative return. The merits of the strategy functions of the integrated submodules are calculated by weighting the fusion weights.

7. The system according to claim 6, characterized in that, After the batch testing is completed, the score is further evaluated based on the current status. Calculate the new fusion weights and decision threshold adjustment coefficients, and calculate the average score across multiple batches. The threshold for fine-tuning the feature fusion weights of multiple agents is obtained. And the threshold for fine-tuning the authenticity judgment , in, and As the adaptive learning rate parameter, fine-tuning thresholds are used to fine-tune the multi-agent feature fusion weights and the high and low thresholds for true / false judgments, forming a feedback loop to achieve adaptive performance optimization. The score at time t satisfies: When the current strategy is the optimal strategy, then the current strategy is the optimal strategy.

8. The system according to any one of claims 1-7, characterized in that, In the multi-agent integrated processing module, a convolutional neural network architecture is trained to obtain an offline static image feature extractor. Static features, including signature texture and morphology, are extracted from the offline static features of handwritten signatures. The static signature is linearly mapped and transformed into a static feature encoding vector. An online dynamic sequence feature extraction agent is constructed using a recurrent neural network (RNN) or Transformer model. Based on the online dynamic signature sequence data, dynamic features such as signature stroke order, speed changes, and pressure fluctuations are obtained, and the dynamic features of the handwriting sequence are output. A lightweight convolutional neural network is used to train an agent for extracting environmental information features by utilizing handwriting level and handwriting stability features and classification labels. The output is an environmental information feature vector and the classification results of handwriting level and handwriting stability.

9. The system according to any one of claims 1-7, characterized in that, The process of converting a static signature into a static feature encoding vector includes: an offline static image feature extractor using a residual network to enhance the expression of static signature features, and an attention mechanism to highlight key areas of the static signature, thereby extracting a static signature feature encoding vector; the process of outputting a dynamic feature encoding vector for the handwriting sequence includes: acquiring the real-time trajectory point time-series information of the signature corresponding to the offline paper document image of the handwritten signature, constructing a multi-channel input sequence, performing deep learning modeling on the time-series information of each channel, training the model through a Transformer network architecture, obtaining the long-term dependencies between handwriting sequence data through a self-attention mechanism, obtaining a handwriting sequence feature extractor, which acts as an online dynamic sequence feature extraction agent, processes the signature trajectory point signal, and outputs a dynamic feature encoding vector for the handwriting sequence describing the writing rhythm and style of the signature sequence.

10. A handwritten signature verification method based on multimodal data and multi-agent collaboration, characterized in that, Simultaneously acquire static images of handwritten signatures and corresponding online dynamic time-series data, record the writing environment and writing status, and classify and label the writing level and stability. Based on the static and dynamic data of electronic signatures and the writing level and stability labels, train, process, and classify the data, and output the static image features, dynamic time-series features, and writing environment information features of the handwritten signatures. By continuously optimizing the handwriting authenticity verification strategy through reinforcement learning mechanisms and dynamically integrating the output of the multi-agent integrated processing module, the authenticity of signature handwriting can be accurately determined.

11. The method according to claim 10, characterized in that, The continuously optimized handwriting authenticity verification strategy includes a collaborative decision-making agent repeatedly sampling writing state-action pairs and iteratively optimizing the state-action pairs to the optimal verification strategy based on the Bellman residual minimization criterion.

12. The method according to claim 11, characterized in that, The iterative optimization strategy to reach the optimal state includes: recording the state, actions, and immediate rewards at all writing moments to construct a data sample and store it in the experience pool. From the experience pool Small and medium batches of sampled data are used to optimize the target network parameters based on the data samples at any time i. When the global loss function converges and the difference between the Q values ​​before and after updating time i is less than the stability coefficient, that is, the average cumulative reward corresponding to the policy is stable, it is considered to have reached the optimal policy.

13. The method according to claim 12, characterized in that, Using the state-action value function Measure in state Next action Expected cumulative return Q value According to the formula: ， Calculate the Q value, where, This represents the expectation of all possible trajectories under this strategy. For a moment Discount factor, For a moment The instant reward, Q-value updates follow the Bellman optimality equation to calculate the optimal reward value. : To select an action in the next state The maximum reward that can be obtained; in terms of state-action value function Approximate the expected cumulative return function and determine the loss function. for: ， in, , For the target network parameters, The value function is the action that yields the maximum reward by selecting the target network parameters in the next state.

14. The method according to any one of claims 11-13, characterized in that, Through statistical analysis, confidence estimation, and robustness assessment, the convergence and generalization capabilities of the current policy are determined. Threshold tuning and model updates are then performed. Specifically, this includes constructing a policy function for the policy evaluation submodule to receive the output of the collaborative decision-making agent module, based on a signature dataset containing different user samples and multiple environmental perturbation conditions. Each sample consists of a mini-batch of sampled data, which is composed of a (state-action-reward-next state) quadruple. Simultaneously, it integrates offline static features, dynamic sequence features, and environmental features, and uses the Monte Carlo simulation method to randomly sample and evaluate the policy function of the evaluation submodule in multiple signature scenarios.

15. The system according to claim 14, characterized in that, The strategy function of the random sampling evaluation verification submodule includes: calculating the discounted sum of the return sequence from time t to the end, as the empirical return value for evaluating the strategy at time t. , Where T is the termination time step, and k is the offset relative to the current time t; for each batch test, the evaluation module calculates and calculates and statistics: false positive rate (FPR), false negative rate (FNR), accuracy (ACC), and expected cumulative return. The submodule's policy is comprehensively evaluated using a weighted scoring function. Threshold tuning and model updates are performed through statistical analysis, confidence estimation, and robustness assessment. Specifically, this includes constructing a policy function for the policy evaluation submodule to receive the output from the collaborative decision-making agent module, based on a signature dataset containing different user samples and multiple environmental perturbation conditions. Each sample consists of a mini-batch of sampled data, which is composed of a (state-action-reward-next state) quadruple. Simultaneously integrating offline static features, dynamic sequence features, and environmental features, the strategy function of the evaluation submodule is evaluated and verified through random sampling in multiple signature scenarios using the Monte Carlo simulation method. The strategy function of the random sampling evaluation and verification submodule includes: calculating the discounted sum of the return sequence from time t to the end, as an empirical return value for evaluating the strategy at time t. , Where T is the termination time step, and k is the offset relative to the current time t; for each batch test, the evaluation module calculates and calculates and statistics: false positive rate (FPR), false negative rate (FNR), accuracy (ACC), and expected cumulative return. The merits of the submodule strategy functions are comprehensively evaluated by weighted fusion.

16. An electronic device comprising: processor; And a memory for storing a program, characterized in that the program includes instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 10-15.

17. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, in, The computer instructions are used to cause the computer to perform the method according to any one of claims 10-15.

Images