An intelligent evaluation method and system based on multi-modal and reinforcement learning

Abstract

The application relates to the field of artificial intelligence technology evaluation, and discloses an intelligent evaluation method and system based on multi-modal and reinforcement learning, which comprises the following steps: generating a feature tensor through improved data cleaning and cross-modal alignment processing; performing weight optimization based on a pre-training model and Bayesian inference; generating an optimized model through homomorphic encryption and secure aggregation; generating a weight matrix through reinforcement learning strategy iteration and stability verification; realizing blockchain notarization by combining hash encryption and a smart contract; and finally generating an evaluation report based on fuzzy comprehensive evaluation and visual processing. The application effectively improves the accuracy and robustness of multi-modal data evaluation, guarantees data security and traceability, and enhances the explainability and practicality of the result through dynamic weight optimization and visual output.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology evaluation, and in particular to an intelligent evaluation method and system based on multimodal and reinforcement learning. Background Technology

[0002] In the field of intelligent evaluation of technological achievements, existing solutions related to multimodal and reinforcement learning-based intelligent evaluation methods and systems typically employ single-modal data analysis and static weight allocation mechanisms. These solutions suffer from limitations such as large subjective scoring errors, fixed evaluation index weights, and low efficiency in processing multi-source data. Existing methods often rely on manual experience to set feature extraction rules and scoring weights, which can lead to cross-modal feature alignment deviations and insufficient adaptability to dynamic environments in multimodal scenarios involving patent images, test texts, and experimental data, failing to meet the objectivity and timeliness requirements of technological achievement evaluation. For the joint processing of patent images and test text data, existing technologies generally lack a collaborative optimization mechanism for text semantic understanding and image feature recognition, and fixed weight allocation models cannot respond to changes in the evaluation environment, resulting in decreased stability and reliability of evaluation results. Furthermore, traditional methods suffer from fragmented processes in multimodal dataset preprocessing, feature matrix generation, and dynamic weight iteration, making it difficult to achieve a closed-loop processing chain from data cleaning and feature alignment to weight optimization, leading to reduced evaluation efficiency and increased resource consumption. Summary of the Invention

[0003] This invention provides a multimodal and reinforcement learning-based intelligent evaluation method and system to address the problems of large subjective scoring errors, fixed weight allocation, and low data processing efficiency in the evaluation of existing technological achievements, based on the patented image and test text data and through the multimodal learning and reinforcement learning module links.

[0004] To address the aforementioned technical problems, this invention provides a multimodal and reinforcement learning-based intelligent evaluation method, comprising:

[0005] Raw data is obtained from patent images, test texts, and experimental data. Data cleaning based on improved Chebyshev norm and Shannon entropy, and cross-modal alignment based on an attention mechanism are performed to generate feature tensor structures. The patent images include schematic diagrams, structural diagrams, and flowcharts. The test texts cover patent specifications, claims, abstracts, and related technical documents, with text formats supporting plain text, rich text, and structured text. The experimental data includes numerical experimental results, time-series data, and multi-dimensional signals collected by sensors.

[0006] Based on the feature tensor structure, the pre-trained XGBoost model in the patent technology field is evaluated and the weight parameters are optimized based on Bayesian inference to generate a weight matrix to be optimized.

[0007] Perform homomorphic gradient encryption and aggregate computation based on secure multi-party computation and differential privacy to generate an optimized evaluation model structure;

[0008] Based on the optimized evaluation model structure, state-encoding-based reinforcement learning policy iteration and weighted moving average-based stability verification are performed to generate the final weight allocation matrix.

[0009] Data is obtained from the final weight allocation matrix, subjected to SHA-256 hash encryption and smart contract verification based on data legality and permission verification, and a blockchain evidence record is generated.

[0010] Based on the weight matrix and the optimized evaluation model structure, fuzzy comprehensive evaluation based on triangular or trapezoidal membership functions and visualization processing based on bar charts, line charts, radar charts or heat maps are performed to generate the final evaluation report structure.

[0011] Furthermore, the process of generating the feature tensor structure also includes:

[0012] We acquire patent images, test texts, and experimental data, and perform data cleaning and processing based on improved Chebyshev norm image quality detection and Shannon entropy text language consistency detection to obtain a standardized multimodal dataset.

[0013] Text feature vectors and image feature vectors are extracted from the preprocessed dataset, and cross-modal alignment processing based on an attention mechanism is performed to generate a joint feature matrix;

[0014] The joint feature matrix is ​​normalized based on the dimension of the learnable weight vector to generate a feature tensor.

[0015] Furthermore, the process of generating the weight matrix to be optimized also includes:

[0016] The feature tensor is input into the pre-trained XGBoost model in the patent technology field for initial evaluation calculation to obtain the initial score set;

[0017] The indicator weight parameters are extracted from the initial score set, and a credibility verification process based on Bayesian inference is performed to generate a weight parameter sequence.

[0018] The weight parameter sequence is subjected to outlier filtering based on box plot and local outlier factor algorithm to generate the weight matrix to be optimized.

[0019] Furthermore, the process of generating the optimized evaluation model structure also includes:

[0020] Obtain the model parameters of each federation node, perform homomorphic encryption processing, and obtain encrypted gradient data;

[0021] Valid update parameters are extracted from the cryptographic gradient, and aggregated computation based on secure multi-party computation and differential privacy is performed to generate global parameters.

[0022] The global parameters are subjected to model validation based on cross-validation and leave-one-out method to generate an optimized evaluation model structure.

[0023] Furthermore, the process of generating the final weight assignment matrix also includes:

[0024] Obtain the optimized evaluation model structure, initialize the state-encoding-based reinforcement learning agent, and generate the policy space;

[0025] Select a weight adjustment strategy from the policy space, perform improved Q-learning iterative calculation, and generate dynamic weights;

[0026] The dynamic weights are subjected to stability verification based on weighted moving average to generate the final weight allocation matrix.

[0027] Furthermore, the process of generating blockchain-based evidence records also includes:

[0028] Obtain the final weight allocation matrix, perform SHA-256 hash encryption, and generate encrypted data packets;

[0029] Extract key parameters from encrypted data packets, perform smart contract verification based on data validity and permission verification, and generate block data;

[0030] The block data is verified using a consensus mechanism based on chain links to generate evidence records.

[0031] Furthermore, the process of generating the final assessment report structure also includes:

[0032] Obtain the weight matrix and the optimized evaluation model structure, perform fuzzy membership degree calculation based on triangular or trapezoidal membership functions, and generate fuzzy scores;

[0033] Positive and negative ideal solutions are extracted from the fuzzy scores, and TOPSIS distance metric processing based on Euclidean distance is performed to generate evaluation results;

[0034] The evaluation results are visualized using bar charts, line charts, radar charts, or heat maps to generate the final evaluation report structure.

[0035] Furthermore, the expression for generating the feature tensor structure also includes:

[0036] Based on the improved Chebyshev norm form of quality assessment, the image quality coefficient is defined as follows: ,

[0037] in, Represents the cleaning quality coefficient of the i-th patent image; It is the summation index used to iterate through pixel blocks; This represents the total number of pixel blocks after the i-th type of image is divided; For the i-th class of images, the i-th Measured resolution of each pixel block; and Let be the mean and standard deviation of the ideal resolution of the i-th class of images, respectively; For the improved Chebyshev norm order;

[0038] when Time-triggered image enhancement algorithm;

[0039] Consistency scoring using an improved Shannon entropy computational language: ,

[0040] in, Let be the language entropy value of the j-th test text; j represents the category index of the text. It is the summation index used to iterate through all character categories; Number of character categories; Indicates the first In the text class, the first The frequency of occurrence of each character; It is a smoothing factor; Indicates the first Total number of characters in the text class;

[0041] Achieving cross-modal alignment using an improved attention mechanism:

[0042] in, Align the attention matrix between text and image; and It is a trainable projection matrix; These are the projection dimension parameters; This represents the semantic feature vector of the c-th text sample; The visual feature vector representing the d-th image sample; These are sample indices; c corresponds to the text modality, and d corresponds to the image modality. This is the normalization function; This represents the matrix transpose operation;

[0043] After dynamically adjusting the feature weights in the attention matrix, a joint feature matrix is ​​generated. :

[0044]

[0045] An improved dimension normalization process is performed on the joint feature matrix to construct the feature tensor:

[0046] in, Indicates the first Normalized feature tensor of a mode; Represents the joint characteristic matrix belonging to the first That part of the modality; and The joint characteristic matrix is ​​at the th Mean and standard deviation in the modal dimension; A learnable weight vector; It represents the Hadamardi (or Hadama) stack; is the numerical stability constant.

[0047] Furthermore, the process of generating the final weight assignment matrix also includes:

[0048] Constructing the state space of a reinforcement learning agent:

[0049]

[0050] in, Let be the state vector at time t; Represents the state coding function; An environmental parameter vector; To optimize the model's weight parameters; This represents a vector concatenation operation;

[0051] State vector initialization policy exploration space :

[0052]

[0053] in, Indicates the first The numerical values ​​of the weight adjustment actions; This represents the m-th optional action;

[0054] Furthermore, an improved Q-learning algorithm is used for policy iteration:

[0055]

[0056] in, The learning rate is dynamic. Discount factor; For instant rewards; Indicates the state Next action Expected value; Indicates the next state Below, all possible actions The highest Q-value estimate; It is the action space Any action within; This represents the old action value function before the update;

[0057] Dynamic weight parameters are generated after 20 rounds of iteration. ;

[0058] For dynamic weight parameters Perform improved weighted moving average processing:

[0059]

[0060] in, Assign the final weight matrix; For smoothing coefficients; Let be the weight matrix at time t.

[0061] Furthermore, a multimodal and reinforcement learning-based intelligent evaluation system, applied to any of the methods described above, includes:

[0062] The environmental calibration module is used to acquire evaluation environmental parameters and optimization models, and to complete system initialization calibration.

[0063] The multimodal acquisition module is used to acquire patent images, test texts, and experimental data, and output a standardized multimodal dataset within a defined framework.

[0064] The feature processing module is used to extract text feature vectors and image feature vectors and generate a joint feature matrix;

[0065] The arbitration module is enhanced to perform iterative calculations of the strategy exploration space and output dynamic weight parameters;

[0066] The decision execution module is used to receive dynamic weight parameters and generate multimodal evaluation results;

[0067] The parameter update module is used to record the weight allocation timestamp and update the strategy exploration space parameters.

[0068] The key innovations of this invention include:

[0069] (1) By using multimodal learning feature extraction technology, BERT model and CNN technology are integrated to achieve joint processing of text data and image data, and generate a cross-modal aligned joint feature matrix.

[0070] (2) The dynamic weight allocation algorithm based on reinforcement learning uses Q-learning to iteratively calculate and dynamically adjust the evaluation weights to generate the final weight allocation matrix.

[0071] (3) By combining federated learning and blockchain technology, cross-agency data sharing and dispute resolution can be achieved, ensuring data security and the objectivity of evaluation results.

[0072] The following are its main beneficial effects:

[0073] (1) By jointly processing text data and image data, the accuracy of feature extraction and the alignment effect of multimodal data are improved, and the scoring error caused by feature alignment deviation is reduced, which is suitable for multi-source data evaluation scenarios.

[0074] (2) Dynamically adjusting the evaluation weights enhances the evaluation model's adaptability to environmental changes, improves the stability and reliability of the evaluation results, and is suitable for dynamic evaluation environments.

[0075] (3) By combining federated learning with blockchain, the security of data is ensured during cross-institutional sharing, the risk of data leakage is reduced, and the objectivity and credibility of the evaluation results are improved through decentralization. Attached Figure Description

[0076] Figure 1 A flowchart illustrating an intelligent evaluation method using multimodal and reinforcement learning, provided as an embodiment of this application;

[0077] Figure 2 This is a structural block diagram of an intelligent evaluation system based on multimodal and reinforcement learning, provided in an embodiment of this application. Detailed Implementation

[0078] Example 1: Refer to Figure 1 This is a flowchart illustrating an intelligent evaluation method using multimodal and reinforcement learning provided in an embodiment of the present invention. The flowchart may include at least steps S100-S600:

[0079] S100. Obtain raw data from patent images, test texts and experimental data, perform data cleaning based on improved Chebyshev norm and Shannon entropy and cross-modal alignment processing based on attention mechanism to generate feature tensor structure;

[0080] S200: Based on the feature tensor structure, perform evaluation of the pre-trained XGBoost model in the patent technology field and optimization of weight parameters based on Bayesian inference to generate a weight matrix to be optimized.

[0081] S300: Perform homomorphic gradient encryption and aggregate computation processing based on secure multi-party computation and differential privacy to generate an optimized evaluation model structure;

[0082] S400. Based on the optimized evaluation model structure, perform reinforcement learning policy iteration based on state encoding and stability verification processing based on weighted moving average to generate the final weight allocation matrix.

[0083] S500: Obtain data from the final weight allocation matrix, perform SHA-256 hash encryption and smart contract verification based on data legality verification and permission verification, and generate blockchain evidence records;

[0084] S600: Based on the weight matrix and the optimized evaluation model structure, perform fuzzy comprehensive evaluation based on triangular or trapezoidal membership functions and visualization processing based on bar charts, line charts, radar charts or heat maps to generate the final evaluation report structure.

[0085] Step S100 includes at least steps S110-S130:

[0086] S110. Obtain patent images, test texts, and experimental data, perform data cleaning and processing, and obtain a standardized multimodal dataset;

[0087] Specifically, the multimodal feature extraction module S100 first receives multi-source patent image data from the patent image acquisition interface. The patent images include various types such as technical solution diagrams, structural diagrams, and flowcharts, with image formats covering bitmaps, vector graphics, and scanned images. Simultaneously, the module obtains patent-related test text information from the test text receiving end. The test text covers patent specifications, claims, abstracts, and related technical documents, with text formats supporting plain text, rich text, and structured text. Further, it acquires experimental data related to the patent technology from the experimental data acquisition system. The experimental data includes numerical experimental results, time-series data, and multi-dimensional signals collected by sensors. For the acquired multimodal raw data, the module employs a data cleaning process. Specifically, it performs image quality detection on the patent images, including resolution detection, noise identification, and blur determination. Abnormal images are preprocessed using image enhancement algorithms, and images with severe blurring or missing data are marked as abnormal and stored in an anomaly log. For test text, a unified text format conversion is performed to remove redundant symbols and invalid characters. A language detection module from Natural Language Processing (NLP) is used to confirm text language consistency. Abnormal text triggers an anomaly handling mechanism and is recorded. Regarding experimental data, missing value detection and outlier removal are performed. Statistical methods are used to identify outliers, and abnormal data is imputed using interpolation or regression models. Data that cannot be repaired is marked and isolated. This data cleaning process combines a rule engine and a machine learning model, dynamically adjusting the cleaning strategy to adapt to the diversity of different patent fields. After data cleaning, the module performs standardization processing on patent images, test text, and experimental data, specifically including unified image size adjustment, color space conversion, text encoding standardization, and experimental data normalization, ensuring the compatibility and consistency of multimodal data in subsequent processing. The standardization process is controlled through a configuration parameter manager, supporting multiple preset standards and custom standards. All processing steps are logged with operation logs and anomaly information for easy traceability and debugging. Finally, step S110 outputs a standardized multimodal dataset with the field name "preprocessed dataset". This dataset serves as the input for the subsequent step S120, "preprocessed dataset", for joint extraction of cross-modal features. This output also supports subsequent modules such as the multimodal data generated by the initial score in S200, forming a complete data flow chain.

[0088] S120. Extract text feature vectors and image feature vectors from the preprocessed dataset, perform cross-modal alignment processing, and generate a joint feature matrix;

[0089] Specifically, the steps take the "preprocessed dataset" output by S110 as input. First, for the test text, a BERT (Bidirectional Encoder Representations from Transformers) model based on the Transformer architecture is used for text feature extraction. The BERT model loads a patent domain corpus through a pre-trained language model, combining word embedding and contextual semantic encoding to generate a high-dimensional semantic vector. This text feature vector contains multi-layered semantic expressions at the word and sentence levels. Further, for the patent images, the steps employ a Convolutional Neural Network (CNN) structure to extract image features, specifically including multiple convolutional layers, pooling layers, and fully connected layers, to extract texture features, edge information, and structural layout features. The image feature vector can characterize the authenticity information and key visual elements of the image. The text feature vector and image feature vector are respectively processed by feature normalization to make the feature dimensions and numerical ranges of different modalities comparable. Furthermore, the step employs a cross-modal alignment algorithm to map and fuse text and image features in the semantic space. Specifically, it includes a multimodal alignment module based on an attention mechanism, which adjusts feature weights using the principle of maximizing mutual information to achieve semantic correspondence matching. The cross-modal alignment process dynamically captures the correlation between text and images by constructing a multimodal attention matrix, eliminating the influence of heterogeneity between modalities. After alignment, the step concatenates the aligned text feature vectors and image feature vectors according to a predefined feature dimension order to form a unified joint feature matrix. This joint feature matrix structure supports multidimensional tensor representation and can reflect the interactive information of multimodal data. Throughout the processing flow, the step includes anomaly detection and fault tolerance mechanisms for input data. For feature extraction failures or alignment anomalies, compensation strategies or rollback mechanisms are triggered to ensure the integrity and accuracy of the joint feature matrix. Step S120 outputs the joint feature matrix, with the output field named "Joint Feature Matrix." This product serves as the input to the "Joint Feature Matrix" in step S130 and simultaneously provides unified feature input to the initial score generation module in S200, ensuring the multimodal data foundation for the evaluation model.

[0090] S130. Perform dimension normalization on the joint feature matrix to generate a feature tensor;

[0091] Specifically, the step takes the "joint feature matrix" output by S120 as input. First, a multi-dimensional normalization algorithm is executed. For each feature dimension in the joint feature matrix, Min-Max normalization, Z-score standardization, or adaptive normalization methods are used to eliminate differences in feature scales and improve the balance of feature distribution. The normalization process dynamically selects an appropriate algorithm based on the feature type; linear normalization is used for numerical features, and one-hot encoding followed by normalization is used for categorical features. Further, the step converts the normalized joint feature matrix into a high-order tensor structure. The feature tensor is defined as a multi-dimensional array with unified dimension identifiers and indexing rules, supporting tensor operations and parallel processing. The feature tensor structure is designed to support batch data input, facilitating subsequent machine learning model processing and compatibility with the data format requirements of various deep learning frameworks. During feature tensor generation, the step includes an outlier detection mechanism to identify extreme values ​​and perform truncation or smoothing to ensure the stability of the tensor data. This step simultaneously annotates the metadata of the feature tensor, including feature source, dimensional information, and normalization parameters, facilitating model interpretation and debugging. Step S130 outputs a standardized feature tensor, with the output field named "Feature Tensor." This output serves as the input to the "Feature Tensor" in the initial score generation step S200, while also providing a unified, high-quality multimodal feature foundation for the federated learning optimization module S300 and the dynamic weight adjustment module S400, supporting cross-module data flow and collaborative processing.

[0092] In another embodiment, in step S110, bitmap and vector image data from the patent image acquisition interface are first received. Combined with the structured text of the claims from the test text receiving end and the multi-dimensional signals from the experimental data acquisition system, a standardized multimodal dataset is constructed through a composite data cleaning process. A quality assessment based on an improved Chebyshev norm is then performed on the patent images, defining the image quality coefficient: Formula ①

[0093]

[0094] in:

[0095] The cleaning quality coefficient of the i-th type of patent image is derived from the resolution detection data of the patent image acquisition interface.

[0096] It is the summation index used to iterate through pixel blocks;

[0097] This represents the total number of pixel blocks after the i-th type of image is divided;

[0098] For the i-th class of images, the i-th Measured resolution of each pixel block;

[0099] and These are the mean and standard deviation of the ideal resolution of the i-th type of image, respectively, derived from a pre-set patent image standard database;

[0100] An improved Chebyshev norm order is used to enhance the detection sensitivity of anomalous pixel blocks;

[0101] This formula transforms the raw image data into an intermediate indicator that can determine the cleaning quality. The image enhancement algorithm is triggered on time. For the test text data, an improved Shannon entropy is used to calculate the language consistency score: Formula ②

[0102]

[0103] in:

[0104] The language entropy value of the j-th type of test text is derived from the character frequency statistics of the NLP language detection module;

[0105] It is a summation index used to iterate through all character categories. The range is from 1 to ;

[0106] For the number of character categories, =5000 represents the size of the patent corpus;

[0107] Indicates the first In the text class, the first The frequency of each character is derived from word frequency analysis after removing redundant symbols from the text;

[0108] It is a smoothing factor used to handle the zero-probability problem of low-frequency characters;

[0109] Indicates the first Total number of characters in the text class;

[0110] The output field of this step is named "Preprocessed Dataset", which is consumed by "Preprocessed Dataset" of S120 and used as the input source for cross-modal feature extraction.

[0111] In step S120, the preprocessed dataset is used to extract text semantic vectors using the BERT model. CNN models extract image feature vectors Cross-modal alignment is achieved using an improved attention mechanism: Equation ③

[0112]

[0113] in:

[0114] The text-image alignment attention matrix is ​​derived from the image data processed by Formula ① and the text data processed by Formula ②.

[0115] and This is a trainable projection matrix, and its parameters are derived from the initialization configuration of the cross-modal alignment module;

[0116] These are the projection dimension parameters;

[0117] The semantic feature vector representing the c-th text sample is derived from the BERT model. ;

[0118] The visual feature vector representing the d-th image sample is derived from the CNN model. ;

[0119] is the sample index, c corresponds to the text modality, and d corresponds to the image modality.

[0120] The normalization function normalizes each row of the matrix so that the sum of all rows is 1, representing the attention weight distribution;

[0121] This represents the matrix transpose operation.

[0122] After dynamically adjusting the feature weights, the attention matrix generates a joint feature matrix. The output field of this step is named "Joint Feature Matrix", which is consumed by "Joint Feature Matrix" in S130.

[0123] Step S130 performs improved dimension normalization on the joint feature matrix to construct the feature tensor: Formula ④

[0124]

[0125] in:

[0126] Indicates the first The normalized feature tensor of the modality is derived from the joint feature matrix generated by formula ③;

[0127] Represents the joint characteristic matrix belonging to the first That part of the modality;

[0128] and The joint characteristic matrix is ​​at the th Mean and standard deviation in the modal dimension;

[0129] The weight vector is a learnable weight vector, and the parameters are derived from the preset standards of the configuration parameter manager;

[0130] This represents the Hadamard product, which is element-wise multiplication.

[0131] is the numerical stability constant.

[0132] This section summarizes the technical effects: Multimodal data cleaning is achieved through improved Chebyshev quality assessment and Shannon entropy language analysis. A joint feature matrix is ​​generated by aligning the data using an attention mechanism, and finally, a dimension-normalized feature tensor structure is constructed to provide high-dimensional fusion feature input for subsequent model evaluation.

[0133] Step S200 includes at least steps S210-S230:

[0134] S210. Input the feature tensor into the XGBoost model to perform initial evaluation calculations and obtain the initial score set.

[0135] Specifically, the step takes the "feature tensor" output by S130 as input. First, the feature tensor undergoes format adaptation processing, converting it to the feature matrix format required by the XGBoost (eXtreme Gradient Boosting) model, ensuring compatibility between the data structure and the model input interface. The XGBoost model is an ensemble learning algorithm based on Gradient Boosting Trees, capable of handling high-dimensional sparse data and multimodal features. Further, the step loads a pre-trained XGBoost evaluation model, trained on large-scale training samples in the patent technology field. This model contains multiple decision trees, each splitting based on different dimensions of the feature tensor, calculating the leaf node output corresponding to the sample. The initial evaluation calculation process includes traversing the feature tensor sample by sample, combining the decision tree model for feature weighting and gradient updates, generating corresponding predicted scores. The scores cover multiple evaluation dimensions, such as innovativeness, practicality, and technological maturity, with each dimension corresponding to an independent score output. During runtime, the step dynamically adjusts the weight allocation of input features based on the model's feature importance index, improving the accuracy of the evaluation. The anomaly detection module runs in parallel, performing interpolation imputation or removal operations for outliers or missing values ​​in the input feature tensor to ensure the stability and robustness of the model computation. This step also records the model inference log, including the input feature distribution, scoring results, and computation time, facilitating subsequent analysis and debugging. Finally, step S210 outputs the initial score set for each dimension, with the output field named "Initial Score Set." This score set serves as the input for the subsequent "Initial Score Set" in step S220, and also provides basic scoring data for the S400 dynamic weight adjustment module, supporting weight optimization and strategy iteration.

[0136] S220. Extract the indicator weight parameters from the initial score set, perform credibility verification processing, and generate a weight parameter sequence.

[0137] Specifically, the steps take the "initial score set" output by S210 as input. First, statistical analysis is performed on the scores of each dimension to calculate the mean, variance, and skewness of the score distribution, which are used to assess the central tendency and dispersion of the scores. Further, based on a preset confidence threshold, the steps verify the confidence of the weight parameters of each indicator in the initial score set, combining historical score data and an expert rule base. The confidence verification includes confidence interval estimation and confidence score calculation, using Bayesian inference to quantify the confidence of the weight parameters and identify abnormal fluctuations or potential biases in the scores. The steps also incorporate multi-source data fusion technology to compare score results from different evaluation nodes, using weighted averaging and confidence weighting mechanisms to enhance the stability and consistency of the weight parameters. For weight parameters with low confidence, the steps implement a weight correction strategy, specifically including weight smoothing, interval adjustment, and anomaly removal, to ensure the rationality of the weight sequence and optimization space. The steps also introduce a rule-based anomaly detection module to automatically trigger alarms and record anomaly logs for sudden changes and extreme values ​​in the weight parameters, facilitating subsequent auditing. After verifying the credibility of the weight parameters in the above steps, a weight parameter sequence that meets the optimization requirements is generated. The weight sequence is a one-dimensional array containing the weight values ​​and confidence indicators of each evaluation index. The weight sequence structure is designed to support dynamic adjustment and online updates, facilitating policy iteration in subsequent reinforcement learning modules. The weight parameter sequence output in step S220, with the output field named "Weight Parameter Sequence," serves as the input to the "Weight Parameter Sequence" in step S320 and provides basic data for weight optimization in the dynamic weight adjustment module S400, supporting further filtering and matrix construction of weight parameters.

[0138] S230. Perform outlier filtering on the weight parameter sequence to generate the weight matrix to be optimized.

[0139] Specifically, the step takes the "weight parameter sequence" output by S220 as input. First, outlier detection is performed on the weight sequence using a box plot and Local Outlier Factor (LOF) algorithm to identify outliers in the weight data. The outlier filtering module then removes or replaces weight parameters marked as outliers based on the detection results. Replacement operations include filling with the mean of neighboring weights and interpolation based on historical trends. Further, the step converts the filtered weight parameter sequence into a weight matrix structure. This weight matrix is ​​a two-dimensional array where rows represent evaluation index categories and columns represent weight allocation time points or sample batches, supporting dynamic adjustment and historical tracing of multi-dimensional weights. The weight matrix structure includes metadata fields recording weight sources, filtering status, and timestamp information for easy model training and audit tracking. This step leverages the sparsity of the weight matrix, applying compression storage and indexing techniques to improve data access efficiency. The next step further normalizes the weight matrix using the L1 norm normalization method to ensure that the sum of the weights in the matrix is ​​a unit value, meeting the input requirements of the reinforcement learning optimization algorithm. An anomaly handling module runs throughout the entire process, implementing fault tolerance mechanisms, including automatic resampling and weight reallocation, to address data gaps and anomalies during the weight matrix construction process, ensuring the integrity of the output results. After the above steps are completed, the weight matrix to be optimized is output, with the output field named "Weight Matrix to be Optimized". This weight matrix serves as the input to step S320 and also provides a structured weight data foundation for the S400 dynamic weight adjustment module, supporting iterative optimization of the reinforcement learning strategy.

[0140] Step S300 includes at least steps S310-S330:

[0141] S310. Obtain the model parameters of each federation node, perform gradient encryption processing, and obtain encrypted gradient data.

[0142] The step takes the locally trained model parameters from each federated node as input. Specifically, the model parameters of each federated node are intermediate products obtained after local training by each distributed computing unit, and their specific contents include:

[0143] Core gradient data: The model gradient vector is calculated by each node based on its locally stored multimodal feature tensor and the weight matrix to be optimized, through the local evaluation model. Specifically, it is represented by the update amount of the weights and bias parameters.

[0144] Secure Ancillary Data: To ensure secure and private transmission, the gradient data includes parameters required for homomorphic encryption, hash digests for integrity verification, and node digital signatures.

[0145] Training context data: Metadata describing local training attributes, mainly including the number of local samples, computation timestamp, and training iteration number. This data is used to assign appropriate weights to different nodes in the aggregation calculation.

[0146] The federated nodes comprise multiple distributed computing units. Each unit, based on its own multimodal feature tensor and weight matrix, performs local model training to generate corresponding model gradient information. For the model gradients, gradient encryption is first performed using a homomorphic encryption algorithm. This algorithm supports gradient addition and multiplication operations within the encrypted domain, ensuring data privacy and security during transmission and aggregation. Specifically, this step encrypts the gradient vector of each federated node, generating irreversible encrypted gradient data. The encryption process includes key generation, encryption algorithm initialization, and encryption operations. The key management module is responsible for key distribution and updates, ensuring key security and availability. Further, this step includes a gradient integrity verification mechanism, performing hash digest calculations on the encrypted gradient data to generate a data fingerprint for subsequent integrity verification and anomaly detection. This step also implements an abnormal gradient identification mechanism, triggering retransmission or local retraining for abnormal or fluctuating gradient data to ensure the accuracy and availability of transmitted data. During data transmission, the encrypted gradients are transmitted via secure communication protocols (such as TLS / SSL) to prevent man-in-the-middle attacks and data leakage. Throughout the entire processing flow, the steps dynamically adjust the update frequency and data size of model parameters for each federated node to balance communication overhead and model update efficiency. Step S310 outputs encrypted model gradient data, with the output field named "encrypted gradient." This data serves as the input for the subsequent "encrypted gradient" in step S320 and also provides an encrypted data foundation for the S500 blockchain notarization module, enabling secure data sharing across institutions.

[0147] S320. Extract effective update parameters from the encryption gradient, perform aggregation calculations, and generate global parameters.

[0148] Specifically, the step takes the "encrypted gradient" output by S310 as input. First, it performs format parsing and preprocessing on the received encrypted gradient data, removing invalid or duplicate data packets to ensure the data quality for subsequent aggregation calculations. Further, the step employs Secure Multiparty Computation (SMPC) technology supported by homomorphic encryption to perform weighted aggregation of the encrypted gradients. This weighted aggregation dynamically allocates weights based on the sample size and model contribution of each federated node. Specifically, the aggregation calculation module adds the encrypted gradients of each node according to preset weights to obtain the overall encrypted gradient vector, which maintains privacy protection within the encrypted domain. This step incorporates a differential privacy mechanism, introducing noise perturbation during the aggregation process to balance model performance and privacy protection requirements. The aggregation result is then partially decrypted by the key management module to extract valid update parameters, including weight adjustment coefficients and bias update vectors, reflecting the latest training state of the global model. The step also integrates model version control functionality, recording the parameter version number and timestamp for each aggregation, facilitating model iteration management and backtracking. The anomaly detection module runs in parallel, performing filtering and correction operations for outliers or inconsistencies in the aggregated parameters to ensure the stability of the global model parameters. After the above steps are completed, global model parameters conforming to the specified format are generated, with the output field named "Global Parameters". This parameter serves as the input for the "Global Parameters" in the subsequent S330 step, and also provides the latest model base data for the S400 dynamic weight adjustment module, supporting intelligent iteration of the weight strategy.

[0149] S330. Perform model validation processing on the global parameters to generate an optimized evaluation model structure;

[0150] Specifically, the steps take the "global parameters" output by S320 as input. First, the global parameters are loaded into the evaluation model framework, which is based on an ensemble learning structure and supports the fusion of multimodal features and score calculation. Further, the steps perform model consistency verification, using cross-validation and leave-one-out method to evaluate the model's prediction performance. Verification metrics include accuracy, recall, F1 score, and mean squared error, ensuring the generalization ability of the global parameters on different subset datasets. The steps also perform model stability detection, evaluating the convergence and robustness of the model during training by monitoring the fluctuation range of the model output and the gradient change trend. For anomalies or performance degradation discovered during verification, the steps trigger a model fine-tuning process, automatically adjusting some parameters or retraining local sub-models to improve overall model performance. The model verification process incorporates automated logging, recording detailed input data, parameter changes, performance metrics, and anomalies during verification, supporting subsequent auditing and optimization. After verification, the steps generate an optimized model structure conforming to the system interface specifications. This structure includes model weight parameters, feature mapping relationships, and scoring strategy configurations, supporting dynamic loading and online updates. The optimized model structure is encapsulated as a modular component, facilitating cross-platform deployment and invocation. Step S330 outputs the optimized evaluation model structure, with the output field named "Optimized Evaluation Model Structure". This structure serves as the input for the subsequent step S410 and provides a high-quality model foundation for the S600 anti-interference scoring aggregation module, supporting the generation of the final evaluation results.

[0151] Step S400 includes at least steps S410-S430:

[0152] S410. Obtain the optimized evaluation model structure, initialize the reinforcement learning agent, and generate the policy space.

[0153] Specifically, the step takes the "optimized evaluation model structure" output by S330 as input. First, it receives current evaluation environment parameters from the system environment monitoring module. These parameters include, but are not limited to, multi-dimensional information such as evaluation task type, data distribution characteristics, historical scoring trends, and external interference indicators. These environment parameters are continuously updated through a real-time data acquisition interface and stored in an environment parameter cache. Further, based on the structure and weight configuration of the optimized model, the step loads the corresponding reinforcement learning framework. The reinforcement learning agent is constructed based on the policy gradient method and includes a state representation layer, a policy network, and a value network, supporting multi-dimensional state input and continuous action output. Specifically, the state representation layer fuses and encodes the current environment parameters and model weight parameters to form a high-dimensional state vector. The policy network adopts a deep neural network structure, combining convolutional layers and fully connected layers, mapping the state vector to an action probability distribution. The action space is defined as a set of weight adjustment policies, covering operations such as weight increase / decrease, weight redistribution, and weight freezing. The value network is used to estimate the expected return of the current policy, providing a value benchmark for policy optimization. The next step further generates a policy exploration space based on preset policy initialization rules. This space includes an initial policy set and its corresponding probability distribution, supporting an epsilon-greedy policy selection mechanism to ensure policy diversity and exploratory nature. During agent initialization, a state normalization module normalizes input parameters to ensure consistent numerical ranges across different parameter dimensions, preventing gradient vanishing or exploding during training. An anomaly detection mechanism is implemented throughout the initialization process, triggering warnings and recording anomaly logs for abnormal fluctuations in environmental parameters or model weights, supporting subsequent auditing and debugging. After agent initialization, the next step outputs the policy exploration space, named "Policy Space." This policy space serves as the input for the subsequent S420 step's "Policy Space" and provides foundational data for the dynamic weight adjustment module's policy iteration, supporting continuous learning for weight optimization.

[0154] S420. Select a weight adjustment strategy from the policy space, perform Q-learning iterative calculation, and generate dynamic weights;

[0155] Specifically, the step takes the "policy space" output by S410 as input. First, based on the current state vector and the policy probability distribution, an ε-greedy policy selection method is used to select a weight adjustment policy from the policy space. The policy is defined as an adjustment operation on the weights of the evaluation index, including weight increment adjustment, weight ratio redistribution, and weight threshold constraints. Further, the step constructs an action value function Q(s,a) based on the Q-learning algorithm, where state s consists of current environmental parameters and optimized model weights, and action a corresponds to the selected weight adjustment policy. The Q-learning iterative process includes state evaluation, action execution, reward calculation, and Q-value update. Specifically, after executing the selected weight adjustment policy, the system calculates a new scoring error and model performance index through the evaluation module, which serves as an immediate reward signal r and is fed back to the agent. The step uses a Temporal Difference (TD) learning method to update the Q-value, and the update formula combines historical experience and future expectations using the learning rate and discount factor. To improve training efficiency, this step introduces an experience replay mechanism, storing the historical state-action-reward-new state quadruple in an experience pool and randomly sampling it for Q-value updates, reducing sample correlation. This step combines a dual-network structure of the target network and the main network, periodically synchronizing the target network parameters to stabilize the learning process. Furthermore, this step includes an action constraint module, applying upper and lower bound constraints to the weight adjustment strategy to prevent excessive weight shifts that could lead to model instability. An anomaly monitoring module monitors Q-value fluctuations and reward anomalies in real time, executing strategy rollback or resampling for abnormal situations to ensure the robustness of the iteration process. After completing multiple rounds of Q-learning iterations, this step generates dynamic weight parameters, which are one-dimensional arrays containing the real-time weight values ​​and confidence indicators of each evaluation metric. The dynamic weight parameters are normalized to satisfy the constraint that the weights are summed in units. This step outputs "Dynamic Weights," which serves as the input for the subsequent "Dynamic Weights" in step S430, and also provides real-time adjusted weight data to the S600 anti-interference scoring aggregation module, supporting adaptive optimization of the scoring.

[0156] S430. Perform stability verification on the dynamic weights and generate the final weight allocation matrix.

[0157] Specifically, the steps take the "dynamic weights" output by S420 as input. First, a time-series stability analysis is performed on the dynamic weight parameters. This analysis uses a sliding window technique to calculate the mean, variance, and autocorrelation coefficient of the weight parameters to determine the fluctuation range and trend of the weight sequence. Further, the steps combine Statistical Process Control (SPC) methods to monitor weight fluctuations using control charts, identify abnormal fluctuation points and potential weight drift, and trigger weight smoothing. The weight smoothing uses a Weighted Moving Average (WMA) algorithm, combining historical weight data with the current dynamic weights to generate smoothed weight values, reducing the impact of short-term anomalies. Simultaneously, the steps perform consistency verification on the weight parameters, using a correlation coefficient matrix to calculate the correlation between different weight dimensions, identifying highly correlated or conflicting weight combinations, and adjusting the weight allocation based on preset weight constraint rules to prevent excessive weight concentration or mutual exclusion. The steps further convert the smoothed and verified dynamic weight parameters into a weight allocation matrix structure. This weight matrix is ​​a two-dimensional array, with rows representing each evaluation index category and columns representing time series or iterative batches, supporting historical tracking and version management of weights. The weight matrix includes metadata fields that record the weight source, stability indicators, and timestamp information, facilitating subsequent model auditing and dynamic adjustment. This step also integrates an anomaly logging module, which records in detail the anomaly type, time, and handling measures for any anomalies discovered during weight verification, supporting subsequent problem tracing. After completing weight stability verification and matrix generation, this step outputs the final weight allocation matrix, with the output field named "Final Weight Allocation Matrix." This weight matrix serves as the input for the subsequent S510 step "Weight Matrix," and simultaneously provides a stable and structured weight foundation for the S600 anti-interference scoring aggregation module, supporting improved robustness of the scoring results.

[0158] In another embodiment, in step S410, the optimized model parameters and real-time environmental monitoring data output from S330 are first received to construct the state space of the reinforcement learning agent:

[0159] Formula⑤

[0160]

[0161] in:

[0162] The state vector at time t is derived from the evaluation task type data and optimization model weights in the environmental parameter cache.

[0163] This represents the state encoding function, implemented as a three-layer fully connected neural network.

[0164] This is an environmental parameter vector, containing data distribution characteristics and historical scoring trend indicators;

[0165] The weight parameters of the model are derived from the global parameters aggregated through federated learning, in order to optimize the model.

[0166] This represents a vector concatenation operation;

[0167] The state vector initializes the policy exploration space. It includes 10 weight adjustment strategies. Among them, Indicates the first The numerical values ​​of the weight adjustment actions; This represents the m-th optional action, belonging to the space. ;

[0168] The output field of this step is named "Policy Space" and is consumed by the "Policy Space" of S420.

[0169] Step S420 uses an improved Q-learning algorithm for policy iteration:

[0170] Formula⑥

[0171]

[0172] in:

[0173] The learning rate is dynamic and automatically adjusted based on the stability of the state vector generated by formula ⑤.

[0174] This is a discount factor used to balance immediate rewards and long-term benefits;

[0175] The immediate reward is derived from the scoring error of the XGBoost model;

[0176] Indicates the state Next action Expected value;

[0177] Indicates the next state Below, all possible actions The highest Q-value estimate;

[0178] It is the action space Any action within the range is used to iterate and find the maximum value;

[0179] This represents the old action value function before the update;

[0180] Dynamic weight parameters are generated after 20 rounds of iteration. The output field for this step is named "Dynamic Weights" and is consumed by the "Dynamic Weights" function of S430.

[0181] Step S430 performs an improved weighted moving average processing on the dynamic weights:

[0182]

[0183] in:

[0184] Assign the final weight matrix;

[0185] The smoothing coefficient is dynamically adjusted based on the monitoring results of the control chart.

[0186] Let K be the weight matrix at time t, where K=8 represents the number of evaluation index categories and T=50 is the time window length.

[0187] Technical effect of this section: By implementing weight strategy iteration through state space encoding and improved Q-learning, and combining time-varying smoothing coefficients to generate a weight matrix after stability verification, a structured dynamic weight foundation is provided for the blockchain evidence storage module.

[0188] Step 500 includes at least steps S510-S530:

[0189] S510. Obtain the final weight allocation matrix, perform hash encryption processing, and generate encrypted data packets;

[0190] Specifically, the step takes the "final weight allocation matrix" output by S430 as input. First, it receives transmission data containing local model update results and weight allocation information from each federation node. This transmission data includes encrypted gradients, weight matrix snapshots, and related metadata. The federation node comprises multiple distributed computing units, each corresponding to a different institution or data holder. For the transmission data, the step first performs data integrity verification by using a hash function to calculate a digest of the data content, generating a unique hash value for subsequent data consistency verification. The hash encryption process is based on a cryptographic hash algorithm, such as SHA-256 (Secure Hash Algorithm 256-bit), ensuring the immutability and uniqueness of the data fingerprint. Further, the transmission data is bound to its corresponding hash value to form a blockchain transaction data packet structure containing a data packet header, data body, and hash digest. The data packet structure design supports multi-level nesting and includes a transaction identifier, timestamp, sending node authentication information, and digital signature. The digital signature is implemented using an asymmetric encryption algorithm to ensure the legitimacy and non-repudiation of the data source. The steps also integrate a data anonymization module to anonymize sensitive information such as personal identification and trade secrets, including data masking, anonymization, and generalization techniques to meet cross-institutional data privacy protection requirements. An anomaly detection mechanism is implemented throughout the data packet generation process. For hash value mismatches, signature verification failures, or abnormal data formats, retransmission or alarm procedures are triggered, and all anomalies are recorded in the security log system. The steps transmit the generated blockchain transaction data packets to blockchain network nodes via secure communication protocols such as TLS (Transport Layer Security) / SSL (Secure Sockets Layer), supporting asynchronous transmission and batch processing, balancing transmission efficiency and security. Finally, step S510 outputs the generated blockchain transaction data packet, with the output field named "encrypted data packet." This data packet serves as the input for the subsequent "encrypted data packet" in step S520, providing an encrypted data foundation for cross-institutional data sharing and notarization, and supporting the operation of the blockchain notarization and verification module.

[0191] S520: Extract key parameters from encrypted data packets, perform smart contract verification processing, and generate block data;

[0192] Specifically, the steps take the "encrypted data packet" output by S510 as input. First, the encrypted data packet is parsed to extract key parameters such as transaction identifier, timestamp, sending node identity information, hash digest, and digital signature. The steps then execute verification logic by calling a blockchain smart contract. The smart contract is an automated code module pre-deployed on the blockchain, containing data validity verification, permission verification, transaction sequence management, and exception handling rules. Specifically, the smart contract first checks the uniqueness of the transaction identifier to prevent duplicate transactions or replay attacks. Next, it verifies the validity of the digital signature to confirm the identity and authorization scope of the data packet sending node. Further, the smart contract compares the hash digest to ensure the data packet content has not been tampered with. Combined with on-chain evidence storage history, it performs consistency verification to prevent data conflicts and forgery. The smart contract also includes a permission control module that dynamically adjusts the verification process and evidence storage strategy based on node roles and access permissions, supporting multi-level permission management and compliance audit requirements. The exception handling mechanism is integrated into the smart contract execution process. For transactions that fail verification, the smart contract automatically triggers exception event recording and pushes the exception information to the off-chain monitoring system for security auditing. The next step further encapsulates the verified encrypted data packet into evidence storage block data. This evidence storage block data includes transaction data, verification results, blockchain node signatures, and timestamp information, forming a standard blockchain block structure. The evidence storage block data supports cross-chain interaction and multi-chain synchronization, is compatible with main chain and side chain architectures, and meets the needs of cross-institutional multi-party collaboration. After generating the evidence storage block data, this step submits it to the consensus mechanism module of the blockchain network, preparing for on-chain evidence storage. Finally, step S520 outputs the generated evidence storage block data, with the output field named "Block Data." This data serves as the input for the subsequent "Block Data" in step S530, and also provides the basic data for the formation of the on-chain evidence storage record structure, ensuring the automation and reliability of the evidence storage process.

[0193] S530: Perform consensus mechanism verification on the block data and generate evidence storage records;

[0194] Specifically, the steps take the "block data" output by S520 as input. First, the block data is broadcast to each consensus node in the blockchain network. The consensus nodes verify and confirm the block data according to a preset consensus algorithm, which includes, but is not limited to, Proof of Stake (PoS), Proof of Work (PoW), and Byzantine Fault Tolerance (BFT) algorithms. The steps involve consensus nodes performing multiple rounds of verification on the validity, integrity, and transaction legality of the block data, combining timestamps and node signatures to ensure the block data conforms to network protocol specifications. Furthermore, the consensus mechanism performs a chain-linking operation on the block data, associating new blocks with previous blocks through hash pointers to form an immutable blockchain structure. The steps include a fork detection and handling module, which executes the longest chain rule or weighted chain rule to ensure the uniqueness and consistency of the chain in the event of a fork on the chain. The steps also incorporate incentive and penalty mechanisms in the consensus process to regulate node behavior and promote network security and stable operation. An anomaly handling mechanism monitors abnormal node behavior, latency, and attack attempts during the consensus process, triggering automatic isolation and alarm measures. After consensus confirmation is completed in the aforementioned steps, block data is written to the on-chain evidence storage database, generating an on-chain evidence storage record structure. This record structure includes block header information, a transaction list, verification status, and an on-chain index, supporting efficient querying and auditing. The on-chain evidence storage record structure provides access control through a smart contract interface, supporting cross-institutional data sharing and dispute resolution. Finally, step S530 outputs the generated on-chain evidence storage record structure, with the output field named "Evidence Storage Record." This record serves as input for the subsequent step S610, providing on-chain proof and traceability for overall system data security and trust assessment.

[0195] Step S600 includes at least steps S610-S630:

[0196] S610. Obtain the weight matrix and the optimized evaluation model structure, perform fuzzy membership degree calculation, and generate fuzzy scores.

[0197] Specifically, the steps take the "weight matrix" output by S430 and the "optimized evaluation model structure" output by S330 as input. First, the weight allocation of each evaluation indicator in the weight matrix is ​​read element-by-element. Then, combined with the feature mapping relationship and scoring strategy configuration in the optimized model, a multi-dimensional evaluation indicator space is constructed. The fuzzy membership degree calculation is based on fuzzy set theory. The membership degree is defined as the degree of membership of the evaluation indicator at different scoring levels, specifically including three fuzzy subsets: low, medium, and high. Triangular or trapezoidal membership functions are used for numerical mapping. The steps form a fuzzy evaluation matrix by weighting the weight values ​​in the weight matrix with the scoring outputs of the corresponding features in the optimized model. Matrix elements represent the degree of belonging of each indicator weight to different scoring levels. Further, the steps introduce a fuzzy rule base, containing fuzzy inference rules based on expert experience and historical data. The rules describe the fuzzy relationship between weights and scores in an "if-then" form, supporting inference based on multiple conditions and multiple results. The fuzzy inference process employs a Mamdani-type inference engine. Specifically, the fuzzy evaluation matrix is ​​used as input, and after rule matching, fuzzy synthesis, and membership aggregation, a fuzzy output set is obtained. This step includes normalization of fuzzy membership degrees, using a maximum-min method to adjust the range of membership degree values ​​to ensure numerical consistency in the output results. An anomaly detection module is integrated throughout the calculation process, performing interpolation or weight reallocation for outliers or missing data in the input weight matrix to ensure the continuity and completeness of membership degree calculation. This step also records a fuzzy calculation log, including input weights, membership function parameters, inference rule triggering status, and the output fuzzy set, supporting subsequent debugging and rule optimization. Finally, step S610 outputs an anti-interference scoring sequence generated by the fuzzy membership degree calculation, with the output field named "Fuzzy Score." This scoring sequence serves as the input for the subsequent "Fuzzy Score" in step S620, and also provides a multi-dimensional scoring basis based on fuzzy logic for the anti-interference scoring aggregation module, supporting improved robustness of the scoring results.

[0198] S620. Extract positive and negative ideal solutions from fuzzy scores, perform TOPSIS distance metric processing, and generate evaluation results;

[0199] Specifically, the step takes the "fuzzy score" output by S610 as input. First, the fuzzy score sequence undergoes data preprocessing, including missing value imputation and outlier removal. Statistical methods are used to identify extreme values ​​in the scores, and these are corrected through nearest neighbor interpolation or weighted averaging to ensure data integrity. Further, based on the TOPSIS (Technique for Order Preference by Similarity to Ideal Solution) algorithm framework in multi-attribute decision-making methods, a decision matrix is ​​constructed. Rows in the matrix correspond to different evaluation objects, and columns correspond to various fuzzy score indicators. The positive ideal solution is defined as the maximum score value among all indicators, representing the optimal evaluation level, while the negative ideal solution is defined as the minimum score value among all indicators, representing the worst evaluation level. The step normalizes the decision matrix using vector normalization, mapping each indicator score to the [0,1] interval to eliminate differences in the dimensions and magnitudes of different indicators. Further, the step combines the "weight matrix" output by S430 to assign corresponding weights to each indicator in the normalized matrix, forming a weighted normalized decision matrix. The distance measurement process includes calculating the Euclidean distance from each evaluation object to the positive and negative ideal solutions, denoted as D+ and D-, respectively. The distance calculation uses the Euclidean distance formula in multidimensional space, comprehensively considering the weighted score differences of various indicators. Based on the distance measurement results, this step calculates the relative proximity index, defined as the ratio of D- to the sum of D+, with a value ranging from 0 to 1. The closer the value is to 1, the closer the evaluation object is to the ideal solution. This step includes distance calculation anomaly detection, implementing an anomaly handling mechanism for numerical overflow, zero distance, or invalid distances during the calculation process, including re-normalization, weight adjustment, or removal of abnormal indicators. This step also integrates a multi-round iterative mechanism, dynamically adjusting the weight matrix and fuzzy scoring parameters based on historical evaluation results and feedback, improving the stability and accuracy of TOPSIS calculation. This step records the TOPSIS calculation log, including normalization parameters, weight distribution, distance values, and proximity index, facilitating subsequent evaluation result analysis and verification. Finally, step S620 outputs a standardized evaluation result, with the output field named "Evaluation Result". This result serves as the input for the subsequent step S630, "Evaluation Result", and also provides the anti-interference scoring aggregation module with comprehensive scoring data based on the TOPSIS algorithm, supporting the objective ranking and decision-making of the final evaluation.

[0200] S630. Visualize and render the evaluation results to generate the final evaluation report structure;

[0201] Specifically, the steps take the "evaluation results" output by the S620 as input. First, the evaluation result data is formatted to adapt to the data structure required by the visualization rendering engine, including JSON key-value pair mappings and multidimensional array representations, supporting various front-end display frameworks. Further, based on a predefined evaluation report template, and combining key indicators and comprehensive scores from the evaluation results, the steps automatically generate structured report content, covering the total score, score distribution across dimensions, weight allocation explanations, and historical trend comparisons. The report structure supports multi-level directories and chapter divisions, facilitating quick browsing and in-depth analysis. The visualization rendering process employs various graphical representations, including bar charts, line charts, radar charts, and heatmaps, with the specific chart type dynamically selected based on the nature of the evaluation indicators and user configuration. The steps utilize a graphics rendering engine to create charts, supporting interactive operations such as zooming, hovering tips, and data filtering, enhancing report readability and user experience. Furthermore, the steps integrate anomaly data annotation functionality, highlighting or annotating points or indicators detected during the evaluation process in the report through highlights, icons, or annotations to assist users in identifying potential problems. The steps also integrate an improvement suggestion generation module. Based on the evaluation results and historical data, it calls a preset improvement strategy library to automatically generate targeted suggestion text, covering technical optimization directions, data quality improvement, and model adjustment schemes. The report generation process includes access control and data anonymization mechanisms, dynamically adjusting the visibility of report content and the display of sensitive information for different user roles. An anomaly handling module monitors format errors, data missingness, and rendering failures during the report generation process, performing automatic repairs or manual alerts to ensure the integrity and accuracy of the report output. Finally, step S630 outputs the final evaluation report structure, with the output field named "Final Evaluation Report Structure." This report serves as direct data output by the system, used for user interface display, archiving, and subsequent analysis, while also providing closed-loop feedback data for the overall system evaluation process.

[0202] Example 2: Figure 2 A block diagram of a multimodal and reinforcement learning-based intelligent evaluation system according to an embodiment of the present invention is shown. Figure 2 As shown, the structure may include:

[0203] The environment calibration module 01 is used to acquire evaluation environment parameters and optimize the model, and complete system initialization calibration. Specifically, it receives evaluation environment parameters from the system environment monitoring module. These parameters include multi-dimensional information such as evaluation task type, data distribution characteristics, historical scoring trends, and external interference indicators. This information is continuously updated through a real-time data acquisition interface and stored in the environment parameter cache. Combining the structure and weight configuration of the optimization model, a reinforcement learning framework is loaded to construct an agent using the policy gradient method. This agent includes a state representation layer, a policy network, and a value network, supporting multi-dimensional state input and continuous action output. The state representation layer fuses and encodes the current environment parameters and model weight parameters to form a high-dimensional state vector. The policy network maps the state vector to the action probability distribution, and the action space is defined as a set of weight adjustment policies. The value network estimates the expected return of the current policy, providing a value benchmark. A policy exploration space is generated, containing an initial policy set and its probability distribution, supporting an ε-greedy policy selection mechanism to ensure policy diversity and exploratory nature. The state normalization module normalizes the input parameters to avoid gradient vanishing or exploding. An anomaly detection mechanism is implemented throughout the initialization process. For abnormal fluctuations in environment parameters or abnormal model weights, an alert is triggered and an anomaly log is recorded. After the agent is initialized, the policy exploration space is output and passed to the reinforcement arbitration module as the basic data for policy iteration.

[0204] The multimodal acquisition module 02 is used to acquire patent images, test text, and experimental data, and output a standardized multimodal dataset within a defined framework. Specifically, it receives multi-source patent image data from the patent image acquisition interface. Images include technical solution diagrams, structural diagrams, flowcharts, etc., covering bitmaps, vector graphics, and scanned images. Simultaneously, it acquires patent-related test text information from the test text receiving end. This text covers patent specifications, claims, abstracts, and related technical documents, supporting plain text, rich text, and structured text formats. The experimental data acquisition system provides experimental data related to the patent technology, including numerical experimental results, time-series data, and multi-dimensional signals collected by sensors. Data cleaning is performed on the acquired multimodal raw data. Patent images undergo quality inspection; abnormal images are preprocessed using image enhancement algorithms; and images with severe blurring or missing data are marked as abnormal and stored in an anomaly log. Test text undergoes unified format conversion, and NLP technology is used to confirm text language consistency. Abnormal text triggers an anomaly handling mechanism and is recorded. Experimental data undergoes missing value detection and outlier removal. Abnormal data is filled using interpolation or regression models; data that cannot be repaired is marked and isolated. Data cleaning combines a rule engine with a machine learning model to dynamically adjust cleaning strategies to adapt to the diversity of different patent fields. After data cleaning, patent images, test texts, and experimental data are standardized to ensure the compatibility and consistency of multimodal data. The standardization process is controlled by a configuration parameter manager, supporting multiple preset standards and custom standards, and recording operation logs and anomaly information for easy traceability and debugging. The standardized multimodal dataset is output and passed to the feature processing module for joint extraction of cross-modal features.

[0205] Feature processing module 03 is used to extract text and image feature vectors and generate a joint feature matrix. Specifically, it receives a standardized multimodal dataset from the multimodal acquisition module. First, for the test text, a BERT model based on the Transformer architecture is used for text feature extraction. The BERT model loads a patent domain corpus through a pre-trained language model, combines word embeddings and contextual semantic encoding to generate high-dimensional semantic vectors. The text feature vectors contain multi-layer semantic expressions at the word and sentence levels. For the patent images, a convolutional neural network structure is used for feature extraction, including multiple convolutional layers, pooling layers, and fully connected layers, to extract texture features, edge information, and structural layout features of the images. The image feature vectors characterize the authenticity information and key visual elements of the images. The text and image feature vectors are respectively processed by feature normalization to make the feature dimensions and numerical ranges of different modalities comparable. A cross-modal alignment algorithm is used to map and fuse text and image features in the semantic space, construct a multimodal attention matrix, dynamically capture the correlation between text and images, and eliminate the influence of heterogeneity between modalities. After alignment, the aligned text feature vectors and image feature vectors are concatenated according to a predefined feature dimension order to form a unified joint feature matrix. This supports multi-dimensional tensor representation and reflects the interactive information of multimodal data. Input data anomaly detection and fault tolerance mechanisms ensure the integrity and accuracy of the joint feature matrix. The output joint feature matrix is ​​then passed to the reinforcement arbitration module for policy iteration calculation.

[0206] The enhanced arbitration module 04 is used to perform iterative calculations of the policy exploration space and output dynamic weight parameters. Specifically, based on the joint feature matrix from the feature processing module, policy selection and action execution are completed. First, based on the current state vector and the policy probability distribution, an ε-greedy policy selection method is used to select a weight adjustment policy from the policy space. The policy is defined as an adjustment operation on the weights of the evaluation index, including weight increment adjustment, weight ratio redistribution, and weight threshold constraints. Based on the Q-learning algorithm, an action value function Q(s,a) is constructed, where state s consists of the current environmental parameters and the optimized model weights, and action a corresponds to the selected weight adjustment policy. The Q-learning iterative process includes state evaluation, action execution, reward calculation, and Q-value update. After executing the selected weight adjustment policy, the system calculates the new scoring error and model performance index through the evaluation module, which are fed back to the agent as an immediate reward signal. The temporal difference learning method is used to update the Q-value, using the learning rate and discount factor to combine historical experience and future expectations. An experience replay mechanism is introduced, storing the historical state-action-reward-new state quadruple in the experience pool, and randomly sampling it for Q-value updates to reduce sample correlation. Combining a dual-network structure of a target network and a main network, the target network parameters are periodically synchronized to stabilize the learning process. An action constraint module applies upper and lower bounds to the weight adjustment strategy to prevent excessive weight shifts that could lead to model instability. An anomaly monitoring module monitors Q-value fluctuations and reward anomalies in real time, implementing policy rollback or resampling to ensure the robustness of the iteration process. After multiple rounds of Q-learning iterations, dynamic weight parameters are generated; a one-dimensional array contains the real-time weight values ​​and confidence levels of each evaluation metric. These dynamic weight parameters are output and passed to the decision execution module for use in generating multimodal evaluation results.

[0207] The decision execution module 05 receives dynamic weight parameters and generates multimodal evaluation results. Specifically, based on the dynamic weight parameters from the enhanced arbitration module, it generates multimodal evaluation results. First, it performs time-series stability analysis on the dynamic weight parameters, using a sliding window technique to calculate the mean, variance, and autocorrelation coefficient of the weight parameters to determine the fluctuation range and trend of the weight sequence. Combined with statistical process control methods, it monitors weight fluctuations using control charts, identifies abnormal fluctuation points and potential weight drift, and triggers weight smoothing. Weight smoothing employs a weighted moving average algorithm, combining historical weight data with the current dynamic weights to generate smoothed weight values, reducing the impact of short-term anomalies. Consistency verification is performed on the weight parameters, using a correlation coefficient matrix to calculate the correlation between different weight dimensions, identifying highly correlated or conflicting weight combinations, and adjusting weight allocation based on preset weight constraint rules to prevent excessive weight concentration or mutual exclusion. The smoothed and verified dynamic weight parameters are converted into a weight allocation matrix structure. The rows of this two-dimensional array represent each evaluation index category, and the columns represent time series or iterative batches, supporting historical tracking and version management of weights. The weight matrix includes metadata fields that record the source of the weights, stability metrics, and timestamp information, facilitating subsequent model auditing and dynamic adjustments. An integrated exception logging module records detailed exception types, timestamps, and handling measures for any anomalies discovered during weight verification, supporting subsequent problem tracing. After completing weight stability verification and matrix generation, the final weight allocation matrix is ​​output and passed to the parameter update module for weight allocation timestamp recording and strategy exploration space parameter updates.

[0208] The parameter update module 06 records weight allocation timestamps and updates strategy exploration space parameters. Specifically, it receives the final weight allocation matrix from the decision execution module and dynamic weight parameters from the reinforcement arbitration module, and performs weight allocation timestamp recording and strategy exploration space parameter updates. First, it performs time series analysis on the weight allocation matrix, combining historical weight allocation data with the current timestamp to generate a weight allocation time series, recording the temporal changes and version information of the weight allocation. Then, it updates the strategy probability distribution and action value function in the strategy exploration space using dynamic weight parameters, adjusting the search range and priority of the strategy space to support dynamic strategy optimization and exploration. The anomaly monitoring module detects anomalies during the time series recording and strategy update process, executing anomaly handling mechanisms, including time series reconstruction, strategy space reconfiguration, and weight parameter reassignment, ensuring the integrity and accuracy of the update process. After completing the weight allocation timestamp recording and strategy exploration space parameter updates, it outputs the updated strategy exploration space for the environment calibration module to use for parameter backfeedback and link refresh, forming a complete closed-loop control link.

Claims

1. An intelligent assessment method of multi-modal and reinforcement learning, characterized in that, The method comprises the following steps: Obtain raw data from patent images, test texts and experimental data, perform data cleaning based on improved Chebyshev norm and Shannon entropy, and cross-modal alignment processing based on attention mechanism to generate feature tensor structure; the patent images include technical scheme schematic diagram, structure diagram and flowchart; the test texts cover patent specification, claim, abstract and related technical literature, and the text format supports plain text, rich text and structured text; the experimental data includes numerical experimental results, time series data and multi-dimensional signals collected by sensors; Based on the feature tensor structure, perform XGBoost model evaluation pre-trained in the patent technology field and weight parameter optimization processing based on Bayesian inference to generate a weight matrix to be optimized; Perform homomorphic gradient encryption and aggregation calculation processing based on secure multi-party computation and differential privacy to generate an optimized evaluation model structure; Based on the optimized evaluation model structure, perform reinforcement learning policy iteration based on state coding and stability verification processing based on weighted moving average to generate the final weight distribution matrix; Obtain data from the final weight distribution matrix, perform SHA-256 hash encryption, and smart contract verification processing based on data legitimacy verification and permission verification to generate a blockchain storage record; Based on the weight matrix and the optimized evaluation model structure, perform fuzzy comprehensive evaluation based on triangular or trapezoidal membership function and visualization processing based on column chart, line chart, radar chart or heat map to generate the final evaluation report structure.

2. The method of claim 1, wherein, The process of generating the feature tensor structure further comprises: Obtain patent images, test texts and experimental data, perform image quality detection based on improved Chebyshev norm and text language consistency detection based on Shannon entropy to obtain a standardized multi-modal data set; Extract text feature vectors and image feature vectors from the preprocessed data set, perform cross-modal alignment processing based on attention mechanism to generate a joint feature matrix; Perform dimension normalization processing on the joint feature matrix based on a learnable weight vector to generate a feature tensor.

3. The method of claim 1, wherein, The process of generating the weight matrix to be optimized further comprises: Input the feature tensor into the XGBoost model pre-trained in the patent technology field to perform initial evaluation calculation and obtain an initial score set; Extract index weight parameters from the initial score set, perform credibility verification processing based on Bayesian inference to generate a weight parameter sequence; Perform outlier filtering processing on the weight parameter sequence based on box plot and local outlier factor algorithm to generate the weight matrix to be optimized.

4. The method of claim 1, wherein, The process of generating the optimized evaluation model structure further comprises: Obtain the model parameters of each federal node, perform homomorphic encryption processing to obtain encrypted gradient data; Extract effective update parameters from the encrypted gradient, perform aggregation calculation processing based on secure multi-party computation and differential privacy to generate global parameters; Perform model verification processing on the global parameters based on cross-validation and leave-one-out method to generate the optimized evaluation model structure.

5. The method of claim 1, wherein, The process of generating the final weight distribution matrix further comprises: Obtain the optimized evaluation model structure, initialize the reinforcement learning agent based on state coding to generate a policy space; Select the weight adjustment strategy from the strategy space, perform improved Q-learning iteration calculation, and generate dynamic weights; Perform stability verification processing based on weighted moving average on the dynamic weights to generate the final weight allocation matrix.

6. The method of claim 1, wherein, The process of generating a blockchain storage record also includes: Obtain the final weight allocation matrix, perform SHA-256 hash encryption processing, and generate an encrypted data packet; Extract key parameters from the encrypted data packet and perform smart contract verification processing based on data legitimacy verification and permission verification to generate block data; Perform consensus mechanism verification processing based on chain linkage on the block data to generate a storage record.

7. The method of claim 1, wherein, The process of generating the final evaluation report structure also includes: Obtain the weight matrix and the optimized evaluation model structure, perform fuzzy membership calculation processing based on triangular or trapezoidal membership functions, and generate fuzzy scores; Extract positive and negative ideal solutions from the fuzzy scores and perform TOPSIS distance measurement processing based on Euclidean distance to generate evaluation results; Perform visualization rendering processing on the evaluation results based on bar charts, line charts, radar charts, or heat maps to generate the final evaluation report structure.

8. The method of claim 1, wherein, The expression of generating the feature tensor structure also includes: Based on the improved Chebyshev norm form of quality evaluation, the image quality coefficient is defined: ; wherein, represents the cleaning quality coefficient of the i-th type of patent image; is the summation index for traversing the pixel blocks; represents the total number of pixel blocks after the i-th type of image is divided; is the resolution measured value of the j-th pixel block of the i-th type of image; is the resolution measured value of the j-th pixel block of the i-th type of image; and are the ideal resolution mean value and standard deviation of the i-th type of image, respectively; is the order of the improved Chebyshev norm. When the image enhancement algorithm is triggered; The language consistency score is calculated using the improved Shannon entropy: ; in, Let be the language entropy value of the j-th test text; j represents the category index of the text. It is the summation index used to iterate through all character categories; Number of character categories; Indicates the first In the text class, the first The frequency of occurrence of each character; It is a smoothing factor; Indicates the first Total number of characters in the text class; Cross-modal alignment is achieved by using an improved attention mechanism: ; wherein, is a text-image alignment attention matrix; and is a trainable projection matrix; is a projection dimension parameter; represents a semantic feature vector of the c-th text sample; represents a visual feature vector of the d-th image sample; is a sample index, c corresponds to the text modality, and d corresponds to the image modality; is a normalization function; denotes a transpose operation of a matrix; After the attention matrix dynamically adjusts the feature weight, a joint feature matrix is generated : ， performing improved dimension normalization processing on the joint feature matrix to construct a feature tensor: , in, Indicates the first Normalized feature tensor of a mode; Represents the joint characteristic matrix belonging to the first That part of the modality; and The joint characteristic matrix is ​​at the th Mean and standard deviation in the modal dimension; A learnable weight vector; It represents the Hadamardi (or Hadama) stack; is the numerical stability constant.

9. The method of claim 1, wherein, The process of generating the final weight allocation matrix also includes: Construct the state space of the reinforcement learning agent: ， wherein, is the state vector at time t; denotes a state encoding function; is the environment parameter vector; is the weight parameter of the optimization model; denotes a vector concatenation operation; State vector initialization strategy exploration space : ， wherein, represents the value of the mth weight adjustment action; represents the mth alternative action; Further, the improved Q-learning algorithm is used for policy iteration: ， wherein, is a dynamic learning rate; is a discount factor; is an immediate reward; represents the expected value of performing action in state ; represents the highest Q-value estimate among all possible actions in the next state ; is an arbitrary action in the action space ; represents the old action value function before the update; Generating dynamic weight parameters after 20 rounds of iteration ; To dynamic weight parameters Performing improved weighted moving average processing: ， wherein, is a most weighted assignment matrix; is a smoothing coefficient; is a weight matrix at time t.

10. An intelligent assessment system of multi-modal and reinforcement learning, applied to the method of any one of claims 1-9, characterized in that, Including: The environment calibration module is used to obtain the evaluation environment parameters and the optimized model and complete the system initialization calibration; The multi-modal acquisition module is used to acquire patent images, test texts, and experimental data and output standardized multi-modal data sets within the limited framework; The feature processing module is used to extract text feature vectors and image feature vectors and generate a joint feature matrix; The reinforcement arbitration module is used to perform policy exploration space iteration calculation and output dynamic weight parameters; The decision execution module is used to receive dynamic weight parameters and generate multi-modal evaluation results; The parameter update module is used to record the weight allocation timestamp and update the policy exploration space parameters.

Images