Large model interaction method and system based on multi-model collaborative dialogue

Abstract

The present invention provides a large-model interaction method and system based on multi-model collaborative dialogue, which relates to the field of artificial intelligence technology, including: dividing multimodal data into pixel block matrices and extracting feature vectors, constructing a hexagonal honeycomb coding matrix to store data information, and using a multi-head attention mechanism to generate an attention score matrix; constructing a model communication network based on transmission data units, and calculating professional scores through graph convolution; collecting performance data to generate an evaluation graph, combining advantage learning to output the optimal node collaboration strategy, thereby improving the efficiency and accuracy of multi-model collaborative dialogue and enhancing the robustness and scalability of the system.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a large model interaction method and system based on multi-model collaborative dialogue. Background Art

[0002] With the rapid development of artificial intelligence technology, large model interaction methods are increasingly used in various fields. Traditional large model interaction methods mainly rely on a single model for processing, which is difficult to cope with complex multimodal data and highly dynamic task environments. In recent years, methods based on multi-model collaboration have gradually attracted attention. By integrating the advantages of multiple professional models, the overall performance and adaptability of the system have been improved;

[0003] However, the existing multi-model collaborative dialogue methods still have the problem of insufficient processing capabilities for multi-modal data, difficulty in effectively extracting and integrating feature information of different modalities, relatively simple collaboration mechanisms between models, lack of flexible task allocation and resource scheduling strategies, and difficulty in adaptively adjusting according to real-time task requirements and system status, resulting in insufficient robustness and scalability of the system.

[0004] Therefore, a solution is urgently needed to solve the problems existing in the prior art. Summary of the invention

[0005] The embodiments of the present invention provide a large model interaction method and system based on multi-model collaborative dialogue, which can at least solve some of the problems existing in the prior art.

[0006] A first aspect of an embodiment of the present invention provides a large model interaction method based on multi-model collaborative dialogue, including:

[0007] Receive multimodal data and divide the multimodal data into a matrix of pixel blocks of the same size, generate a perceptual feature vector by calculating the brightness mean and color distribution of each pixel block, extract the high-frequency feature component and the low-frequency feature component corresponding to the perceptual feature vector by wavelet transform and construct a quantization threshold area, map the high-frequency feature component and the low-frequency feature component to a hexagonal honeycomb coding matrix according to the quantization threshold area, fill the main data information in the core layer, write the data time series relationship into the association layer, generate a data check bitmap in combination with the density estimation algorithm and store it in the verification layer, perform a multi-head attention mechanism on the coding matrix to generate an attention score matrix, superimpose the self-attention residual unit to perform feature dimensionality reduction, obtain a compressed vector and divide it into data blocks of fixed length, construct a redundant check package in combination with the data check bitmap, and generate a transmission data unit;

[0008] A model communication network is constructed based on the transmission data unit and the attention score matrix, and a plurality of basic models are deployed as model nodes. A plurality of model nodes are randomly selected from the model nodes as consensus nodes. The perception feature vector, the high-frequency feature component, and the low-frequency feature component are added to the model node. The professional scores between nodes are calculated by graph convolution and a weighted model collaboration graph is constructed. A bidirectional information channel is established on the model collaboration graph. The transmission data unit is encoded into a standard representation format in combination with the attention score matrix. The task affinity is calculated in combination with the professional score and the feature association of the encoding matrix. An adaptive routing table is generated in combination with the computational load of the model node and the verification result of the consensus node. The transmission data unit is transmitted between model nodes based on the adaptive routing table, and data integrity verification is performed through a data check bitmap.

[0009] The performance data of the model node when processing the transmission data unit is collected, and the response delay, throughput and resource utilization are modeled online in combination with the attention score matrix to generate a performance evaluation graph, the performance evaluation graph and the model collaboration graph are integrated to obtain a comprehensive evaluation network and converted into a state space representation, the state space representation is used as the input of the strategy network based on advantage learning, the candidate node collaboration strategy is output, the candidate node collaboration strategy is converted into an evaluation vector form, the evaluation score is calculated on each evaluation dimension in combination with the preset performance indicators, a multidimensional strategy competition space is constructed based on the evaluation score, the optimal solution set is solved by a fast non-dominated sorting algorithm, and the key collaboration unit is solved by a principal component analysis algorithm, correlation grouping is performed based on the key collaboration unit and the spectral clustering algorithm, a strategy recombination template is obtained, and a cross recombination operation is performed to obtain the optimal node collaboration strategy.

[0010] In an optional embodiment,

[0011] Receiving multimodal data and dividing the multimodal data into pixel block matrices of the same size, generating a perceptual feature vector by calculating the brightness mean and color distribution of each pixel block, extracting high-frequency feature components and low-frequency feature components corresponding to the perceptual feature vector by wavelet transform, and constructing a quantization threshold region includes:

[0012] Receive a multimodal data stream, extract a key frame sequence of video data in the multimodal data stream and parse it into an RGB three-channel matrix, convert the audio data in the multimodal data stream into a time-frequency spectrum diagram through a short-time Fourier transform, encode a character sequence of text data in the multimodal data stream into a word embedding vector, and use a unified data preprocessing module to map the RGB three-channel matrix, the time-frequency spectrum diagram and the word embedding vector into a pixel-level representation;

[0013] Calculating the local entropy value distribution of the pixel-level representation, identifying the boundary of the data density change area based on the local entropy value distribution and marking the high entropy value area and the low entropy value area, using a first block size in the high entropy value area, and using a second block size larger than the first block size in the low entropy value area, determining the values ​​of the first block size and the second block size by iterative bisection, and dividing the pixel-level representation into a pixel block matrix;

[0014] An adaptive threshold segmentation algorithm is used to divide the pixel value of each pixel block in the pixel block matrix into multiple brightness levels, the pixel block is scanned through a sliding window and a brightness distribution histogram of the pixels in the window is accumulated and counted, the first-order moment, the second-order moment and the third-order moment of the brightness distribution histogram are calculated to obtain brightness features, the K-means clustering algorithm is applied in the RGB color space to extract the main color tone and calculate the inter-class distance and the intra-class distance to obtain the color distribution features, and the brightness features and the color distribution features are combined to form a perceptual feature vector;

[0015] Constructing a wavelet filter group including a low-pass decomposition filter and a high-pass decomposition filter, performing four-layer decomposition on the perceptual feature vector to obtain an approximate component and a detail component of each layer, passing the approximate component through the low-pass decomposition filter to obtain an input signal of the next layer, retaining high-frequency information in three directions for the detail component, and denoising the wavelet coefficients by a soft threshold method;

[0016] The probability density distribution of the wavelet coefficients is calculated and local extreme points are identified. The coefficient interval is divided using a splitting strategy based on the variance ratio within the interval and a merging strategy based on the distance between intervals to obtain a non-uniform quantization threshold area. The high-frequency feature components and the low-frequency feature components are mapped to the non-uniform quantization threshold area.

[0017] In an optional embodiment,

[0018] According to the quantization threshold area, the high-frequency feature component and the low-frequency feature component are mapped to a hexagonal honeycomb coding matrix, the main data information is filled in the core layer, the data time series relationship is written into the association layer, and the data check bitmap is generated in combination with the density estimation algorithm and stored in the verification layer. The multi-head attention mechanism is performed on the coding matrix to generate an attention score matrix, and the self-attention residual unit is superimposed to perform feature dimensionality reduction to obtain a compressed vector and divide it into data blocks of fixed length. A redundant check package is constructed in combination with the data check bitmap, and the transmission data unit is generated, including:

[0019] Constructing a honeycomb coding matrix composed of concentrically surrounded multi-layer hexagonal units, filling the high-frequency characteristic components and the low-frequency characteristic components in the core layer of the coding matrix by a spirally unfolded serpentine scanning method, recording the temporal dependency of data blocks in the association layer of the coding matrix, and storing a data density distribution check bitmap calculated based on a Gaussian kernel function in the verification layer of the coding matrix;

[0020] Project the encoding matrix to the query space, key space and value space, calculate the inner product of the feature vectors in multiple attention heads to obtain attention scores, normalize the attention scores and perform weighted combination with the value vectors, and concatenate the output features of multiple attention heads through linear transformation to form an attention score matrix;

[0021] Constructing a self-attention residual unit including a main branch and a bypass branch, performing a self-attention operation on the main branch, keeping the original features unchanged on the bypass branch, and fusing the outputs of the main branch and the bypass branch by addition and reducing the dimension by a multi-layer perceptron to obtain a compressed vector;

[0022] Divide the compressed vector into a plurality of data segments of fixed length, count the distribution density of valid bits in the data segments, calculate the horizontal bit pattern characteristics and the vertical bit pattern characteristics of the data segments, generate a feature identification code according to the bit pattern characteristics and the distribution density, the feature identification code includes bit pattern type information and density level information, use the position-weighted Hamming distance to measure the similarity between the data segments, dynamically adjust the similarity threshold based on data reliability requirements, and form the data segments with similarity higher than the similarity threshold into data clusters;

[0023] Allocating a check codeword length to the data cluster according to the density of the data segments in the data cluster, mapping the binary data sequence in the data cluster to elements in a finite field according to byte grouping, performing a polynomial division operation on the elements in the finite field based on a generating polynomial to obtain a redundant check code, rearranging the data bytes in the data cluster so that continuous data bytes are mapped to different data segments, determining an interleaving depth based on a burst error statistical characteristic, and performing Reed-Solomon coding on the rearranged data to generate an interleaving check code;

[0024] Perform two-level cascade coding, wherein the first level coding generates an internal check code for the data segment for correcting random errors, and the second level coding generates an external check code for the data cluster for processing burst errors, and records the coding scheme type of the data segment, the redundant check code length, the interleaving depth and the error correction capability to generate a check parameter management table, package the data segment, the redundant check code, the interleaving check code, the internal check code, the external check code and the check parameter management table, and add a sequence number, a timestamp and a data length to generate a transmission data unit.

[0025] In an optional embodiment,

[0026] Based on the transmission data unit and the attention score matrix, a model communication network is constructed and multiple basic models are deployed as model nodes. Multiple model nodes are randomly selected from the model nodes as consensus nodes. The perception feature vector, the high-frequency feature component and the low-frequency feature component are added to the model node. The professional scores between nodes are calculated by graph convolution and a weighted model collaboration graph is constructed. A two-way information channel is established on the model collaboration graph. The transmission data unit is encoded into a standard representation format in combination with the attention score matrix. The task affinity is calculated in combination with the professional score and the feature association of the encoding matrix. An adaptive routing table is generated in combination with the computing load of the model node and the verification result of the consensus node. The transmission data unit is transmitted between model nodes based on the adaptive routing table. Data integrity verification is performed through a data check bitmap, including:

[0027] A node structure of a model communication network is constructed based on the transmission data unit and the attention score matrix, a plurality of pre-selected basic models are deployed as model nodes in the communication network, a reliability score is calculated in the model node based on the node credit value, computing resource occupancy rate and network response time, a model node with a reliability score higher than a pre-set reliability threshold is used as a candidate consensus node, and a plurality of candidate consensus nodes are randomly selected as consensus nodes;

[0028] Writing feature data into the model node, the feature data including a perceptual feature vector, a high-frequency feature component and a low-frequency feature component, storing the feature data in a key-value pair data structure, wherein the key is a feature identifier, and the value includes the feature data and corresponding timestamp information, and regularly clearing outdated data and updating the latest feature data according to the timestamp information;

[0029] The professionalism scores of the model nodes are calculated by graph convolution, a node adjacency matrix is ​​constructed to represent the node connection relationship, a performance indicator matrix is ​​generated based on the historical task processing accuracy, computing latency and resource utilization efficiency of the model nodes, the professionalism scores are updated through multiple rounds of graph convolution iterations, a weighted model collaboration graph is constructed based on the professionalism scores, and the collaboration weights are determined according to the differences in professionalism scores between the model nodes, wherein the collaboration weights represent the closeness of collaboration between the model nodes;

[0030] A bidirectional data transmission channel is established on the model collaboration diagram, the transmission bandwidth of the data transmission channel is dynamically adjusted, the transmission data unit is encoded in combination with the attention score matrix, the data encoding accuracy is determined according to the importance score in the attention score matrix, a high-precision encoding method is used for the transmission data unit whose importance score is higher than the average importance score, and a standard format data including a data header, a data payload and verification information is generated;

[0031] Based on the standard format data, a feature association matrix is ​​constructed through structural analysis and feature analysis, and the feature association of the professional score and the coding matrix is ​​combined to set a weight coefficient and perform weighted fusion to obtain an initial affinity, and the task affinity is obtained by performing data correction in combination with real-time feedback data;

[0032] An adaptive routing mechanism is constructed based on the task affinity, the computing load status of the model nodes is monitored, a network topology view is constructed in combination with the verification results of the consensus nodes, and a data transmission routing table is generated based on the network topology view, wherein the data transmission routing table is updated in real time with the status of the model nodes, the transmission data unit is transferred between the model nodes by data slicing, the network status of the transmission path is monitored in real time, and when a network performance degradation is detected, it is switched to an alternative transmission path, and the data integrity is verified by a data check bitmap after the data slicing arrives at the target node.

[0033] In an optional embodiment,

[0034] Based on the standard format data, a feature association matrix is ​​constructed through structural analysis and feature analysis, and the feature association of the professional score and the coding matrix is ​​combined to set a weight coefficient and perform weighted fusion to obtain the initial affinity. The task affinity is obtained by performing data correction in combination with real-time feedback data, including:

[0035] Receive and parse standard format information, the standard format information including data header information, data payload information and metadata information, extract data type identifier, data length and timestamp from the data header information, and extract data feature description and processing requirements from the metadata information;

[0036] Performing structural analysis on the data load information, extracting time series features in the data load information by using a sliding window method, extracting numerical features by using a numerical segmentation method, extracting categorical features by using a one-hot encoding method, and constructing the time series features, the numerical features, and the categorical features into a feature vector;

[0037] Calculating the distribution characteristics of the feature vectors, calculating the mean, variance and data skewness of the feature vectors, calculating the correlation coefficients between the feature vectors based on the distribution characteristics, and generating a feature correlation matrix;

[0038] Obtaining a professionalism score of a model node, wherein the professionalism score is calculated based on the historical task processing accuracy, processing delay, and resource consumption of the model node, extracting feature information from a coding matrix, matching and analyzing the feature information with the feature association matrix, calculating the degree of overlap between the feature information and the feature association matrix, and generating a feature association degree;

[0039] Normalizing the professionalism score and the feature correlation, setting weight coefficients in an adaptive manner, and obtaining initial affinity through weighted fusion;

[0040] A real-time feedback mechanism is established to collect task processing status information of the model node, wherein the task processing status information includes real-time accuracy, processing delay and resource occupancy rate. When the difference between the task processing status information and the expected value exceeds a preset threshold, the weight coefficient is adjusted based on the difference, and the initial affinity is corrected to obtain the task affinity.

[0041] In an optional embodiment,

[0042] The performance data of the model node when processing the transmission data unit is collected, and the response delay, throughput and resource utilization are modeled online in combination with the attention score matrix to generate a performance evaluation map, the performance evaluation map and the model collaboration map are integrated to obtain a comprehensive evaluation network and converted into a state space representation, the state space representation is used as the input of the strategy network based on advantage learning, the candidate node collaboration strategy is output, the candidate node collaboration strategy is converted into an evaluation vector form, and the evaluation score is calculated in each evaluation dimension in combination with the pre-set performance indicators, and a multi-dimensional strategy competition space is constructed based on the evaluation score, the optimal solution set is solved by a fast non-dominated sorting algorithm, and the key collaboration unit is solved by a principal component analysis algorithm, and correlation grouping is performed based on the key collaboration unit and the spectral clustering algorithm, and a strategy reorganization template is obtained and a cross-reorganization operation is performed, and the optimal node collaboration strategy is obtained, including:

[0043] Deploy distributed performance probes on model nodes to collect performance data, record request arrival timestamps and response completion timestamps, use a sliding window method to calculate the average delay, maximum delay and delay variance of requests to obtain delay feature data, count the number of completed tasks according to a preset sampling period and calculate the moving average to obtain throughput feature data, sample at preset time intervals to obtain CPU usage, memory occupancy and network traffic, analyze resource usage patterns through cumulative distribution functions to obtain resource utilization feature data, detect and eliminate abnormal data points in the delay feature data, the throughput feature data and the resource utilization feature data based on the standard deviation threshold, and use a linear interpolation method to complete the missing data to obtain a complete performance data set;

[0044] Extracting long-term performance features and short-term performance features from the complete performance data set at different time scales through an attention mechanism, calculating performance indicator weights based on the long-term performance features and the short-term performance features, constructing a time series matrix for the complete performance data set according to time units, performing extreme value normalization processing on the time series matrix combined with the performance indicator weights to obtain a standardized performance matrix, and performing trend prediction on the standardized performance matrix based on a smoothing coefficient to obtain a performance evaluation map;

[0045] Establishing a node collaboration relationship diagram, calculating the historical data interaction frequency and the historical data interaction amount to obtain the node collaboration strength and setting the node collaboration strength as the connection weight, setting the performance evaluation map as the node feature attribute, setting the node collaboration strength as the connection feature attribute to obtain a hybrid feature network, and extracting the structural features of the hybrid feature network through convolution operation to obtain a feature map;

[0046] Generate an initial candidate collaboration strategy based on the node characteristic attributes and the connection characteristic attributes, evaluate the initial candidate collaboration strategy according to the long-term performance characteristics and the short-term performance characteristics, and obtain a strategy fitness distribution through normalization, and screen the initial candidate collaboration strategy according to the strategy fitness distribution to obtain a candidate node collaboration strategy, wherein the candidate node collaboration strategy includes a node grouping scheme, a task allocation ratio, and resource quota information;

[0047] The delay characteristic data is used to calculate the ratio of the actual task completion time to the theoretical completion time of the candidate node cooperation strategy to obtain a computing efficiency score, the resource utilization characteristic data is used to calculate the node load distribution variance of the candidate cooperation strategy to obtain a load balancing score, and the throughput characteristic data is combined to calculate the resource utilization score, communication overhead score, reliability score, scalability score, response time score and throughput score of the candidate cooperation strategy, and a piecewise function is used to normalize the computing efficiency score, the load balancing score and other scores to obtain an evaluation score;

[0048] Performing uniform grid division on each dimension of the evaluation score to construct a multidimensional strategy competition space, mapping the evaluation score to the multidimensional strategy competition space to obtain strategy position coordinates, sorting the strategy position coordinates by a fast non-dominated sorting algorithm to obtain an optimal solution set, performing a principal component analysis algorithm on the optimal solution set, extracting the main feature directions and calculating the feature contribution rate, and selecting the feature directions whose cumulative contribution rate exceeds a preset threshold as key collaboration units;

[0049] A strategy affinity matrix is ​​constructed based on the key collaboration unit, and the similarity between strategies is calculated to obtain a strategy similarity matrix. A spectral clustering algorithm is used to perform feature decomposition on the strategy similarity matrix and perform correlation grouping according to the size of the eigenvalue to obtain a strategy recombination template. A cross-recombination operation is performed on the strategy recombination template to generate a new candidate strategy, and an evaluation score of the new candidate strategy is calculated, and the strategy with the highest evaluation score is selected as the optimal node collaboration strategy.

[0050] In an optional embodiment,

[0051] Perform uniform grid division on each dimension of the evaluation score to construct a multidimensional strategy competition space, map the evaluation score to the multidimensional strategy competition space to obtain strategy position coordinates, sort the strategy position coordinates by a fast non-dominated sorting algorithm to obtain an optimal solution set, perform a principal component analysis algorithm on the optimal solution set, extract the main feature directions and calculate the feature contribution rate, and select the feature directions whose cumulative contribution rate exceeds a preset threshold as key collaboration units, including:

[0052] Performing maximum and minimum value normalization processing on the scoring data of the evaluation score to obtain normalized scoring data, determining the grid division strength according to the system performance requirements and performing equidistant division on each dimension of the normalized scoring data to obtain grid units, assigning a dimension index value as a unique identifier to each grid unit and constructing the multidimensional strategy competition space;

[0053] Determine the grid unit position of each candidate strategy according to the normalized scoring data of each candidate strategy in different dimensions, and map the evaluation score to the multidimensional strategy competition space based on the dimension index value of the grid unit to obtain the strategy position coordinates;

[0054] Construct a strategy dominance relationship matrix, determine the dominance relationship between candidate strategies by comparing their scores on each evaluation dimension, calculate the dominated count and dominance set of each candidate strategy based on the strategy dominance relationship matrix, classify candidate strategies with a dominated count of zero into a non-dominated strategy hierarchy, update the dominated counts of strategies dominated by candidate strategies in the non-dominated strategy hierarchy, repeat the operation of classifying into the non-dominated strategy hierarchy until all strategies are stratified, and determine the highest-level strategy in the non-dominated strategy hierarchy as the optimal solution set by a fast non-dominated sorting algorithm;

[0055] Construct a strategy scoring matrix, where the number of rows of the strategy scoring matrix is ​​the number of strategies in the optimal solution set, and the number of columns is the number of evaluation dimensions, calculate the mean vector and standard deviation vector of each evaluation dimension, perform centralization on the strategy scoring matrix, calculate the correlation coefficients between the evaluation dimensions to construct a correlation coefficient matrix, perform a principal component analysis algorithm on the correlation coefficient matrix, perform eigenvalue decomposition on the correlation coefficient matrix to obtain an eigenvalue sequence and an eigenvector sequence, extract the main feature direction, arrange the eigenvalue sequence in descending order, calculate the variance contribution rate and cumulative variance contribution rate of each eigenvalue, and determine the number of eigenvectors to be retained based on a preset cumulative variance contribution rate threshold;

[0056] The retained feature vector is determined as a key collaboration unit, and the contribution of the original evaluation dimension to the feature direction is determined according to the size of each component in the key collaboration unit.

[0057] A second aspect of an embodiment of the present invention provides a large model interaction system based on multi-model collaborative dialogue, including:

[0058] The first unit is used to receive multimodal data and divide the multimodal data into a matrix of pixel blocks of the same size, generate a perceptual feature vector by calculating the brightness mean and color distribution of each pixel block, extract the high-frequency feature component and the low-frequency feature component corresponding to the perceptual feature vector by wavelet transform and construct a quantization threshold area, map the high-frequency feature component and the low-frequency feature component to a hexagonal honeycomb coding matrix according to the quantization threshold area, fill the main data information in the core layer, write the data time series relationship into the association layer, generate a data check bitmap in combination with the density estimation algorithm and store it in the verification layer, perform a multi-head attention mechanism on the coding matrix to generate an attention score matrix, superimpose the self-attention residual unit to perform feature dimensionality reduction, obtain a compressed vector and divide it into data blocks of fixed length, construct a redundant check packet in combination with the data check bitmap, and generate a transmission data unit;

[0059] The second unit is used to build a model communication network based on the transmission data unit and the attention score matrix and deploy multiple basic models as model nodes, randomly select multiple model nodes from the model nodes as consensus nodes, add the perception feature vector, the high-frequency feature component and the low-frequency feature component to the model node, calculate the professional score between nodes through graph convolution and build a weighted model collaboration graph, establish a two-way information channel on the model collaboration graph, encode the transmission data unit into a standard representation format in combination with the attention score matrix, calculate the task affinity in combination with the professional score and the feature correlation of the encoding matrix, generate an adaptive routing table in combination with the computing load of the model node and the verification result of the consensus node, transfer the transmission data unit between model nodes based on the adaptive routing table, and perform data integrity verification through a data check bitmap;

[0060] The third unit is used to collect performance data when the model node processes the transmission data unit, and to perform online modeling of the response delay, throughput and resource utilization in combination with the attention score matrix to generate a performance evaluation graph, to fuse the performance evaluation graph and the model collaboration graph to obtain a comprehensive evaluation network and convert it into a state space representation, to use the state space representation as the input of the strategy network based on advantage learning, to output candidate node collaboration strategies, to convert the candidate node collaboration strategies into an evaluation vector form, to calculate the evaluation scores on each evaluation dimension in combination with pre-set performance indicators, to construct a multi-dimensional strategy competition space based on the evaluation scores, to solve the optimal solution set through a fast non-dominated sorting algorithm and to solve the key collaboration units through a principal component analysis algorithm, to perform correlation grouping based on the key collaboration units and a spectral clustering algorithm, to obtain a strategy recombination template and to perform a cross-recombination operation to obtain the optimal node collaboration strategy.

[0061] According to a third aspect of the embodiments of the present invention,

[0062] An electronic device is provided, comprising:

[0063] processor;

[0064] a memory for storing processor-executable instructions;

[0065] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0066] A fourth aspect of the embodiments of the present invention is:

[0067] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0068] In the present invention, multimodal data is divided into pixel block matrices and perceptual feature vectors are extracted, high and low frequency feature components are obtained by combining wavelet transform, a hexagonal honeycomb coding matrix is ​​used to store data, and feature dimension reduction is performed through a multi-head attention mechanism and a self-attention residual unit to improve the accuracy and reliability of data processing. A model communication network is constructed based on a transmission data unit and an attention score matrix, node professionalism scores are calculated through graph convolution, a two-way information channel is established, and efficient collaboration between model nodes is achieved by combining task affinity and an adaptive routing table, thereby improving the overall collaborative efficiency of the system and the stability of data transmission. Performance data is collected and online modeling is performed, and a comprehensive evaluation network is constructed by integrating the performance evaluation graph and the model collaboration graph. Advantage learning and a fast non-dominated sorting algorithm are used to select the optimal collaboration strategy. The principal component analysis and a spectral clustering algorithm are combined for strategy reorganization, thereby achieving dynamic optimization of model collaboration and efficient utilization of resources, and significantly improving the overall performance and adaptability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 A schematic diagram of a flow chart of a large model interaction method based on multi-model collaborative dialogue according to an embodiment of the present invention;

[0070] Figure 2 It is a structural diagram of a large model interaction system based on multi-model collaborative dialogue in an embodiment of the present invention. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0073] Figure 1 FIG. 1 is a flow chart of a large model interaction method based on multi-model collaborative dialogue according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0074] S1. Receive multimodal data and divide the multimodal data into a matrix of pixel blocks of the same size, generate a perceptual feature vector by calculating the brightness mean and color distribution of each pixel block, extract the high-frequency feature component and the low-frequency feature component corresponding to the perceptual feature vector by wavelet transform and construct a quantization threshold region, map the high-frequency feature component and the low-frequency feature component to a hexagonal honeycomb coding matrix according to the quantization threshold region, fill the main data information in the core layer, write the data time series relationship into the association layer, generate a data check bitmap in combination with the density estimation algorithm and store it in the verification layer, perform a multi-head attention mechanism on the coding matrix to generate an attention score matrix, superimpose the self-attention residual unit to perform feature dimensionality reduction, obtain a compressed vector and divide it into data blocks of fixed length, construct a redundant check package in combination with the data check bitmap, and generate a transmission data unit;

[0075] The quantization threshold area refers to the range in which data or signals are divided according to a preset threshold during discretization processing, which is used to represent different levels or states. The encoding matrix is ​​a matrix that maps information into a specific form, which is used for data compression, error correction or information representation. The density estimation algorithm is a statistical method for estimating the density of data distribution in space, which is used to analyze data aggregation or distribution patterns. The data check bitmap is an information representation method that records data check status in the form of a bitmap, which is used to quickly mark and locate data integrity issues. The redundant check package is a mechanism for verifying and restoring data integrity by adding redundant information, which is used to improve the reliability of communication or storage systems.

[0076] In an optional embodiment,

[0077] Receiving multimodal data and dividing the multimodal data into pixel block matrices of the same size, generating a perceptual feature vector by calculating the brightness mean and color distribution of each pixel block, extracting high-frequency feature components and low-frequency feature components corresponding to the perceptual feature vector by wavelet transform, and constructing a quantization threshold region includes:

[0078] Receive a multimodal data stream, extract a key frame sequence of video data in the multimodal data stream and parse it into an RGB three-channel matrix, convert the audio data in the multimodal data stream into a time-frequency spectrum diagram through a short-time Fourier transform, encode a character sequence of text data in the multimodal data stream into a word embedding vector, and use a unified data preprocessing module to map the RGB three-channel matrix, the time-frequency spectrum diagram and the word embedding vector into a pixel-level representation;

[0079] Calculating the local entropy value distribution of the pixel-level representation, identifying the boundary of the data density change area based on the local entropy value distribution and marking the high entropy value area and the low entropy value area, using a first block size in the high entropy value area, and using a second block size larger than the first block size in the low entropy value area, determining the values ​​of the first block size and the second block size by iterative bisection, and dividing the pixel-level representation into a pixel block matrix;

[0080] An adaptive threshold segmentation algorithm is used to divide the pixel value of each pixel block in the pixel block matrix into multiple brightness levels, the pixel block is scanned through a sliding window and a brightness distribution histogram of the pixels in the window is accumulated and counted, the first-order moment, the second-order moment and the third-order moment of the brightness distribution histogram are calculated to obtain brightness features, the K-means clustering algorithm is applied in the RGB color space to extract the main color tone and calculate the inter-class distance and the intra-class distance to obtain the color distribution features, and the brightness features and the color distribution features are combined to form a perceptual feature vector;

[0081] Constructing a wavelet filter group including a low-pass decomposition filter and a high-pass decomposition filter, performing four-layer decomposition on the perceptual feature vector to obtain an approximate component and a detail component of each layer, passing the approximate component through the low-pass decomposition filter to obtain an input signal of the next layer, retaining high-frequency information in three directions for the detail component, and denoising the wavelet coefficients by a soft threshold method;

[0082] The probability density distribution of the wavelet coefficients is calculated and local extreme points are identified. The coefficient interval is divided using a splitting strategy based on the variance ratio within the interval and a merging strategy based on the distance between intervals to obtain a non-uniform quantization threshold area. The high-frequency feature components and the low-frequency feature components are mapped to the non-uniform quantization threshold area.

[0083] The time-frequency spectrum is a chart that visualizes the distribution of a signal in time and frequency, and is used to analyze signal characteristics or changing trends. The first-order moment refers to the mean or expected value of data or distribution, reflecting the statistical characteristics of its center position. The K-means clustering algorithm is an unsupervised learning algorithm that divides data into K clusters based on distance, and is used for data grouping and pattern recognition. The local extreme point refers to the maximum or minimum value of a local area in a function or data sequence, and is used for feature extraction or pattern recognition. The splitting strategy based on the variance ratio within the interval is a division method that splits the interval according to the principle of maximizing the variance ratio within the data interval, and is used to improve classification or clustering effects. The merging strategy based on the distance between intervals is a clustering method that merges adjacent intervals according to the distance measurement between different intervals to ensure the rationality of the overall structure.

[0084] Receive multimodal data streams and extract video, audio, and text data from them. For video data, extract key frame sequences and parse them into RGB three-channel matrices. For example, a scene switch detection algorithm can be used to extract key frames, selecting 1-2 representative images per second. For audio data, convert it into a time-frequency spectrum graph through short-time Fourier transform. Specifically, a 20 millisecond time window and a 10 millisecond step size can be selected, the audio signal can be framed and a Hamming window can be applied, and then the spectrum of each frame can be calculated. For text data, the character sequence is encoded into a word embedding vector. A pre-trained word embedding model such as Word2Vec or GloVe can be used to map each word to a 300-dimensional vector space.

[0085] A unified data preprocessing module is used to map the RGB three-channel matrix, time-frequency spectrum graph, and word embedding vector into pixel-level representation. This can be achieved through a convolutional neural network to convert data of different modalities into feature maps of the same size, for example, mapping all data to a 256x256 feature map.

[0086] Calculate the local entropy distribution of pixel-level representation. You can use a 3x3 sliding window to calculate the probability distribution of pixel values ​​in each window, and calculate the local entropy value according to the information entropy formula. Based on the local entropy distribution, identify the boundaries of the data density change area and mark the high entropy area and low entropy area. You can set the entropy threshold, for example, mark the area with a value higher than 1.5 times the average entropy as a high entropy area, and mark the area with a value lower than 0.5 times the average entropy as a low entropy area.

[0087] A first block size is used in a high entropy region, and a second block size larger than the first block size is used in a low entropy region. The values ​​of the first block size and the second block size are determined by iterative bisection. For example, starting from the initial sizes of 8x8 and 16x16, the block size can be adjusted according to the change in the regional entropy value until the entropy value change rate is less than a preset threshold (such as 0.1%). Finally, the pixel-level representation is divided into a pixel block matrix.

[0088] For each pixel block, an adaptive threshold segmentation algorithm is used to divide the pixel values ​​into multiple brightness levels. The Otsu algorithm is used to automatically determine the optimal threshold and divide the pixel values ​​into 3-5 brightness levels. The pixel block is scanned through an 8x8 sliding window, and the brightness distribution histogram of the pixels in the window is accumulated and counted. The first-order moment (mean), second-order moment (variance), and third-order moment (skewness) of the brightness distribution histogram are calculated to obtain the brightness feature.

[0089] The K-means clustering algorithm is applied in the RGB color space to extract the main color. For example, K=5 is set to extract 5 main color clusters. The inter-cluster distance (the average value of the Euclidean distance between different cluster centers) and the intra-cluster distance (the average distance from each pixel in each cluster to the cluster center) are calculated to obtain the color distribution features. The brightness features and color distribution features are combined to form a perceptual feature vector.

[0090] Construct a wavelet filter bank containing low-pass decomposition filters and high-pass decomposition filters. You can choose Daubechies wavelet (such as db4) as the basis function. Perform a four-layer decomposition on the perceptual feature vector to obtain the approximate component and detail component of each layer. The approximate component is passed through a low-pass decomposition filter to obtain the input signal of the next layer, and the detail component retains the high-frequency information in the horizontal, vertical and diagonal directions.

[0091] The wavelet coefficients are denoised by a soft threshold method. You can use the general threshold method and set the threshold to the noise standard deviation multiplied by the logarithmic term. For coefficients less than the threshold, set them to zero; for coefficients greater than the threshold, subtract the threshold value.

[0092] The probability density distribution of the wavelet coefficients is calculated using the kernel density estimation method, and the local extreme points are identified as the initial interval boundaries. A splitting strategy based on the variance ratio within the interval is adopted. When the variance ratio of an interval exceeds the preset threshold (such as 1.5), the interval is split into two sub-intervals. A merging strategy based on the distance between intervals is adopted. When the center distance of adjacent intervals is less than the preset threshold (such as 0.1 times the standard deviation), these intervals are merged. In this way, a non-uniform quantization threshold area is obtained.

[0093] The high-frequency feature components and low-frequency feature components are mapped to the non-uniform quantization threshold region. For each coefficient, the region to which it belongs is found, and the original coefficient value is replaced with the representative value of the region (such as the midpoint of the region) to complete the quantization process.

[0094] In this embodiment, through a unified data preprocessing module, effective fusion and representation of different modal data such as video, audio and text are achieved. The unified pixel-level representation provides a good foundation for subsequent feature extraction and analysis, helps to make full use of complementary information in multimodal data, and improves the comprehensiveness and accuracy of data processing. An adaptive blocking strategy based on local entropy value is adopted, which can dynamically adjust the block size according to the complexity of the data. While ensuring the detail information of the high entropy value area, it can also effectively reduce the computational complexity of the low entropy value area, realize the reasonable allocation and utilization of computing resources, improve the overall processing efficiency, realize multi-scale analysis and efficient compression of perceptual features, not only can capture the characteristics of data in different frequency domains, but also retain key information through adaptive quantization strategy, achieve a good balance between data compression and information retention, and provide optimized feature representation for subsequent data storage, transmission and analysis.

[0095] In an optional embodiment,

[0096] According to the quantization threshold area, the high-frequency feature component and the low-frequency feature component are mapped to a hexagonal honeycomb coding matrix, the main data information is filled in the core layer, the data time series relationship is written into the association layer, and the data check bitmap is generated in combination with the density estimation algorithm and stored in the verification layer. The multi-head attention mechanism is performed on the coding matrix to generate an attention score matrix, and the self-attention residual unit is superimposed to perform feature dimensionality reduction to obtain a compressed vector and divide it into data blocks of fixed length. A redundant check package is constructed in combination with the data check bitmap, and the transmission data unit is generated, including:

[0097] Constructing a honeycomb coding matrix composed of concentrically surrounded multi-layer hexagonal units, filling the high-frequency characteristic components and the low-frequency characteristic components in the core layer of the coding matrix by a spirally unfolded serpentine scanning method, recording the temporal dependency of data blocks in the association layer of the coding matrix, and storing a data density distribution check bitmap calculated based on a Gaussian kernel function in the verification layer of the coding matrix;

[0098] Project the encoding matrix to the query space, key space and value space, calculate the inner product of the feature vectors in multiple attention heads to obtain attention scores, normalize the attention scores and perform weighted combination with the value vectors, and concatenate the output features of multiple attention heads through linear transformation to form an attention score matrix;

[0099] Constructing a self-attention residual unit including a main branch and a bypass branch, performing a self-attention operation on the main branch, keeping the original features unchanged on the bypass branch, and fusing the outputs of the main branch and the bypass branch by addition and reducing the dimension by a multi-layer perceptron to obtain a compressed vector;

[0100] Divide the compressed vector into a plurality of data segments of fixed length, count the distribution density of valid bits in the data segments, calculate the horizontal bit pattern characteristics and the vertical bit pattern characteristics of the data segments, generate a feature identification code according to the bit pattern characteristics and the distribution density, the feature identification code includes bit pattern type information and density level information, use the position-weighted Hamming distance to measure the similarity between the data segments, dynamically adjust the similarity threshold based on data reliability requirements, and form the data segments with similarity higher than the similarity threshold into data clusters;

[0101] Allocating a check codeword length to the data cluster according to the density of the data segments in the data cluster, mapping the binary data sequence in the data cluster to elements in a finite field according to byte grouping, performing a polynomial division operation on the elements in the finite field based on a generating polynomial to obtain a redundant check code, rearranging the data bytes in the data cluster so that continuous data bytes are mapped to different data segments, determining an interleaving depth based on a burst error statistical characteristic, and performing Reed-Solomon coding on the rearranged data to generate an interleaving check code;

[0102] Perform two-level cascade coding, wherein the first level coding generates an internal check code for the data segment for correcting random errors, and the second level coding generates an external check code for the data cluster for processing burst errors, and records the coding scheme type of the data segment, the redundant check code length, the interleaving depth and the error correction capability to generate a check parameter management table, package the data segment, the redundant check code, the interleaving check code, the internal check code, the external check code and the check parameter management table, and add a sequence number, a timestamp and a data length to generate a transmission data unit.

[0103] The serpentine scanning mode is a mode of scanning data in a serpentine path sequence, which is used for image processing or storage layout optimization. The position-weighted Hamming distance is an improved method of assigning different weights according to positions when calculating the Hamming distance, which is used to more accurately evaluate data similarity. The finite field is an algebraic structure containing a finite number of elements, which is used for mathematical operations in coding theory and cryptography. The interleaving depth refers to the number of rows or columns of the interleaving block in the data interleaving process, which is used to improve the anti-interference ability of the data. The Reed-Solomon code is an error correction coding method based on finite field operations, which is used to detect and repair multi-bit errors in data transmission. The interleaving check code is a coding method that combines interleaving technology and error correction code, which is used to further improve the data verification and error correction capabilities.

[0104] A hexagonal honeycomb coding matrix structure is constructed, which consists of multiple layers of concentrically surrounded hexagonal units, and is divided into a core layer, an association layer, and a verification layer from the inside to the outside. The core layer uses a spirally unfolded serpentine scanning method to fill in the quantized high-frequency and low-frequency feature components. Taking the 32x32 feature map as an example, it is divided into 4x4 sub-blocks, and the feature values ​​in each sub-block are mapped to the hexagonal units corresponding to the core layer according to the spiral path. The association layer records the temporal dependency between data blocks and represents the relative position information through the position coding vector. The verification layer stores the data density distribution check bitmap calculated based on the Gaussian kernel function, and uses a sliding window method to count the data distribution characteristics of the local area.

[0105] Based on the multi-head attention mechanism, the encoding matrix is ​​projected into the query space, key space and value space respectively, and each attention head calculates the attention score independently. Taking 8 attention heads as an example, the output dimension of each head is 64, and the output features of multiple heads are concatenated through linear transformation to obtain a 512-dimensional attention score matrix. The attention score is normalized by Softmax and then weighted combined with the value vector.

[0106] Construct a self-attention residual unit. The main branch performs self-attention operations, and the bypass branch keeps the original features unchanged. The outputs of the two branches are fused by addition, and feature dimension reduction is performed through a two-layer perceptron network to obtain a 256-dimensional compressed vector. The output dimension of the first layer of perceptrons is 384, and the output dimension of the second layer is 256.

[0107] The compressed vector is divided into data segments with a fixed length of 128 bytes. The proportion of 1s in each data segment is counted as the effective bit distribution density. The bit pattern features in the horizontal and vertical directions are calculated to generate an 8-bit feature identification code, where the first 4 bits represent the bit pattern type and the last 4 bits represent the density level. The position-weighted Hamming distance is used to calculate the similarity between data segments. The similarity threshold is initially set to 0.85 and can be dynamically adjusted according to data reliability requirements. Data segments with similarity higher than the threshold are classified into the same data cluster.

[0108] The length of the check codeword is assigned based on the density of the data cluster. The higher the density of the data cluster, the longer the check codeword is assigned. The data sequence is mapped to the elements in the finite field by byte grouping, and the polynomial division operation is performed to generate the redundant check code. The data bytes in the data cluster are rearranged so that the consecutive bytes are mapped to different data segments. The interleaving depth is determined to be 16 according to the statistical characteristics of the burst error, and the Reed-Solomon coding is performed to generate the interleaving check code.

[0109] Perform two-level cascade coding. The first level generates a 32-byte internal checksum for the data segment, and the second level generates a 64-byte external checksum for the data cluster. Generate a checksum parameter management table to record information such as the coding scheme type, checksum length, interleaving depth, and error correction capability. Pack the data segment, various checksums, and parameter tables, add a 32-bit sequence number, a 64-bit timestamp, and a 32-bit data length field, and generate a complete transmission data unit.

[0110] In this embodiment, a hexagonal honeycomb coding matrix structure is adopted to make full use of the space filling characteristics of the hexagonal units and improve the data storage efficiency. At the same time, the multi-layer structure realizes the effective organization of data features, temporal relationships and verification information. Combined with the multi-head attention mechanism and the self-attention residual unit, feature dimension reduction and information extraction are achieved while maintaining the original features, reducing data redundancy and improving feature expression capabilities. Through two-level cascade coding and multiple verification mechanisms, the joint correction of random errors and burst errors is realized, and the reliability of data transmission is improved. At the same time, the adaptive check code allocation strategy based on data density optimizes the storage overhead.

[0111] S2. Based on the transmission data unit and the attention score matrix, a model communication network is constructed and multiple basic models are deployed as model nodes. Multiple model nodes are randomly selected from the model nodes as consensus nodes. The perception feature vector, the high-frequency feature component and the low-frequency feature component are added to the model node. The professional scores between nodes are calculated by graph convolution and a weighted model collaboration graph is constructed. A two-way information channel is established on the model collaboration graph. The transmission data unit is encoded into a standard representation format in combination with the attention score matrix. The task affinity is calculated in combination with the professional score and the feature correlation of the encoding matrix. An adaptive routing table is generated in combination with the computational load of the model node and the verification result of the consensus node. The transmission data unit is transmitted between model nodes based on the adaptive routing table, and data integrity verification is performed through a data check bitmap.

[0112] The basic model refers to the core model used when solving problems or performing tasks. It is usually used as the construction basis or reference standard of the entire system. The consensus node is a key node in a distributed system for negotiation and reaching consensus decisions to ensure data consistency and system reliability. The graph convolution is an algorithm that performs convolution operations on graph structured data to extract features of nodes and their neighborhoods in the graph. The professionalism score is a score for evaluating an object based on knowledge or capabilities in a specific field, which is used to measure its professionalism or skill level. The weighted model collaboration graph is a graph structure that represents the strength of the collaborative relationship between models by assigning different weights to nodes and edges, which is used to optimize collaboration efficiency. The bidirectional information channel is a communication mechanism that allows data to flow in two directions, which is used to improve the flexibility and efficiency of system interaction. The task affinity refers to the degree of match between tasks and resources or capabilities, which is used to evaluate the rationality of resource allocation or task scheduling. The adaptive routing table is a routing table that is dynamically adjusted according to network or task requirements, which is used to improve the efficiency and flexibility of network transmission.

[0113] In an optional embodiment,

[0114] Based on the transmission data unit and the attention score matrix, a model communication network is constructed and multiple basic models are deployed as model nodes. Multiple model nodes are randomly selected from the model nodes as consensus nodes. The perception feature vector, the high-frequency feature component and the low-frequency feature component are added to the model node. The professional scores between nodes are calculated by graph convolution and a weighted model collaboration graph is constructed. A two-way information channel is established on the model collaboration graph. The transmission data unit is encoded into a standard representation format in combination with the attention score matrix. The task affinity is calculated in combination with the professional score and the feature association of the encoding matrix. An adaptive routing table is generated in combination with the computing load of the model node and the verification result of the consensus node. The transmission data unit is transmitted between model nodes based on the adaptive routing table. Data integrity verification is performed through a data check bitmap, including:

[0115] A node structure of a model communication network is constructed based on the transmission data unit and the attention score matrix, a plurality of pre-selected basic models are deployed as model nodes in the communication network, a reliability score is calculated in the model node based on the node credit value, computing resource occupancy rate and network response time, a model node with a reliability score higher than a pre-set reliability threshold is used as a candidate consensus node, and a plurality of candidate consensus nodes are randomly selected as consensus nodes;

[0116] Writing feature data into the model node, the feature data including a perceptual feature vector, a high-frequency feature component and a low-frequency feature component, storing the feature data in a key-value pair data structure, wherein the key is a feature identifier, and the value includes the feature data and corresponding timestamp information, and regularly clearing outdated data and updating the latest feature data according to the timestamp information;

[0117] The professionalism scores of the model nodes are calculated by graph convolution, a node adjacency matrix is ​​constructed to represent the node connection relationship, a performance indicator matrix is ​​generated based on the historical task processing accuracy, computing latency and resource utilization efficiency of the model nodes, the professionalism scores are updated through multiple rounds of graph convolution iterations, a weighted model collaboration graph is constructed based on the professionalism scores, and the collaboration weights are determined according to the differences in professionalism scores between the model nodes, wherein the collaboration weights represent the closeness of collaboration between the model nodes;

[0118] A bidirectional data transmission channel is established on the model collaboration diagram, the transmission bandwidth of the data transmission channel is dynamically adjusted, the transmission data unit is encoded in combination with the attention score matrix, the data encoding accuracy is determined according to the importance score in the attention score matrix, a high-precision encoding method is used for the transmission data unit whose importance score is higher than the average importance score, and a standard format data including a data header, a data payload and verification information is generated;

[0119] Based on the standard format data, a feature association matrix is ​​constructed through structural analysis and feature analysis, and the feature association of the professional score and the coding matrix is ​​combined to set a weight coefficient and perform weighted fusion to obtain an initial affinity, and the task affinity is obtained by performing data correction in combination with real-time feedback data;

[0120] An adaptive routing mechanism is constructed based on the task affinity, the computing load status of the model nodes is monitored, a network topology view is constructed in combination with the verification results of the consensus nodes, and a data transmission routing table is generated based on the network topology view, wherein the data transmission routing table is updated in real time with the status of the model nodes, the transmission data unit is transferred between the model nodes by data slicing, the network status of the transmission path is monitored in real time, and when a network performance degradation is detected, it is switched to an alternative transmission path, and the data integrity is verified by a data check bitmap after the data slicing arrives at the target node.

[0121] The data encoding accuracy refers to the accuracy of numerical or information representation during the data encoding process, which is used to control the error range. The network topology view refers to the visual representation of network nodes and connection relationships, which is used to analyze network structure and behavior. The data sharding method is a method of dividing large-scale data into small fragments for distributed storage or parallel processing.

[0122] The model communication network is constructed based on the transmission data unit and the attention score matrix. The model communication network adopts a distributed architecture, and each node represents a basic model. The network topology adopts a mesh network to ensure high connectivity and redundancy between nodes. Each node is equipped with independent computing resources, storage space, and network interface, and multiple pre-selected basic models are deployed as model nodes in the constructed communication network. These basic models can include different types of machine learning models, such as convolutional neural networks, recurrent neural networks, transformers, etc. Each model node has a unique identifier for identification and communication in the network.

[0123] Select consensus nodes from the model nodes. First, calculate the reliability score of each node, which is determined by three factors: node credit, computing resource utilization, and network response time. Node credit is based on historical performance evaluation, computing resource utilization is obtained by monitoring CPU, memory, and storage usage, and network response time is determined by regular ping tests. Nodes with reliability scores higher than a preset threshold (such as 0.8) are listed as candidate consensus nodes, and then a portion of them (such as 20%) are randomly selected as the final consensus nodes using the Fisher-Yates shuffle algorithm.

[0124] Write feature data to the model node. Feature data includes perceptual feature vectors, high-frequency feature components, and low-frequency feature components. Use a key-value pair data structure to store these feature data, where the key is the feature identifier and the value contains the feature data and the corresponding timestamp information. For example, a key-value pair is: {"sensor_1_high_freq":{"data":[0.5,0.3,0.8],"timestamp":1648656000}}. The system will check the timestamp regularly (such as every hour), clean up data older than 24 hours, and update the latest feature data.

[0125] The professionalism scores of the model nodes are calculated by graph convolution, and the node adjacency matrix A is constructed to represent the connection relationship between the nodes. For example, for a network of 5 nodes, A may be a 5x5 matrix, where A[i][j]=1 means that nodes i and j are directly connected, otherwise it is 0. The performance indicator matrix P is generated based on the historical task processing accuracy, computing latency, and resource utilization efficiency of the model nodes. Multiple rounds of graph convolution iterative calculations are performed to update the professionalism score. In each iteration, the new score of the node is determined by its original score and the weighted score of the neighboring nodes. The iterative formula can be expressed as: new score = original score*0.5+weighted sum of neighbor scores*0.5.

[0126] Based on the calculated professional scores, a weighted model collaboration graph is constructed. The collaboration weight between nodes is determined by the professional score difference, calculated as: weight = 1 / (1+|score difference|). This ensures that nodes with similar professional scores have higher collaboration weights.

[0127] A bidirectional data transmission channel is established on the model collaboration graph. The transmission bandwidth of the channel is dynamically adjusted according to the collaboration weight between nodes. The higher the weight, the larger the allocated bandwidth. The transmission data unit is encoded in combination with the attention score matrix. Each element in the attention score matrix represents the importance of the corresponding data unit, ranging from 0 to 1. For data units with importance scores higher than the average, high-precision encoding (such as 32-bit floating point numbers) is used, otherwise low-precision encoding (such as 16-bit floating point numbers) is used. The encoded data contains a data header (including source node ID, target node ID, data type, etc.), data payload, and verification information (such as CRC32 checksum).

[0128] Based on the encoded standard format data, the feature association matrix R is constructed through structural analysis and feature analysis. R[i][j] represents the correlation between the i-th feature and the j-th feature, with a value range of -1 to 1. The task affinity A is calculated by combining the professionalism score S and the feature correlation R of the encoding matrix. The weight coefficients w1 and w2 (w1+w2=1) are set, and the initial affinity calculation formula is: Ainitial=w1*S+w2*R. Finally, combined with real-time feedback data (such as task execution success rate F) for correction, the final task affinity is obtained: Afinal=Ainitial*(1+F) / 2.

[0129] Build an adaptive routing mechanism based on task affinity. Monitor the computing load status of model nodes, including CPU usage, memory usage, storage space and other indicators. Combine the verification results of the consensus node to build a network topology view, which contains information such as node status, connection status and network delay. Generate a data transmission routing table based on the network topology view. The routing table contains information such as the target node, next hop node, number of hops and link quality. The routing table will be updated according to the real-time status of the model node, and the update frequency can be set to once every 5 minutes.

[0130] Data transmission uses data sharding to divide large data sets into fixed-size data blocks (such as 1MB). Each data block is attached with a unique identifier and sequence number. The network status of the transmission path is monitored in real time, including bandwidth utilization, packet loss rate, and latency. When network performance degradation is detected (such as packet loss rate exceeding 5% or latency increasing by more than 50%), the system will immediately switch to the pre-calculated backup transmission path.

[0131] After the data fragment arrives at the target node, the data integrity is verified through the check bitmap. The check bitmap is a binary array, each bit corresponds to a data fragment, 1 means it has been received, and 0 means it has not been received. By comparing the check bitmap with the actual received data fragment, the receiver can accurately locate the lost data fragment and request retransmission.

[0132] In this embodiment, distributed collaborative processing is achieved by constructing a model communication network and selecting consensus nodes, which improves the reliability and fault tolerance of the system. The mechanism of dynamically selecting consensus nodes ensures the fairness and load balancing of the system and avoids single point failure problems. By establishing a two-way information channel, an adaptive routing mechanism and data integrity verification, the efficiency and reliability of data transmission are guaranteed. The dynamic bandwidth adjustment and real-time routing update mechanism can adapt to complex and changeable network environments, improve the robustness and adaptability of the system, and use graph convolution to calculate professional scoring and construct a weighted model collaboration graph to achieve intelligent task allocation and resource scheduling, fully utilizing the expertise of each model node and improving overall processing efficiency and accuracy.

[0133] In an optional embodiment,

[0134] Based on the standard format data, a feature association matrix is ​​constructed through structural analysis and feature analysis, and the feature association of the professional score and the coding matrix is ​​combined to set a weight coefficient and perform weighted fusion to obtain the initial affinity. The task affinity is obtained by performing data correction in combination with real-time feedback data, including:

[0135] Receive and parse standard format information, the standard format information including data header information, data payload information and metadata information, extract data type identifier, data length and timestamp from the data header information, and extract data feature description and processing requirements from the metadata information;

[0136] Performing structural analysis on the data load information, extracting time series features in the data load information by using a sliding window method, extracting numerical features by using a numerical segmentation method, extracting categorical features by using a one-hot encoding method, and constructing the time series features, the numerical features, and the categorical features into a feature vector;

[0137] Calculating the distribution characteristics of the feature vectors, calculating the mean, variance and data skewness of the feature vectors, calculating the correlation coefficients between the feature vectors based on the distribution characteristics, and generating a feature correlation matrix;

[0138] Obtaining a professionalism score of a model node, wherein the professionalism score is calculated based on the historical task processing accuracy, processing delay, and resource consumption of the model node, extracting feature information from a coding matrix, matching and analyzing the feature information with the feature association matrix, calculating the degree of overlap between the feature information and the feature association matrix, and generating a feature association degree;

[0139] Normalizing the professionalism score and the feature correlation, setting weight coefficients in an adaptive manner, and obtaining initial affinity through weighted fusion;

[0140] A real-time feedback mechanism is established to collect task processing status information of the model node, wherein the task processing status information includes real-time accuracy, processing delay and resource occupancy rate. When the difference between the task processing status information and the expected value exceeds a preset threshold, the weight coefficient is adjusted based on the difference, and the initial affinity is corrected to obtain the task affinity.

[0141] The numerical feature refers to a data feature expressed in numerical form, which is used for quantitative input in model analysis and training. The feature association matrix is ​​a matrix that represents the correlation or similarity between features, which is used to analyze the relationship between features or select key features.

[0142] Parse and process the received standard format information. The standard format information consists of three main parts: data header information, data payload information, and metadata information. Extract the data type identifier (such as IMG for image data, TXT for text data, etc.), data length (in bytes), and timestamp (accurate to milliseconds) from the data header information. Extract data feature descriptions (such as data dimensions, sampling rate, etc.) and processing requirements (such as real-time requirements, accuracy requirements, etc.) from the metadata information.

[0143] Perform in-depth structural analysis of data load information. Use sliding window method to extract time series features. The window size can be set to 100 data points and the step size is 10 data points to extract trend change features within the window. For numerical features, divide the numerical range into multiple intervals, such as dividing the numerical range of 0-100 into 10 intervals, and count the data distribution of each interval. For categorical features, use one-hot encoding to convert, such as converting color features "red, green, blue" into the encoding form of "100, 010, 001". Combine the extracted features to construct feature vectors.

[0144] Analyze the statistical distribution characteristics of the feature vector. Calculate the mean (e.g., the mean value of a sensor data is 25.6), variance (reflects the degree of data fluctuation, such as the variance value is 3.2), and data skewness (reflects the degree of skewness of data distribution, such as the skewness value is 0.5) of the feature vector. Based on these statistical characteristics, calculate the correlation between the feature vectors and construct a feature association matrix. For example, the correlation coefficient between the temperature and humidity features is 0.75, indicating a strong positive correlation.

[0145] When obtaining the professional rating of a model node, factors such as the accuracy of historical task processing (such as an accuracy of 95%), processing latency (such as an average processing time of 100 milliseconds), and resource consumption (such as a CPU usage of 30%) are comprehensively considered. At the same time, the feature information in the encoding matrix is ​​extracted, matched and analyzed with the feature association matrix, the degree of overlap is calculated, and the feature association is generated. For example, the feature association of a model node in an image processing task is 0.85.

[0146] The professionalism score and feature relevance are normalized and the values ​​are mapped to the range of 0-1. The weight coefficient is set in an adaptive way. The professionalism score weight is set to 0.6 and the feature relevance weight is set to 0.4 initially. The initial affinity is obtained through weighted fusion.

[0147] A real-time feedback mechanism is established to continuously collect task processing status information of model nodes. When the real-time accuracy is lower than 90%, the processing delay exceeds 200 milliseconds, or the resource utilization rate exceeds 80%, that is, the task processing status differs from the expected value by more than the preset threshold, the system will automatically adjust the weight coefficient. For example, when the processing delay is too high, the delay-related weight coefficient can be lowered by 10% to correct the initial affinity and obtain a more accurate task affinity.

[0148] In this embodiment, by performing multi-dimensional feature extraction and association analysis on standard format data, accurate characterization and quantitative expression of data features are achieved, the accuracy and completeness of feature expression are improved, and a reliable data basis is provided for subsequent task allocation. The adaptive weight setting and weighted fusion method is adopted to organically combine the professional capability assessment of model nodes with the task feature matching degree, thereby achieving precise and personalized task allocation, improving the operating efficiency and resource utilization of the overall system, establishing a real-time feedback and dynamic adjustment mechanism, and being able to promptly discover and respond to abnormal situations during task execution. The real-time correction of task affinity is achieved through dynamic adjustment of weight coefficients, thereby ensuring the stability and reliability of the system and improving the quality and effect of task processing.

[0149] S3. Collect the performance data of the model node when processing the transmission data unit, combine the attention score matrix to perform online modeling of the response delay, throughput and resource utilization to generate a performance evaluation map, fuse the performance evaluation map and the model collaboration map to obtain a comprehensive evaluation network and convert it into a state space representation, use the state space representation as the input of the strategy network based on advantage learning, output the candidate node collaboration strategy, convert the candidate node collaboration strategy into an evaluation vector form, calculate the evaluation score on each evaluation dimension in combination with the pre-set performance indicators, construct a multi-dimensional strategy competition space based on the evaluation score, solve the optimal solution set through the fast non-dominated sorting algorithm and solve the key collaboration unit through the principal component analysis algorithm, perform correlation grouping based on the key collaboration unit and the spectral clustering algorithm, obtain the strategy recombination template and perform the cross recombination operation to obtain the optimal node collaboration strategy.

[0150] The strategy network based on advantage learning is a network model that optimizes the decision-making process by learning the advantage relationship between strategies, and is used for strategy improvement in reinforcement learning. The candidate node collaboration strategy is a strategy that designs a collaboration mechanism between multiple candidate nodes, and is used to optimize the interaction and resource sharing between nodes in a network or system. The multidimensional strategy competition space is a space containing multiple strategy dimensions, in which different strategies compete in multiple dimensions to seek the optimal strategy combination. The fast non-dominated sorting algorithm is a sorting method used in multi-objective optimization problems, which efficiently sorts by quickly determining the dominance relationship of solutions. The key collaboration unit is the most important component or node in the system, and its performance has a significant impact on the performance of the overall system, and is usually the focus of collaborative optimization. The spectral clustering algorithm is an algorithm for clustering analysis based on the eigenvalue decomposition of the Laplace matrix of a graph, and can handle complex nonlinear data structures. The strategy reorganization template is a structural template used to define and execute strategy reorganization, and is usually used to adjust strategies as needed in a dynamic environment to adapt to new conditions.

[0151] In an optional embodiment,

[0152] The performance data of the model node when processing the transmission data unit is collected, and the response delay, throughput and resource utilization are modeled online in combination with the attention score matrix to generate a performance evaluation map, the performance evaluation map and the model collaboration map are integrated to obtain a comprehensive evaluation network and converted into a state space representation, the state space representation is used as the input of the strategy network based on advantage learning, the candidate node collaboration strategy is output, the candidate node collaboration strategy is converted into an evaluation vector form, and the evaluation score is calculated in each evaluation dimension in combination with the pre-set performance indicators, and a multi-dimensional strategy competition space is constructed based on the evaluation score, the optimal solution set is solved by a fast non-dominated sorting algorithm, and the key collaboration unit is solved by a principal component analysis algorithm, and correlation grouping is performed based on the key collaboration unit and the spectral clustering algorithm, and a strategy reorganization template is obtained and a cross-reorganization operation is performed, and the optimal node collaboration strategy is obtained, including:

[0153] Deploy distributed performance probes on model nodes to collect performance data, record request arrival timestamps and response completion timestamps, use a sliding window method to calculate the average delay, maximum delay and delay variance of requests to obtain delay feature data, count the number of completed tasks according to a preset sampling period and calculate the moving average to obtain throughput feature data, sample at preset time intervals to obtain CPU usage, memory occupancy and network traffic, analyze resource usage patterns through cumulative distribution functions to obtain resource utilization feature data, detect and eliminate abnormal data points in the delay feature data, the throughput feature data and the resource utilization feature data based on the standard deviation threshold, and use a linear interpolation method to complete the missing data to obtain a complete performance data set;

[0154] Extracting long-term performance features and short-term performance features from the complete performance data set at different time scales through an attention mechanism, calculating performance indicator weights based on the long-term performance features and the short-term performance features, constructing a time series matrix for the complete performance data set according to time units, performing extreme value normalization processing on the time series matrix combined with the performance indicator weights to obtain a standardized performance matrix, and performing trend prediction on the standardized performance matrix based on a smoothing coefficient to obtain a performance evaluation map;

[0155] Establishing a node collaboration relationship diagram, calculating the historical data interaction frequency and the historical data interaction amount to obtain the node collaboration strength and setting the node collaboration strength as the connection weight, setting the performance evaluation map as the node feature attribute, setting the node collaboration strength as the connection feature attribute to obtain a hybrid feature network, and extracting the structural features of the hybrid feature network through convolution operation to obtain a feature map;

[0156] Generate an initial candidate collaboration strategy based on the node characteristic attributes and the connection characteristic attributes, evaluate the initial candidate collaboration strategy according to the long-term performance characteristics and the short-term performance characteristics, and obtain a strategy fitness distribution through normalization, and screen the initial candidate collaboration strategy according to the strategy fitness distribution to obtain a candidate node collaboration strategy, wherein the candidate node collaboration strategy includes a node grouping scheme, a task allocation ratio, and resource quota information;

[0157] The delay characteristic data is used to calculate the ratio of the actual task completion time to the theoretical completion time of the candidate node cooperation strategy to obtain a computing efficiency score, the resource utilization characteristic data is used to calculate the node load distribution variance of the candidate cooperation strategy to obtain a load balancing score, and the throughput characteristic data is combined to calculate the resource utilization score, communication overhead score, reliability score, scalability score, response time score and throughput score of the candidate cooperation strategy, and a piecewise function is used to normalize the computing efficiency score, the load balancing score and other scores to obtain an evaluation score;

[0158] Performing uniform grid division on each dimension of the evaluation score to construct a multidimensional strategy competition space, mapping the evaluation score to the multidimensional strategy competition space to obtain strategy position coordinates, sorting the strategy position coordinates by a fast non-dominated sorting algorithm to obtain an optimal solution set, performing a principal component analysis algorithm on the optimal solution set, extracting the main feature directions and calculating the feature contribution rate, and selecting the feature directions whose cumulative contribution rate exceeds a preset threshold as key collaboration units;

[0159] A strategy affinity matrix is ​​constructed based on the key collaboration unit, and the similarity between strategies is calculated to obtain a strategy similarity matrix. A spectral clustering algorithm is used to perform feature decomposition on the strategy similarity matrix and perform correlation grouping according to the size of the eigenvalue to obtain a strategy recombination template. A cross-recombination operation is performed on the strategy recombination template to generate a new candidate strategy, and an evaluation score of the new candidate strategy is calculated, and the strategy with the highest evaluation score is selected as the optimal node collaboration strategy.

[0160] The strategy fitness distribution is used to measure the distribution of the adaptive performance of the strategy in different environments or tasks, and is used to evaluate the comprehensive performance of the strategy. The node load distribution variance is a statistic that measures the load changes of each node in the network, and is used to evaluate the network load balance. The strategy affinity matrix is ​​a matrix that describes the mutual matching or synergy effects between different strategies, and is used to optimize strategy combinations and collaboration efficiency.

[0161] Deploy performance probes on each model node for data collection. The performance probes record the arrival time and completion time of each request, and use a 60-second sliding window to calculate the delay characteristics. For example, for 100 request samples collected by a node in a sampling period, the average delay is calculated to be 200ms, the maximum delay is 500ms, and the delay variance is 50ms. At the same time, the number of completed tasks is counted according to the 30-second sampling period, and the moving average of the last five periods is calculated as the throughput feature.

[0162] For resource usage, the CPU usage, memory usage, and network traffic are sampled every 10 seconds, and the resource usage pattern is analyzed through the cumulative distribution function. For example, the 95% quantile of the CPU usage is 80%, indicating that the CPU load of the node is heavy. Three times the standard deviation is used as the threshold to detect outliers, and data points that exceed the threshold are removed. For possible missing data, linear interpolation method is used to fill in the missing data to ensure data continuity.

[0163] The attention mechanism is used to extract long-term and short-term performance features. Long-term features focus on the performance change trend at the hour level, while short-term features focus on the fluctuation at the minute level. The weights of performance indicators are calculated based on these two types of features. For example, latency, throughput, and resource utilization are assigned weights of 0.4, 0.3, and 0.3, respectively. The collected performance data is constructed into a matrix according to the time series, normalized with the weights, and a smoothing coefficient of 0.8 is used for trend prediction to generate a performance evaluation map.

[0164] When building node collaboration relationships, the number of data interactions and the amount of data between nodes within 30 minutes are counted to calculate the collaboration strength. For example, the collaboration strength between node A and node B is 0.8, indicating that the two nodes are closely connected. The performance evaluation graph is used as the node attribute, and the collaboration strength is used as the connection attribute to extract structural features through the graph convolutional network.

[0165] Generate an initial set of collaborative strategies based on node attributes and connection attributes, including different node grouping schemes, task allocation ratios, and resource quota information. For example, a candidate strategy divides 8 nodes into 3 groups, each of which bears 40%, 35%, and 25% of the task load. Evaluate the strategy based on performance characteristics, and obtain the strategy fitness distribution through min-max normalization.

[0166] The selected candidate strategies are evaluated in multiple dimensions. The computational efficiency score considers the ratio of the actual completion time to the theoretical time, the load balancing score is based on the node load variance, and other dimensions such as resource utilization and communication overhead are also scored. The score is normalized to the range of 0-1 using a piecewise function.

[0167] In the strategy competition space, the evaluation space is divided into a 10×10×10 grid to map the strategy position coordinates. The optimal solution set is selected through fast non-dominated sorting, and then the main feature directions are extracted through principal component analysis. For example, the feature direction with a cumulative contribution rate of 85% is selected as the key collaboration unit.

[0168] Construct a strategy affinity matrix and calculate the cosine similarity between strategies. Use the spectral clustering algorithm to group and obtain the strategy recombination template. Perform cross-recombination operations to generate new candidate strategies, and select the best strategy as the final node collaboration solution through evaluation scores.

[0169] In this embodiment, through the collection and analysis of distributed performance probes and multi-dimensional performance indicators, accurate perception and anomaly detection of the performance status of model nodes are achieved, the stability and reliability of system operation are improved, and the feature extraction method based on the attention mechanism and graph convolutional network effectively captures the dynamic characteristics of node performance and collaborative relationships, improves the accuracy and prediction ability of performance evaluation, and uses multi-objective optimization and strategy reorganization methods to optimize collaborative strategies, achieving load balancing while ensuring computing efficiency, significantly improving the overall system performance and resource utilization efficiency.

[0170] In an optional embodiment,

[0171] Perform uniform grid division on each dimension of the evaluation score to construct a multidimensional strategy competition space, map the evaluation score to the multidimensional strategy competition space to obtain strategy position coordinates, sort the strategy position coordinates by a fast non-dominated sorting algorithm to obtain an optimal solution set, perform a principal component analysis algorithm on the optimal solution set, extract the main feature directions and calculate the feature contribution rate, and select the feature directions whose cumulative contribution rate exceeds a preset threshold as key collaboration units, including:

[0172] Performing maximum and minimum value normalization processing on the scoring data of the evaluation score to obtain normalized scoring data, determining the grid division strength according to the system performance requirements and performing equidistant division on each dimension of the normalized scoring data to obtain grid units, assigning a dimension index value as a unique identifier to each grid unit and constructing the multidimensional strategy competition space;

[0173] Determine the grid unit position of each candidate strategy according to the normalized scoring data of each candidate strategy in different dimensions, and map the evaluation score to the multidimensional strategy competition space based on the dimension index value of the grid unit to obtain the strategy position coordinates;

[0174] Construct a strategy dominance relationship matrix, determine the dominance relationship between candidate strategies by comparing their scores on each evaluation dimension, calculate the dominated count and dominance set of each candidate strategy based on the strategy dominance relationship matrix, classify candidate strategies with a dominated count of zero into a non-dominated strategy hierarchy, update the dominated counts of strategies dominated by candidate strategies in the non-dominated strategy hierarchy, repeat the operation of classifying into the non-dominated strategy hierarchy until all strategies are stratified, and determine the highest-level strategy in the non-dominated strategy hierarchy as the optimal solution set by a fast non-dominated sorting algorithm;

[0175] Construct a strategy scoring matrix, where the number of rows of the strategy scoring matrix is ​​the number of strategies in the optimal solution set, and the number of columns is the number of evaluation dimensions, calculate the mean vector and standard deviation vector of each evaluation dimension, perform centralization on the strategy scoring matrix, calculate the correlation coefficients between the evaluation dimensions to construct a correlation coefficient matrix, perform a principal component analysis algorithm on the correlation coefficient matrix, perform eigenvalue decomposition on the correlation coefficient matrix to obtain an eigenvalue sequence and an eigenvector sequence, extract the main feature direction, arrange the eigenvalue sequence in descending order, calculate the variance contribution rate and cumulative variance contribution rate of each eigenvalue, and determine the number of eigenvectors to be retained based on a preset cumulative variance contribution rate threshold;

[0176] The retained feature vector is determined as a key collaboration unit, and the contribution of the original evaluation dimension to the feature direction is determined according to the size of each component in the key collaboration unit.

[0177] The dimension index value refers to the position index of a dimension in the overall strategy or data in a high-dimensional space, which is used to locate a specific position in the feature or strategy space. The strategy position coordinates are the specific positions occupied by the strategies in the multidimensional space, and the relative relationship between different strategies is represented by the coordinates. The variance contribution rate refers to the ratio of the variance of each factor to the overall variance in statistics or data analysis, and is usually used for feature selection or data dimensionality reduction.

[0178] Normalize the evaluation score data. Take a certain evaluation system with five evaluation dimensions as an example. The original score data are: cost-effectiveness is 85 points, technological innovation is 92 points, market potential is 78 points, implementation difficulty is 88 points, and risk control is 83 points. Through the maximum and minimum value normalization, the score of each dimension is mapped to the range of 0 to 1, and the normalized score data are: 0.85, 0.92, 0.78, 0.88, and 0.83.

[0179] According to the system performance requirements, the grid division granularity is set to 0.1, and each dimension is divided equally. Taking the cost-effectiveness dimension as an example, the interval from 0 to 1 is divided into 10 grid units, and the interval length of each grid unit is 0.1. A dimension index value is assigned to each grid unit, such as the first grid unit index value of the cost-effectiveness dimension is 1, and so on. Grid division is performed on the five dimensions respectively to construct a five-dimensional strategic competition space.

[0180] For each candidate strategy, the grid unit position is determined according to the normalized score in each dimension. Assume there is a candidate strategy A, whose normalized scores in five dimensions are: 0.85, 0.92, 0.78, 0.88, 0.83, then its corresponding grid unit position is: 9, 10, 8, 9, 9. Through the dimension index value of the grid unit, the evaluation score is mapped to the multidimensional strategy competition space to obtain the strategy position coordinates.

[0181] In the strategy dominance relationship analysis, a strategy dominance relationship matrix is ​​constructed. Assuming there are three candidate strategies A, B, and C, the dominance relationship between them is determined by comparing their scores in five dimensions. If the score of strategy A in all dimensions is not lower than that of strategy B, and is higher than that of strategy B in at least one dimension, then strategy A dominates strategy B. Based on the dominance relationship matrix, the dominated count and dominance set of each strategy are calculated, and the strategies with a dominated count of zero are classified into the first-level non-dominated strategy level, and the dominated counts of other strategies dominated by these strategies are updated. Repeat this process until all strategies are layered. The strategies in the first layer constitute the optimal solution set.

[0182] Perform principal component analysis on the optimal solution set and construct a strategy scoring matrix. Calculate the mean and standard deviation of each evaluation dimension and center the scoring matrix. Calculate the correlation coefficients between the evaluation dimensions and construct a correlation coefficient matrix. Perform eigenvalue decomposition on the correlation coefficient matrix to obtain eigenvalues ​​and eigenvectors, arrange the eigenvalues ​​in descending order, and calculate the variance contribution rate of each eigenvalue. Assume that the cumulative variance contribution rate threshold is set to 85%, and select the eigenvectors whose cumulative contribution rate exceeds the threshold as the key collaboration unit. Analyze the size of each component of the eigenvector to determine the contribution degree of the original evaluation dimension to the feature direction.

[0183] In this embodiment, the maximum and minimum value normalization and uniform grid division are used to achieve standardized processing and spatial discretization of evaluation data, improve the comparability and computational efficiency of strategy evaluation, use a fast non-dominated sorting algorithm to perform multi-level classification of strategies, effectively identify the optimal strategy set, avoid the one-sidedness that may be caused by a single evaluation criterion, and improve the scientificity and comprehensiveness of strategy selection. By extracting key collaborative units through principal component analysis, dimensionality reduction and feature extraction of evaluation dimensions are achieved, revealing the inherent correlation between evaluation dimensions, and providing important data support for strategy optimization and decision-making.

[0184] Figure 2 FIG. 1 is a schematic diagram of the structure of a large model interaction system based on multi-model collaborative dialogue according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0185] The first unit is used to receive multimodal data and divide the multimodal data into a matrix of pixel blocks of the same size, generate a perceptual feature vector by calculating the brightness mean and color distribution of each pixel block, extract the high-frequency feature component and the low-frequency feature component corresponding to the perceptual feature vector by wavelet transform and construct a quantization threshold area, map the high-frequency feature component and the low-frequency feature component to a hexagonal honeycomb coding matrix according to the quantization threshold area, fill the main data information in the core layer, write the data time series relationship into the association layer, generate a data check bitmap in combination with the density estimation algorithm and store it in the verification layer, perform a multi-head attention mechanism on the coding matrix to generate an attention score matrix, superimpose the self-attention residual unit to perform feature dimensionality reduction, obtain a compressed vector and divide it into data blocks of fixed length, construct a redundant check packet in combination with the data check bitmap, and generate a transmission data unit;

[0186] The second unit is used to build a model communication network based on the transmission data unit and the attention score matrix and deploy multiple basic models as model nodes, randomly select multiple model nodes from the model nodes as consensus nodes, add the perception feature vector, the high-frequency feature component and the low-frequency feature component to the model node, calculate the professional score between nodes through graph convolution and build a weighted model collaboration graph, establish a two-way information channel on the model collaboration graph, encode the transmission data unit into a standard representation format in combination with the attention score matrix, calculate the task affinity in combination with the professional score and the feature correlation of the encoding matrix, generate an adaptive routing table in combination with the computing load of the model node and the verification result of the consensus node, transfer the transmission data unit between model nodes based on the adaptive routing table, and perform data integrity verification through a data check bitmap;

[0187] The third unit is used to collect performance data when the model node processes the transmission data unit, and to perform online modeling of the response delay, throughput and resource utilization in combination with the attention score matrix to generate a performance evaluation graph, to fuse the performance evaluation graph and the model collaboration graph to obtain a comprehensive evaluation network and convert it into a state space representation, to use the state space representation as the input of the strategy network based on advantage learning, to output candidate node collaboration strategies, to convert the candidate node collaboration strategies into an evaluation vector form, to calculate the evaluation scores on each evaluation dimension in combination with pre-set performance indicators, to construct a multi-dimensional strategy competition space based on the evaluation scores, to solve the optimal solution set through a fast non-dominated sorting algorithm and to solve the key collaboration units through a principal component analysis algorithm, to perform correlation grouping based on the key collaboration units and a spectral clustering algorithm, to obtain a strategy recombination template and to perform a cross-recombination operation to obtain the optimal node collaboration strategy.

[0188] According to a third aspect of the embodiments of the present invention,

[0189] An electronic device is provided, comprising:

[0190] processor;

[0191] a memory for storing processor-executable instructions;

[0192] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0193] According to a fourth aspect of the embodiments of the present invention,

[0194] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0195] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0196] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A large model interaction method based on multi-model collaborative dialogue, characterized in that: include: Receive multimodal data and divide the multimodal data into a matrix of pixel blocks of the same size, generate a perceptual feature vector by calculating the brightness mean and color distribution of each pixel block, extract the high-frequency feature component and the low-frequency feature component corresponding to the perceptual feature vector by wavelet transform and construct a quantization threshold area, map the high-frequency feature component and the low-frequency feature component to a hexagonal honeycomb coding matrix according to the quantization threshold area, fill the main data information in the core layer, write the data time series relationship into the association layer, generate a data check bitmap in combination with the density estimation algorithm and store it in the verification layer, perform a multi-head attention mechanism on the coding matrix to generate an attention score matrix, superimpose the self-attention residual unit to perform feature dimensionality reduction, obtain a compressed vector and divide it into data blocks of fixed length, construct a redundant check package in combination with the data check bitmap, and generate a transmission data unit; A model communication network is constructed based on the transmission data unit and the attention score matrix, and a plurality of basic models are deployed as model nodes. A plurality of model nodes are randomly selected from the model nodes as consensus nodes. The perception feature vector, the high-frequency feature component, and the low-frequency feature component are added to the model node. The professional scores between nodes are calculated by graph convolution and a weighted model collaboration graph is constructed. A bidirectional information channel is established on the model collaboration graph. The transmission data unit is encoded into a standard representation format in combination with the attention score matrix. The task affinity is calculated in combination with the professional score and the feature association of the encoding matrix. An adaptive routing table is generated in combination with the computational load of the model node and the verification result of the consensus node. The transmission data unit is transmitted between model nodes based on the adaptive routing table, and data integrity verification is performed through a data check bitmap. The performance data of the model node when processing the transmission data unit is collected, and the response delay, throughput and resource utilization are modeled online in combination with the attention score matrix to generate a performance evaluation graph, the performance evaluation graph and the model collaboration graph are integrated to obtain a comprehensive evaluation network and converted into a state space representation, the state space representation is used as the input of the strategy network based on advantage learning, the candidate node collaboration strategy is output, the candidate node collaboration strategy is converted into an evaluation vector form, the evaluation score is calculated on each evaluation dimension in combination with the preset performance indicators, a multidimensional strategy competition space is constructed based on the evaluation score, the optimal solution set is solved by a fast non-dominated sorting algorithm, and the key collaboration unit is solved by a principal component analysis algorithm, correlation grouping is performed based on the key collaboration unit and the spectral clustering algorithm, a strategy recombination template is obtained, and a cross recombination operation is performed to obtain the optimal node collaboration strategy.

2. The method according to claim 1, characterized in that Receiving multimodal data and dividing the multimodal data into pixel block matrices of the same size, generating a perceptual feature vector by calculating the brightness mean and color distribution of each pixel block, extracting high-frequency feature components and low-frequency feature components corresponding to the perceptual feature vector by wavelet transform, and constructing a quantization threshold region includes: Receive a multimodal data stream, extract a key frame sequence of video data in the multimodal data stream and parse it into an RGB three-channel matrix, convert the audio data in the multimodal data stream into a time-frequency spectrum diagram through a short-time Fourier transform, encode a character sequence of text data in the multimodal data stream into a word embedding vector, and use a unified data preprocessing module to map the RGB three-channel matrix, the time-frequency spectrum diagram and the word embedding vector into a pixel-level representation; Calculating the local entropy value distribution of the pixel-level representation, identifying the boundary of the data density change area based on the local entropy value distribution and marking the high entropy value area and the low entropy value area, using a first block size in the high entropy value area, and using a second block size larger than the first block size in the low entropy value area, determining the values ​​of the first block size and the second block size by iterative bisection, and dividing the pixel-level representation into a pixel block matrix; An adaptive threshold segmentation algorithm is used to divide the pixel value of each pixel block in the pixel block matrix into multiple brightness levels, the pixel block is scanned through a sliding window and a brightness distribution histogram of the pixels in the window is accumulated and counted, the first-order moment, the second-order moment and the third-order moment of the brightness distribution histogram are calculated to obtain brightness features, the K-means clustering algorithm is applied in the RGB color space to extract the main color tone and calculate the inter-class distance and the intra-class distance to obtain the color distribution features, and the brightness features and the color distribution features are combined to form a perceptual feature vector; Constructing a wavelet filter group including a low-pass decomposition filter and a high-pass decomposition filter, performing four-layer decomposition on the perceptual feature vector to obtain an approximate component and a detail component of each layer, passing the approximate component through the low-pass decomposition filter to obtain an input signal of the next layer, retaining high-frequency information in three directions for the detail component, and denoising the wavelet coefficients by a soft threshold method; The probability density distribution of the wavelet coefficients is calculated and local extreme points are identified. The coefficient interval is divided using a splitting strategy based on the variance ratio within the interval and a merging strategy based on the distance between intervals to obtain a non-uniform quantization threshold area. The high-frequency feature components and the low-frequency feature components are mapped to the non-uniform quantization threshold area.

3. The method according to claim 1, characterized in that According to the quantization threshold area, the high-frequency feature component and the low-frequency feature component are mapped to a hexagonal honeycomb coding matrix, the main data information is filled in the core layer, the data time series relationship is written into the association layer, and the data check bitmap is generated in combination with the density estimation algorithm and stored in the verification layer. The multi-head attention mechanism is performed on the coding matrix to generate an attention score matrix, and the self-attention residual unit is superimposed to perform feature dimensionality reduction to obtain a compressed vector and divide it into data blocks of fixed length. A redundant check package is constructed in combination with the data check bitmap, and the transmission data unit is generated, including: Constructing a honeycomb coding matrix composed of concentrically surrounded multi-layer hexagonal units, filling the high-frequency characteristic components and the low-frequency characteristic components in the core layer of the coding matrix by a spirally unfolded serpentine scanning method, recording the temporal dependency of data blocks in the association layer of the coding matrix, and storing a data density distribution check bitmap calculated based on a Gaussian kernel function in the verification layer of the coding matrix; Project the encoding matrix to the query space, key space and value space, calculate the inner product of the feature vectors in multiple attention heads to obtain attention scores, normalize the attention scores and perform weighted combination with the value vectors, and concatenate the output features of multiple attention heads through linear transformation to form an attention score matrix; Constructing a self-attention residual unit including a main branch and a bypass branch, performing a self-attention operation on the main branch, keeping the original features unchanged on the bypass branch, and fusing the outputs of the main branch and the bypass branch by addition and reducing the dimension by a multi-layer perceptron to obtain a compressed vector; Divide the compressed vector into a plurality of data segments of fixed length, count the distribution density of valid bits in the data segments, calculate the horizontal bit pattern characteristics and the vertical bit pattern characteristics of the data segments, generate a feature identification code according to the bit pattern characteristics and the distribution density, the feature identification code includes bit pattern type information and density level information, use the position-weighted Hamming distance to measure the similarity between the data segments, dynamically adjust the similarity threshold based on data reliability requirements, and form the data segments with similarity higher than the similarity threshold into data clusters; Allocating a check codeword length to the data cluster according to the density of the data segments in the data cluster, mapping the binary data sequence in the data cluster to elements in a finite field according to byte grouping, performing a polynomial division operation on the elements in the finite field based on a generating polynomial to obtain a redundant check code, rearranging the data bytes in the data cluster so that continuous data bytes are mapped to different data segments, determining an interleaving depth based on a burst error statistical characteristic, and performing Reed-Solomon coding on the rearranged data to generate an interleaving check code; Perform two-level cascade coding, wherein the first level coding generates an internal check code for the data segment for correcting random errors, and the second level coding generates an external check code for the data cluster for processing burst errors, and records the coding scheme type of the data segment, the redundant check code length, the interleaving depth and the error correction capability to generate a check parameter management table, package the data segment, the redundant check code, the interleaving check code, the internal check code, the external check code and the check parameter management table, and add a sequence number, a timestamp and a data length to generate a transmission data unit.

4. The method according to claim 1, characterized in that Based on the transmission data unit and the attention score matrix, a model communication network is constructed and multiple basic models are deployed as model nodes. Multiple model nodes are randomly selected from the model nodes as consensus nodes. The perception feature vector, the high-frequency feature component and the low-frequency feature component are added to the model node. The professional scores between nodes are calculated by graph convolution and a weighted model collaboration graph is constructed. A two-way information channel is established on the model collaboration graph. The transmission data unit is encoded into a standard representation format in combination with the attention score matrix. The task affinity is calculated in combination with the professional score and the feature association of the encoding matrix. An adaptive routing table is generated in combination with the computing load of the model node and the verification result of the consensus node. The transmission data unit is transmitted between model nodes based on the adaptive routing table. Data integrity verification is performed through a data check bitmap, including: A node structure of a model communication network is constructed based on the transmission data unit and the attention score matrix, a plurality of pre-selected basic models are deployed as model nodes in the communication network, a reliability score is calculated in the model node based on the node credit value, computing resource occupancy rate and network response time, a model node with a reliability score higher than a pre-set reliability threshold is used as a candidate consensus node, and a plurality of candidate consensus nodes are randomly selected as consensus nodes; Writing feature data into the model node, the feature data including a perceptual feature vector, a high-frequency feature component and a low-frequency feature component, storing the feature data in a key-value pair data structure, wherein the key is a feature identifier, and the value includes the feature data and corresponding timestamp information, and regularly clearing outdated data and updating the latest feature data according to the timestamp information; The professionalism scores of the model nodes are calculated by graph convolution, a node adjacency matrix is ​​constructed to represent the node connection relationship, a performance indicator matrix is ​​generated based on the historical task processing accuracy, computing latency and resource utilization efficiency of the model nodes, the professionalism scores are updated through multiple rounds of graph convolution iterations, a weighted model collaboration graph is constructed based on the professionalism scores, and the collaboration weights are determined according to the differences in professionalism scores between the model nodes, wherein the collaboration weights represent the closeness of collaboration between the model nodes; A bidirectional data transmission channel is established on the model collaboration diagram, the transmission bandwidth of the data transmission channel is dynamically adjusted, the transmission data unit is encoded in combination with the attention score matrix, the data encoding accuracy is determined according to the importance score in the attention score matrix, a high-precision encoding method is used for the transmission data unit whose importance score is higher than the average importance score, and a standard format data including a data header, a data payload and verification information is generated; Based on the standard format data, a feature association matrix is ​​constructed through structural analysis and feature analysis, and the feature association of the professional score and the coding matrix is ​​combined to set a weight coefficient and perform weighted fusion to obtain an initial affinity, and the task affinity is obtained by performing data correction in combination with real-time feedback data; An adaptive routing mechanism is constructed based on the task affinity, the computing load status of the model nodes is monitored, a network topology view is constructed in combination with the verification results of the consensus nodes, and a data transmission routing table is generated based on the network topology view, wherein the data transmission routing table is updated in real time with the status of the model nodes, the transmission data unit is transferred between the model nodes by data slicing, the network status of the transmission path is monitored in real time, and when a network performance degradation is detected, it is switched to an alternative transmission path, and the data integrity is verified by a data check bitmap after the data slicing arrives at the target node.

5. The method according to claim 4, characterized in that Based on the standard format data, a feature association matrix is ​​constructed through structural analysis and feature analysis, and the feature association of the professional score and the coding matrix is ​​combined to set a weight coefficient and perform weighted fusion to obtain the initial affinity. The task affinity is obtained by performing data correction in combination with real-time feedback data, including: Receive and parse standard format information, the standard format information including data header information, data payload information and metadata information, extract data type identifier, data length and timestamp from the data header information, and extract data feature description and processing requirements from the metadata information; Performing structural analysis on the data load information, extracting time series features in the data load information by using a sliding window method, extracting numerical features by using a numerical segmentation method, extracting categorical features by using a one-hot encoding method, and constructing the time series features, the numerical features, and the categorical features into a feature vector; Calculating the distribution characteristics of the feature vectors, calculating the mean, variance and data skewness of the feature vectors, calculating the correlation coefficients between the feature vectors based on the distribution characteristics, and generating a feature correlation matrix; Obtaining a professionalism score of a model node, wherein the professionalism score is calculated based on the historical task processing accuracy, processing delay, and resource consumption of the model node, extracting feature information from a coding matrix, matching and analyzing the feature information with the feature association matrix, calculating the degree of overlap between the feature information and the feature association matrix, and generating a feature association degree; Normalizing the professionalism score and the feature correlation, setting weight coefficients in an adaptive manner, and obtaining initial affinity through weighted fusion; A real-time feedback mechanism is established to collect task processing status information of the model node, wherein the task processing status information includes real-time accuracy, processing delay and resource occupancy rate. When the difference between the task processing status information and the expected value exceeds a preset threshold, the weight coefficient is adjusted based on the difference, and the initial affinity is corrected to obtain the task affinity.

6. The method according to claim 1, characterized in that The performance data of the model node when processing the transmission data unit is collected, and the response delay, throughput and resource utilization are modeled online in combination with the attention score matrix to generate a performance evaluation map, the performance evaluation map and the model collaboration map are integrated to obtain a comprehensive evaluation network and converted into a state space representation, the state space representation is used as the input of the strategy network based on advantage learning, the candidate node collaboration strategy is output, the candidate node collaboration strategy is converted into an evaluation vector form, and the evaluation score is calculated in each evaluation dimension in combination with the pre-set performance indicators, and a multi-dimensional strategy competition space is constructed based on the evaluation score, the optimal solution set is solved by a fast non-dominated sorting algorithm, and the key collaboration unit is solved by a principal component analysis algorithm, and correlation grouping is performed based on the key collaboration unit and the spectral clustering algorithm, and a strategy reorganization template is obtained and a cross-reorganization operation is performed, and the optimal node collaboration strategy is obtained, including: Deploy distributed performance probes on model nodes to collect performance data, record request arrival timestamps and response completion timestamps, use a sliding window method to calculate the average delay, maximum delay and delay variance of requests to obtain delay feature data, count the number of completed tasks according to a preset sampling period and calculate the moving average to obtain throughput feature data, sample at preset time intervals to obtain CPU usage, memory occupancy and network traffic, analyze resource usage patterns through cumulative distribution functions to obtain resource utilization feature data, detect and eliminate abnormal data points in the delay feature data, the throughput feature data and the resource utilization feature data based on the standard deviation threshold, and use a linear interpolation method to complete the missing data to obtain a complete performance data set; Extracting long-term performance features and short-term performance features from the complete performance data set at different time scales through an attention mechanism, calculating performance indicator weights based on the long-term performance features and the short-term performance features, constructing a time series matrix for the complete performance data set according to time units, performing extreme value normalization processing on the time series matrix combined with the performance indicator weights to obtain a standardized performance matrix, and performing trend prediction on the standardized performance matrix based on a smoothing coefficient to obtain a performance evaluation map; Establishing a node collaboration relationship diagram, calculating the historical data interaction frequency and the historical data interaction amount to obtain the node collaboration strength and setting the node collaboration strength as the connection weight, setting the performance evaluation map as the node feature attribute, setting the node collaboration strength as the connection feature attribute to obtain a hybrid feature network, and extracting the structural features of the hybrid feature network through convolution operation to obtain a feature map; Generate an initial candidate collaboration strategy based on the node characteristic attributes and the connection characteristic attributes, evaluate the initial candidate collaboration strategy according to the long-term performance characteristics and the short-term performance characteristics, and obtain a strategy fitness distribution through normalization, and screen the initial candidate collaboration strategy according to the strategy fitness distribution to obtain a candidate node collaboration strategy, wherein the candidate node collaboration strategy includes a node grouping scheme, a task allocation ratio, and resource quota information; The delay characteristic data is used to calculate the ratio of the actual task completion time to the theoretical completion time of the candidate node cooperation strategy to obtain a computing efficiency score, the resource utilization characteristic data is used to calculate the node load distribution variance of the candidate cooperation strategy to obtain a load balancing score, and the throughput characteristic data is combined to calculate the resource utilization score, communication overhead score, reliability score, scalability score, response time score and throughput score of the candidate cooperation strategy, and a piecewise function is used to normalize the computing efficiency score, the load balancing score and other scores to obtain an evaluation score; Performing uniform grid division on each dimension of the evaluation score to construct a multidimensional strategy competition space, mapping the evaluation score to the multidimensional strategy competition space to obtain strategy position coordinates, sorting the strategy position coordinates by a fast non-dominated sorting algorithm to obtain an optimal solution set, performing a principal component analysis algorithm on the optimal solution set, extracting the main feature directions and calculating the feature contribution rate, and selecting the feature directions whose cumulative contribution rate exceeds a preset threshold as key collaboration units; A strategy affinity matrix is ​​constructed based on the key collaboration unit, and the similarity between strategies is calculated to obtain a strategy similarity matrix. A spectral clustering algorithm is used to perform feature decomposition on the strategy similarity matrix and perform correlation grouping according to the size of the eigenvalue to obtain a strategy recombination template. A cross-recombination operation is performed on the strategy recombination template to generate a new candidate strategy, and an evaluation score of the new candidate strategy is calculated, and the strategy with the highest evaluation score is selected as the optimal node collaboration strategy.

7. The method according to claim 6, characterized in that Perform uniform grid division on each dimension of the evaluation score to construct a multidimensional strategy competition space, map the evaluation score to the multidimensional strategy competition space to obtain strategy position coordinates, sort the strategy position coordinates by a fast non-dominated sorting algorithm to obtain an optimal solution set, perform a principal component analysis algorithm on the optimal solution set, extract the main feature directions and calculate the feature contribution rate, and select the feature directions whose cumulative contribution rate exceeds a preset threshold as key collaboration units, including: Performing maximum and minimum value normalization processing on the scoring data of the evaluation score to obtain normalized scoring data, determining the grid division strength according to the system performance requirements and performing equidistant division on each dimension of the normalized scoring data to obtain grid units, assigning a dimension index value as a unique identifier to each grid unit and constructing the multidimensional strategy competition space; Determine the grid unit position of each candidate strategy according to the normalized scoring data of each candidate strategy in different dimensions, and map the evaluation score to the multidimensional strategy competition space based on the dimension index value of the grid unit to obtain the strategy position coordinates; Construct a strategy dominance relationship matrix, determine the dominance relationship between candidate strategies by comparing their scores on each evaluation dimension, calculate the dominated count and dominance set of each candidate strategy based on the strategy dominance relationship matrix, classify candidate strategies with a dominated count of zero into a non-dominated strategy hierarchy, update the dominated counts of strategies dominated by candidate strategies in the non-dominated strategy hierarchy, repeat the operation of classifying into the non-dominated strategy hierarchy until all strategies are stratified, and determine the highest-level strategy in the non-dominated strategy hierarchy as the optimal solution set by a fast non-dominated sorting algorithm; Construct a strategy scoring matrix, where the number of rows of the strategy scoring matrix is ​​the number of strategies in the optimal solution set, and the number of columns is the number of evaluation dimensions, calculate the mean vector and standard deviation vector of each evaluation dimension, perform centralization on the strategy scoring matrix, calculate the correlation coefficients between the evaluation dimensions to construct a correlation coefficient matrix, perform a principal component analysis algorithm on the correlation coefficient matrix, perform eigenvalue decomposition on the correlation coefficient matrix to obtain an eigenvalue sequence and an eigenvector sequence, extract the main feature direction, arrange the eigenvalue sequence in descending order, calculate the variance contribution rate and cumulative variance contribution rate of each eigenvalue, and determine the number of eigenvectors to be retained based on a preset cumulative variance contribution rate threshold; The retained feature vector is determined as a key collaboration unit, and the contribution of the original evaluation dimension to the feature direction is determined according to the size of each component in the key collaboration unit.

8. A large model interaction system based on multi-model collaborative dialogue, used to implement the method described in any one of claims 1 to 7, characterized in that: include: The first unit is used to receive multimodal data and divide the multimodal data into a matrix of pixel blocks of the same size, generate a perceptual feature vector by calculating the brightness mean and color distribution of each pixel block, extract the high-frequency feature component and the low-frequency feature component corresponding to the perceptual feature vector by wavelet transform and construct a quantization threshold area, map the high-frequency feature component and the low-frequency feature component to a hexagonal honeycomb coding matrix according to the quantization threshold area, fill the main data information in the core layer, write the data time series relationship into the association layer, generate a data check bitmap in combination with the density estimation algorithm and store it in the verification layer, perform a multi-head attention mechanism on the coding matrix to generate an attention score matrix, superimpose the self-attention residual unit to perform feature dimensionality reduction, obtain a compressed vector and divide it into data blocks of fixed length, construct a redundant check packet in combination with the data check bitmap, and generate a transmission data unit; The second unit is used to build a model communication network based on the transmission data unit and the attention score matrix and deploy multiple basic models as model nodes, randomly select multiple model nodes from the model nodes as consensus nodes, add the perception feature vector, the high-frequency feature component and the low-frequency feature component to the model node, calculate the professional score between nodes through graph convolution and build a weighted model collaboration graph, establish a two-way information channel on the model collaboration graph, encode the transmission data unit into a standard representation format in combination with the attention score matrix, calculate the task affinity in combination with the professional score and the feature correlation of the encoding matrix, generate an adaptive routing table in combination with the computing load of the model node and the verification result of the consensus node, transfer the transmission data unit between model nodes based on the adaptive routing table, and perform data integrity verification through a data check bitmap; The third unit is used to collect performance data when the model node processes the transmission data unit, and to perform online modeling of the response delay, throughput and resource utilization in combination with the attention score matrix to generate a performance evaluation graph, to fuse the performance evaluation graph and the model collaboration graph to obtain a comprehensive evaluation network and convert it into a state space representation, to use the state space representation as the input of the strategy network based on advantage learning, to output candidate node collaboration strategies, to convert the candidate node collaboration strategies into an evaluation vector form, to calculate the evaluation scores on each evaluation dimension in combination with pre-set performance indicators, to construct a multi-dimensional strategy competition space based on the evaluation scores, to solve the optimal solution set through a fast non-dominated sorting algorithm and to solve the key collaboration units through a principal component analysis algorithm, to perform correlation grouping based on the key collaboration units and a spectral clustering algorithm, to obtain a strategy recombination template and to perform a cross-recombination operation to obtain the optimal node collaboration strategy.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images