A server data processing method and system based on deep learning

Abstract

The present invention belongs to the field of data processing technology, and specifically relates to a server data processing method and system based on deep learning, which aims to solve the problems of low modal fusion efficiency, poor model adaptability, and insufficient resource utilization in multimodal data processing. First, the mixed data to be processed is preprocessed and divided into structured data, image data, text data, and time series data subsets; a time-varying neural network topology is constructed, and the network edge weights are dynamically adjusted based on server load and data characteristics to achieve preliminary extraction and association capture of multimodal features; a cross-modal feature extractor with an attention mechanism is used to generate a fused feature vector; a dynamic adaptive data processing module is used to match the deep learning model for parallel processing; finally, a reinforcement learning reward function is constructed based on processing efficiency, resource utilization, and accuracy, and the model is jointly iteratively optimized.

Description

Technical Field

[0001] The present invention belongs to the field of data processing technology, and in particular relates to a server data processing method and system based on deep learning. Background Art

[0002] With the rapid development of artificial intelligence (AI), servers are facing an increasingly diverse range of data types, encompassing multimodal forms such as structured, unstructured, and time-series data. Traditional data processing suffers from information fragmentation between modalities: traditional methods typically process data from different modalities independently and lack effective cross-modal correlation modeling mechanisms. For example, image classification and text sentiment analysis use separate models, failing to capture the potential correlation between image content and text semantics.

[0003] Limitations of static feature extraction: Existing cross-modal fusion methods often use fixed network structures, making them difficult to dynamically adapt to changes in data complexity. For example, static attention mechanisms are inefficient for extracting features from long text or complex images, and are unable to automatically adjust mask ratios or attention head allocation based on semantic complexity.

[0004] Lack of dynamic topology: Graph neural networks, as effective tools for processing connected data, often rely on statically preset topologies, making it impossible to dynamically optimize edge connection weights based on real-time data characteristics or server load. For example, when servers are heavily loaded, fixed topologies can block the fusion path for key modal features, increasing processing latency. Traditional architectures rely on manually preset mappings between models and tasks, making them unable to cope with mixed or dynamic data type changes. For example, in live video streaming scenarios, sudden high concurrency can cause servers deployed with fixed models to crash due to insufficient resources. Existing methods lack real-time awareness and dynamic response mechanisms to server load, often resulting in a combination of idle computing resources and overload lag. For example, lightweight tasks occupy high-performance GPU resources, while complex imaging tasks are forced to slow down due to insufficient resources.

[0005] Therefore, there is an urgent need for a server data processing method that can dynamically optimize network structure, efficiently integrate multimodal features, and adaptively adjust resource allocation. Summary of the Invention

[0006] The purpose of the present invention is to provide a server data processing method and system based on deep learning, so as to realize server data processing that dynamically optimizes network structure, efficiently integrates multimodal features and adaptively adjusts resource allocation.

[0007] In order to solve the above technical problems, the technical solutions adopted by the present invention are as follows:

[0008] In a first aspect, a server data processing method based on deep learning is provided, comprising the following steps:

[0009] S1: The server obtains mixed data to be processed, preprocesses the mixed data to be processed, and divides the preprocessed mixed data to be processed into four subsets: structured data, image data, text data, and time series data;

[0010] S2: constructing a time-varying neural network topology structure, inputting the four subsets into corresponding branches in the time-varying neural network topology structure for preliminary feature extraction, and dynamically adjusting the network edge connection weights according to the input data;

[0011] S3: A cross-modal feature extractor based on the attention mechanism captures the correlation features between data features of different modalities to obtain cross-modal correlation features, and then fuses the cross-modal correlation features to obtain a fused feature vector.

[0012] S4: constructing an adaptive data processing module, setting multiple deep learning network models in the adaptive data processing module, and the adaptive data processing module matching the mixed data to be processed with the corresponding deep learning network model based on the fusion feature vector for processing;

[0013] S5: Calculate the processing efficiency, resource utilization, and processing accuracy during data processing, establish the reward function of the reinforcement learning algorithm based on the processing efficiency, resource utilization, and processing accuracy, and perform joint iterative optimization of the deep learning network model based on the reward function;

[0014] S6: Based on the deep learning network model after joint iteration, the real-time mixed data to be processed is processed to obtain multimodal data processing results.

[0015] Preferably, the specific process of step S1 is as follows:

[0016] S11: The server receives mixed data to be processed through a network interface or a storage device interface, performs preliminary verification on the integrity and format of the data, and verifies whether the data file is damaged or whether data fields are missing. If errors or incompleteness are found in the data, an error prompt is triggered and reception is stopped, requiring retransmission of the data.

[0017] S12: performing data cleaning and format standardization on the mixed data to be processed, removing noise data, filling missing values, converting different types of data into a format suitable for processing by a deep learning network model, and adding type labels;

[0018] S13: Divide the preprocessed mixed data into four subsets based on type labels: structured data, image data, text data, and time series data.

[0019] Preferably, the specific process of step S2 is as follows:

[0020] S21: Map structured data subsets into high-dimensional feature vectors through an embedding layer, use a deformable convolutional feature extractor to extract multi-scale features from images, focus on key areas through a spatial attention mechanism to output a feature map sequence, adjust the mask ratio based on the text length and semantic complexity, capture long-range dependencies in the text through a multi-head self-attention mechanism, use the attention mechanism LSTM unit to process time series data, and dynamically allocate attention to different time steps through attention weights to capture long-term and short-term patterns in time series data;

[0021] S22: The modal features of different dimensions output by the four subset branches are used as the initial node set of the deep learning network model. Each node represents the feature representation of a specific modality. The initial edge weight matrix W∈Rn×n represents the connection strength between nodes, where n is the number of nodes. The edge weights are initialized as learnable parameters and initialized by random Gaussian distribution.

[0022] S23: Collects hardware indicators such as CPU utilization, memory usage, and I / O throughput of the server in real time, as well as software indicators such as data processing delay and model inference speed, and converts the multi-dimensional load indicators into weight adjustment coefficients α∈[0,1] through a nonlinear mapping function φ;

[0023] S24: Calculate the statistical characteristics of the input data of each branch, including variance, entropy, and KL divergence of feature distribution, evaluate the complexity and uncertainty of the data, calculate the correlation between different modalities through the cross-modal attention mechanism, and generate the attention weight matrix;

[0024] S25: Fusion of data-driven attention weights with server load-adjusted weights;

[0025] S26: Set a sliding time window T and update the edge weight matrix Wt at each time step t. When new data arrives, use incremental learning to update the edge weights to avoid global recalculation.

[0026] S27: When the edge weight change exceeds the threshold, the topology structure is reorganized, adding or deleting node connections to form a new subnetwork structure.

[0027] Preferably, the specific process of step S3 is as follows:

[0028] S31: Project modal features of different dimensions into a shared latent space, achieve dimension alignment through linear transformation, and enhance feature correlation within each modal feature;

[0029] S32: Capture the associations between different modal features through a query-key-value pair mechanism, map server CPU utilization and memory occupancy indicators to attention weight adjustment coefficients, and integrate server load information with the correlation between modal features of different dimensions;

[0030] S33: Design attention heads of different scales to capture fine-grained to coarse-grained correlation features, construct inter-modal correlation strength matrices, and quantify the mutual influence of different modalities;

[0031] S34: Calculate the attention weights between the two modal features, construct the correlation strength matrix according to the attention weights, perform weighted fusion of the modal features based on the correlation strength matrix, and introduce a residual structure to retain the original feature information.

[0032] Preferably, the specific process of step S33 is as follows:

[0033] S331: Convert the input features into a feature tensor, and evenly split the feature tensor into h sub-tensors according to the channel dimension. Each sub-tensor corresponds to an independent attention head.

[0034] S332: Assign different key-value pair dimensions to each attention head. Small dimensions capture fine-grained associations, while large dimensions capture coarse-grained associations.

[0035] S333: Apply different types of positional encodings to attention heads of different scales to enhance the perception of local and global relationships;

[0036] S334: For each modal feature, calculate its correlation strength matrix.

[0037] Preferably, in step S34, the attention weights between the two modal features are calculated, the correlation strength matrix is ​​calculated according to the attention weights, and the weighted fusion of each modal feature is performed based on the correlation strength matrix. At the same time, the specific process of introducing the residual structure to retain the original feature information is as follows:

[0038] S341: Each modal feature is linearly transformed to obtain a query vector, a key vector, and a value vector, and the attention score is calculated. The attention weight between each modal feature is obtained through the softmax function.

[0039] S342: Constructing a correlation strength matrix based on the attention weights between the two modal features, where each element in the matrix represents the attention weight of modality i to modality j;

[0040] S343: According to the correlation strength matrix M, the features of each modality are weighted and fused. The fused features are globally average pooled to obtain channel descriptors. The channel weights are calculated through two fully connected layers and an activation function. The channel weights are multiplied by the fused features channel by channel to obtain enhanced fused features.

[0041] Preferably, in step S4, the specific process of the adaptive data processing module matching the mixed data to be processed with the corresponding deep learning network model based on the fused feature vector for processing is as follows:

[0042] S41: establishing a model-task mapping relationship, and the adaptive data processing module analyzes the fused feature vector and calculates the variance, entropy and cross-correlation of the fused feature vector;

[0043] S42: Determine the task type of the mixed data to be processed based on the analysis results, and select a preliminary network model based on the task type and the model-task mapping relationship;

[0044] S43: When there are task records of the same type in the historical dynamic optimization table, a matching score of the preliminary calculation model is calculated based on the reinforcement learning algorithm, and the network model is adjusted according to the matching score;

[0045] S44: Extract task priority labels from the fused features. High-priority data directly enters the hot queue of each network model instance pool, while medium and low-priority data enter the ordinary queue. Each network model processes data based on the queue.

[0046] Preferably, the specific process of step S5 is as follows:

[0047] S51: Embed a monitoring module in the data processing process to collect real-time data processing volume, processing time, actual resources used, total available resources, correct prediction data, and total number of samples;

[0048] S52: Calculate the processing efficiency, resource usage, and processing accuracy during data processing. The calculation formula for processing efficiency is as follows:

[0049] Processing efficiency = amount of data processed / processing time;

[0050] The resource utilization rate is calculated as follows:

[0051] Resource utilization rate = actual used resources / total available resources × 100%;

[0052] Processing accuracy = correct prediction data / total number of samples;

[0053] S53: Aggregate indicators according to a fixed time window, generate a state vector, and set an action space, where the action space matches the optimization direction of the model;

[0054] S54: Co-optimization of the original loss function of the deep learning model and the reinforcement learning reward function.

[0055] In a second aspect, a server data processing system based on deep learning is provided, which is used to implement the server data processing method based on deep learning, including a data acquisition module, a preprocessing module, a time-varying neural network topology structure, a feature extraction module, a feature fusion module, an adaptive data processing module, and a reinforcement learning module, wherein the data acquisition module is connected to the preprocessing module, the preprocessing module is connected to the time-varying neural network topology structure, the feature extraction module is arranged in the time-varying neural network topology structure, the feature extraction module is connected to the feature fusion module, the feature fusion module is connected to the adaptive data processing module, and the adaptive data processing module is connected to the reinforcement learning module;

[0056] The data acquisition module is used to acquire mixed data to be processed;

[0057] The preprocessing module is used to preprocess the mixed data to be processed, and divide the preprocessed mixed data to be processed into four subsets: structured data, image data, text data and time series data;

[0058] The time-varying neural network topology structure is used to input the four subsets into corresponding branches in the time-varying neural network topology structure for preliminary feature extraction. The time-varying neural network topology structure dynamically adjusts the network edge connection weights according to the input data to capture the correlation features between the features of different modal data to obtain cross-modal correlation features;

[0059] The feature fusion module is used to perform fusion processing based on cross-modal correlation features to obtain a fused feature vector;

[0060] The adaptive data processing module sets a plurality of deep learning network models, and matches the mixed data to be processed with the corresponding deep learning network models based on the fusion feature vector for processing;

[0061] The reinforcement learning module is used to calculate the processing efficiency, resource utilization, and processing accuracy during the data processing process, establish a reward function for the reinforcement learning algorithm based on the processing efficiency, resource utilization, and processing accuracy, and perform joint iterative optimization on the deep learning network model based on the reward function.

[0062] The beneficial effects of the present invention include:

[0063] The deep learning-based server data processing method and system provided by the present invention preprocesses mixed data to be processed and divides it into structured data, image data, text data, and time series data subsets; constructs a time-varying neural network topology structure, dynamically adjusts network edge weights based on server load and data characteristics, and realizes preliminary extraction and association capture of multimodal features; utilizes a cross-modal feature extractor with an attention mechanism to generate a fused feature vector; matches the deep learning model through an adaptive data processing module for parallel processing; finally, constructs a reinforcement learning reward function based on processing efficiency, resource utilization, and accuracy, and performs joint iterative optimization on the model.

[0064] First, through a dual dynamic adjustment mechanism based on server load and data features, the graph neural network's edge connection weights can optimize the multimodal feature fusion path in real time. In multimodal sentiment analysis tasks, compared to traditional static GNNs, the F1 score for cross-modal correlation feature extraction is significantly improved, and the model's accuracy in image-text semantic alignment is also significantly enhanced.

[0065] Secondly, the dynamic adaptive data processing architecture integrates feature vector analysis variance, entropy, and cross-correlation coefficients with reinforcement learning scheduling to complete model selection in a very short time, improving efficiency compared to manual parameter adjustment. Furthermore, the combination of priority queues and elastic resource scheduling significantly reduces the latency of real-time data processing.

[0066] Thirdly, by employing targeted extraction methods for different modal data, feature extraction accuracy is effectively improved compared to general extraction methods. By combining server load with data characteristics, network edge connection weights are adjusted in real time. When the server is highly loaded, feature fusion paths are automatically optimized to reduce processing latency. Weights are also adjusted based on data complexity, enabling more accurate feature extraction from complex data and improving model generalization. When edge weight changes exceed a threshold, the topology is restructured to quickly adapt to changes in data distribution. This reduces fluctuations in model performance during significant shifts in data patterns, maintaining stable processing results.

[0067] Thirdly, the modal features are projected into a shared latent space and internal correlations are enhanced, so that different modal features can be better integrated in a unified space and cross-modal semantic consistency can be improved. By integrating server load information with modal correlation, key correlation feature extraction is prioritized when resources are tight, avoiding information loss due to insufficient resources and improving the integrity of important feature extraction. Attention heads of different scales are designed to comprehensively capture correlation features from fine-grained to coarse-grained. Compared with a single scale, complex relationships between modalities can be more meticulously explored. Based on the weighted fusion of the correlation strength matrix and the introduction of a residual structure, cross-modal information is fully integrated while retaining the original feature details, which improves the information integrity of the fused feature vector and effectively improves the accuracy of subsequent processing.

[0068] Finally, by analyzing the fused feature vectors to determine the task type and combining them with the model-task mapping relationship to select models, this method improves task-model matching accuracy and processing efficiency compared to fixed model configurations. Reinforcement learning is used to calculate matching scores and adjust the model, continuously optimizing the model selection strategy. Through iteration, this improves the average model processing efficiency and reduces resource waste. A reward function is constructed based on processing efficiency, resource utilization, and processing accuracy. Through collaborative optimization of the combined loss function, this balance is achieved, improving overall server resource utilization compared to single-target optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] Figure 1 Schematic diagram of the flow of the server data processing method based on deep learning of the present invention.

[0070] Figure 2 This is a flow chart of the present invention for capturing cross-modal correlation features and fusing them. DETAILED DESCRIPTION

[0071] The following is combined with Figure 1~Figure 2 The present invention is described in further detail:

[0072] Example 1

[0073] See attached Figure 1 As shown, a server data processing method based on deep learning includes the following steps:

[0074] S1: The server obtains mixed data to be processed, preprocesses the mixed data to be processed, and divides the preprocessed mixed data to be processed into four subsets: structured data, image data, text data, and time series data;

[0075] S2: constructing a time-varying neural network topology structure, inputting the four subsets into corresponding branches in the time-varying neural network topology structure for preliminary feature extraction, and dynamically adjusting the network edge connection weights according to the input data;

[0076] S3: A cross-modal feature extractor based on the attention mechanism captures the correlation features between data features of different modalities to obtain cross-modal correlation features, and then fuses the cross-modal correlation features to obtain a fused feature vector.

[0077] S4: constructing an adaptive data processing module, setting multiple deep learning network models in the adaptive data processing module, and the adaptive data processing module matching the mixed data to be processed with the corresponding deep learning network model based on the fusion feature vector for processing;

[0078] S5: Calculate the processing efficiency, resource utilization, and processing accuracy during data processing, establish the reward function of the reinforcement learning algorithm based on the processing efficiency, resource utilization, and processing accuracy, and perform joint iterative optimization of the deep learning network model based on the reward function;

[0079] S6: Based on the deep learning network model after joint iteration, the real-time mixed data to be processed is processed to obtain multimodal data processing results.

[0080] This embodiment uses an intelligent traffic management system as an application scenario to describe the specific implementation of the aforementioned server data processing method. In intelligent traffic management, the system must process mixed data from multiple devices, including images captured by traffic cameras, time-series data collected by traffic flow sensors, text information related to traffic regulations, and structured data such as vehicle registrations, to achieve efficient traffic monitoring and management.

[0081] The server obtains mixed data to be processed from traffic equipment deployed at various intersections in the city. This data includes real-time video streams captured by traffic cameras, which are subsequently broken down into image data; time-series data such as vehicle volume and speed collected every minute by traffic flow sensors; text data from traffic regulation documents; and structured data from vehicle registration systems, including vehicle ID, vehicle model, and owner information. The server first preprocesses the mixed data. After preprocessing, the mixed data is divided into four subsets: a structured data subset containing structured information such as vehicle registration information; an image data subset containing images of intersection traffic conditions extracted from the video stream; a text data subset storing text related to traffic regulations; and a time-series data subset recording real-time data collected by traffic flow sensors.

[0082] A time-varying neural network topology is constructed, with four branches corresponding to the processing of structured data, image data, text data, and time series data. A subset of structured data is fed into a branch dedicated to structured data processing. This branch uses a multi-layer perceptron architecture to perform preliminary feature extraction on structured information such as vehicle registrations. A subset of image data is fed into a convolutional neural network branch, which extracts features such as traffic signs and vehicle types from the image through operations such as convolutional and pooling layers. A subset of text data is fed into a recurrent neural network or Transformer branch to capture semantic information from traffic regulations. A subset of time series data is fed into a long-short-term memory network branch to extract regular characteristics of traffic flow over time. During the data input process, the time-varying neural network topology dynamically adjusts the network edge connection weights based on the characteristics of the input data using a backpropagation algorithm to optimize feature extraction.

[0083] A cross-modal feature extractor is constructed based on the attention mechanism, which can capture correlations between features in data from different modalities. For example, the information about a congested road section identified in an image is combined with the time-series traffic flow data for that section and textual information on congestion management in traffic regulations. The attention mechanism calculates the correlation weights between features in different modalities, highlighting key features and generating cross-modal correlation features. Then, based on these cross-modal correlation features, a fusion process is performed to integrate feature information from different modalities into a fused feature vector, achieving deep fusion of multimodal data.

[0084] An adaptive data processing module is constructed, incorporating multiple deep learning network models. These models can include the Prophet model for traffic flow prediction, the YOLO model for traffic accident detection, and the BERT model for intelligent question-answering regarding traffic regulations. Based on the fused feature vector, the adaptive data processing module analyzes the type and requirements of the mixed data to be processed and matches it with the appropriate deep learning network model. For example, if the fused feature vector indicates data related to traffic flow prediction, the Prophet model is used for processing; if it involves traffic accident image detection, the YOLO model is used.

[0085] During data processing, processing efficiency is calculated in real time, including the amount of data processed per unit time, resource utilization (including CPU and memory usage), and processing accuracy, such as the accuracy of traffic flow prediction and the correct identification rate of traffic accident detection. These three metrics are used to establish a reward function for the reinforcement learning algorithm. Based on this reward function, the deep learning network model is jointly iteratively optimized using the reinforcement learning algorithm, continuously adjusting model parameters to improve the model's overall performance in terms of processing efficiency, resource utilization, and accuracy. Based on the jointly iterated deep learning network model, mixed data collected in real time is processed. When new traffic data arrives, data preprocessing, feature extraction, fusion, and model matching are performed according to the above steps, ultimately obtaining multimodal data processing results. These results can be used in applications such as real-time traffic flow monitoring, traffic accident warnings, and intelligent traffic law consultation, providing strong support for urban traffic management.

[0086] Example 2

[0087] Based on Example 1, the specific process of step S1 is as follows:

[0088] S11: The server receives mixed data to be processed through a network interface or a storage device interface, performs preliminary verification on the integrity and format of the data, and verifies whether the data file is damaged or whether the data field is missing. If errors or incompleteness are found in the data, an error prompt is triggered and reception is stopped, requiring retransmission of the data.

[0089] The server is equipped with a variety of data receiving interfaces, including network interfaces for real-time data transmission from traffic cameras, flow sensors and other devices, as well as storage device interfaces for reading traffic regulations documents, vehicle registration data, etc. stored in local storage devices. When data transmission begins, the server initiates a preliminary verification procedure. For data integrity verification, the server checks the size, hash value and other information of the data file. Taking the video stream data file transmitted from a traffic camera as an example, the server pre-stores the expected size range and hash value of the normal video stream file of the camera. If the size of the received video stream file exceeds the reasonable range, or the calculated hash value does not match the expectation, the data file is determined to be damaged.

[0090] Regarding data field checks, for structured data such as vehicle registrations, the server checks each data record against a pre-set data format template to ensure it contains all required fields, such as vehicle ID, vehicle model, and owner information. If any fields are missing, the data is considered incomplete. Upon detecting data errors or incompleteness, the server immediately triggers an error message, records the error in the system log, and displays the error details to the administrator on the management interface. The server also stops receiving the current data transmission and sends a request to the data sender to retransmit the data, ensuring the integrity and accuracy of the received data.

[0091] S12: Perform data cleaning and format standardization on the mixed data to be processed, remove noise data, fill missing values, convert different types of data into a format suitable for deep learning network model processing, and add type labels.

[0092] After completing the initial verification, the server performs in-depth cleaning and format standardization on the mixed data to be processed. During the data cleaning phase, if the time series data collected by traffic flow sensors contains abnormally fluctuating noise data, such as a significant and unreasonable increase or decrease in traffic volume at a certain moment, the server uses statistical methods, such as outlier detection based on mean and standard deviation, to identify and remove this noise data. For missing traffic flow and speed data, linear interpolation and time series prediction are used to fill in the missing values.

[0093] Traffic regulations text data may contain garbled characters, repeated paragraphs, and other issues. The server corrects these garbled characters through character encoding conversion and uses a text similarity algorithm to detect and delete repeated paragraphs. Regarding format standardization, image data collected by traffic cameras is uniformly converted to JPEG or PNG format, and the image resolution is adjusted to a size suitable for subsequent deep learning model processing. Structured data is converted to standard formats such as CSV and JSON, and data fields are converted and named in a standardized manner. Text data is preprocessed through word segmentation and part-of-speech tagging, and converted into a word vector representation. After processing, each piece of data is given a clear type label based on its data type, such as "structured data," "image data," "text data," or "time series data."

[0094] S13: Based on the type tags, the preprocessed mixed data to be processed is divided into four subsets: structured data, image data, text data, and time series data. The server divides the preprocessed mixed data to be processed into four subsets based on the added type tags. Using an automated script or program, all data records are traversed and, based on the type tags, data records labeled "structured data" are extracted to form the structured data subset, which contains structured information such as vehicle registrations. Data records labeled "image data" are grouped into the image data subset, which contains images of traffic conditions at intersections extracted from video streams. Text data related to traffic regulations, labeled "text data," is integrated into the text data subset. Real-time data collected by traffic flow sensors, labeled "time series data," is classified into the time series data subset. After the division is completed, the data in each subset is clear and distinct, providing a data foundation for subsequent feature extraction and processing.

[0095] The specific process of step S2 is as follows:

[0096] S21: Map the structured data subset into a high-dimensional feature vector through the embedding layer, use the deformable convolutional feature extractor to extract multi-scale features of the image, focus on the key areas through the spatial attention mechanism to output the feature map sequence, adjust the mask ratio according to the text length and semantic complexity, capture the long-distance dependencies of the text through the multi-head self-attention mechanism, use the attention mechanism LSTM unit to process time series data, dynamically allocate attention to different time steps through the attention weights, and capture the long-term and short-term patterns in the time series data.

[0097] For structured data subsets such as vehicle registration data, the server inputs them into an embedding layer. Through training, the embedding layer maps low-dimensional, discrete structured data (including vehicle IDs and vehicle models) into high-dimensional, continuous feature vectors. This transforms vehicle IDs from simple numeric codes into high-dimensional vectors containing latent semantic information such as vehicle type and age, providing a richer feature representation for subsequent processing. Image data collected by traffic cameras enters a deformable convolutional feature extractor. This extractor dynamically adjusts the sampling position of the convolution kernel based on image content, extracting multi-scale features from the image. In complex traffic scene images, deformable convolutions can more accurately capture the features of objects such as vehicles, pedestrians, and traffic signs. Subsequently, a spatial attention mechanism takes effect, analyzing the feature importance of each region of the image to focus on key areas of the traffic scene, such as intersections and high-accident areas. The output is a sequence of feature maps that highlight information valuable for subsequent processing.

[0098] For text data such as traffic regulations, the server first analyzes the text length and semantic complexity. For longer and more complex regulations, the masking ratio is appropriately increased. During multi-head self-attention processing, the text is masked, allowing the model to learn to predict the masked portions, thereby capturing long-range dependencies within the text. For example, when understanding provisions regarding accident liability determination in traffic regulations, it can correlate related conditions and results across different paragraphs to accurately grasp the regulatory semantics. Time series data collected by traffic flow sensors is processed by LSTM units, an attention mechanism. At each time step, the LSTM unit calculates an output based on the current input and historical state. Simultaneously, the attention mechanism dynamically allocates attention to different time steps based on data characteristics. When analyzing traffic flow trends, it assigns higher attention weights to key time steps, such as peak hours, effectively capturing long-term trends (such as weekday and weekend traffic differences) and short-term patterns (such as traffic fluctuations during morning and evening rush hours) in the time series data.

[0099] S22: The modal features of different dimensions output by the four subset branches are used as the initial node set of the deep learning network model. Each node represents the feature representation of a specific modality. The initial edge weight matrix W∈Rn×n represents the connection strength between nodes, where n is the number of nodes. The edge weights are initialized as learnable parameters and initialized by random Gaussian distribution.

[0100] The server uses the high-dimensional feature vectors obtained by embedding the structured data subset, the feature graph sequence output by processing the image data, the feature representation of the text data obtained through the multi-head self-attention mechanism, and the features of the time series data processed by the attention mechanism's LSTM unit as the initial node set for the deep learning network model. Each node represents the feature representation of a specific modality. For example, the node corresponding to the image data contains the target feature information in the traffic scene. The edge weights are initialized as learnable parameters using a random Gaussian distribution. This ensures a certain level of connection strength in the network's initial state, which will be adjusted based on the data and server load, providing a foundation for information exchange between nodes.

[0101] S23: Collect the server's hardware indicators of CPU utilization, memory occupancy, and I / O throughput in real time, as well as software indicators of data processing delay and model inference speed, and convert the multidimensional load indicators into weight adjustment coefficients α, α∈[0,1], through a nonlinear mapping function φ.

[0102] The server collects its own hardware and software metrics in real time. Hardware metrics include CPU utilization, memory usage, and I / O throughput, as captured through system monitoring tools. Software metrics include data processing latency and model inference speed. These multi-dimensional load metrics are fed into a nonlinear mapping function φ, which, through training and parameter optimization, converts the load metrics into a weight adjustment coefficient α. When server CPU utilization is high and data processing latency is high, α is adjusted accordingly, optimizing the network structure based on load conditions and balancing resource utilization and processing efficiency.

[0103] S24: Calculate the statistical characteristics of the input data of each branch, including variance, entropy, KL divergence of feature distribution, evaluate the complexity and uncertainty of the data, calculate the correlation between different modal features through the cross-modal attention mechanism, and generate the attention weight matrix A. The elements in the attention weight matrix A are A( i , j ), which represents the data-driven attention weight:

[0104] A( i , j )= softmax (Q( x i )·K( x j ) T / ) ;

[0105] in, softmax is the normalized similarity score, x i and x jare input features, Q( x i ) and K( x j ) are query vector and key vector respectively, is the scaling factor, d is the feature dimension, T is the matrix transpose operation;

[0106] S25: Fusion of data-driven attention weights with server load-adjusted weights:

[0107] W''( i , j ) = β ·W'( i , j ) + (1- β )·A( i , j );

[0108] Among them, W''( i , j ) is the weight after the fusion of the data-driven attention weight and the server load-adjusted weight, W'( i , j ) is the weight after server load adjustment, β The learnable balance parameter, A( i , j ) is the data-driven attention weight;

[0109] S26: Set the sliding time window T and update the edge weight matrix W at each time step t t , when new data arrives, incremental learning is used to update edge weights to avoid global recalculation:

[0110] W t ( i , j ) = ρ·W t-1 ( i , j )+ (1-ρ)·W''( i , j );

[0111] Among them, ρ is the decay coefficient, 0≤ρ≤1, which controls the balance between historical weight and new weight, W t-1 ( i , j ) is the edge weight matrix at time step t, W''( i , j ) is the weight after the data-driven attention weight and the server load-adjusted weight are fused;

[0112] S27: When the edge weight change exceeds the threshold, the topology reorganization is triggered, and node connections are added or deleted to form a new subnet structure. The server will continuously monitor the changes in the edge weight matrix. When the change in edge weight exceeds the preset threshold, it means that the network structure needs to be adjusted to better adapt to data and load changes. At this time, the topology reorganization is triggered, and the system determines which node connections need to be added or deleted based on the current weight situation. When there are significant changes in traffic flow patterns or drastic fluctuations in server load, a new subnet structure is formed through reorganization to improve the network's adaptability and efficiency in data processing.

[0113] Through this dual dynamic adjustment mechanism—driven by server load and data features—the time-varying neural network topology can adaptively optimize the multimodal feature fusion path under resource constraints, achieving efficient and accurate data processing. This design not only improves the model's generalization capabilities but also significantly enhances the system's robustness to dynamic data streams.

[0114] Example 3

[0115] On the basis of Example 1 or Example 2, see Figure 2 As shown, the specific process of step S3 is as follows:

[0116] S31: Project modal features of different dimensions into a shared latent space, achieve dimension alignment through linear transformation, and enhance feature correlation within each modal feature;

[0117] S32: Capture the associations between different modal features through a query-key-value pair mechanism, map server CPU utilization and memory occupancy indicators to attention weight adjustment coefficients, and integrate server load information with the correlation between modal features of different dimensions;

[0118] S33: Design attention heads of different scales to capture fine-grained to coarse-grained correlation features, construct inter-modal correlation strength matrices, and quantify the mutual influence of different modalities;

[0119] S34: Calculate the attention weights between the two modal features, construct the correlation strength matrix according to the attention weights, perform weighted fusion of the modal features based on the correlation strength matrix, and introduce a residual structure to retain the original feature information.

[0120] Project the modal features of different dimensions into the shared latent space, design an independent fully connected layer for each modality, and map the features to the shared latent space dimension d =384, establish a structured feature matrix: h struct =ReLU( W struct ⋅[vehicle ID, vehicle model, speed, lane number]), image feature matrix:h image =ReLU( W image ⋅Vehicle feature vector), text feature matrix: h text =ReLU( W text ⋅Regulation word vector), time series feature matrix: h time =ReLU( W time ⋅Traffic time series characteristics).

[0121] Apply spatial self-attention to vehicle features, focusing on key areas such as license plates and vehicle outlines. For example, in congested scenarios, it can enhance the correlation between the taillight features of the vehicle in front and the braking action of the vehicle behind.

[0122] Using “intersection congestion image features” as the query Q , the “traffic flow time series characteristics” of the corresponding period is the key K ,value V , calculate the correlation between the congestion level in the image and the actual traffic data:

[0123] ;

[0124] in, h image is the image feature matrix, h time is the time series feature matrix, W q To learn the interaction weights between images and time series data.

[0125] Real-time collection of CPU utilization U , memory usage M , converted into an adjustment coefficient through the formula:

[0126] ;

[0127] when α When it is >0.8, the attention weight of high-computation relationships such as "image-text regulations" is reduced, and real-time critical relationships such as "image-time series traffic" are prioritized.

[0128] The specific process of step S33 is as follows:

[0129] S331: Convert the input features into a feature tensor, and evenly split the feature tensor into h sub-tensors according to the channel dimension. Each sub-tensor corresponds to an independent attention head.

[0130] S332: Assign different key-value pair dimensions to each attention head. Small dimensions capture fine-grained associations, while large dimensions capture coarse-grained associations.

[0131] S333: Apply different types of positional encodings to attention heads of different scales to enhance the perception of local and global relationships;

[0132] S334: For each modal feature, calculate its correlation strength matrix.

[0133] Fine-grained headers (local associations): Focus on local features within a single modality, such as the temporal data association between "brake light status" and "sudden speed drop" in a vehicle image, or the violation matching between "speed limit 60 km / h" in regulatory text and "current speed 65 km / h" in structured data. Coarse-grained headers (global associations): Model cross-modal global features, such as the comprehensive association between congestion image features at an entire intersection, time-series data on road network traffic flow, and the "no parking during peak hours" clause in regulatory text.

[0134] In step S34, the attention weights between the two modal features are calculated, the correlation strength matrix is ​​calculated based on the attention weights, and the modal features are weighted fused based on the correlation strength matrix. At the same time, the residual structure is introduced to retain the original feature information. The specific process is as follows:

[0135] S341: Each modal feature is linearly transformed to obtain a query vector, a key vector, and a value vector, and the attention score is calculated. The attention weight between each modal feature is obtained through the softmax function.

[0136] S342: Constructing a correlation strength matrix based on the attention weights between the two modal features, where each element in the matrix represents the attention weight of modality i to modality j;

[0137] S343: According to the correlation strength matrix M, the features of each modality are weighted and fused. The fused features are globally average pooled to obtain channel descriptors. The channel weights are calculated through two fully connected layers and an activation function. The channel weights are multiplied by the fused features channel by channel to obtain enhanced fused features.

[0138] This attention-based cross-modal feature extraction method intelligently matches server resources with data-related feature extraction through time-varying weight adjustment and dynamic topology. Experiments have shown that, compared to traditional static attention mechanisms, this method can effectively improve the F1 score and reduce model inference latency in multimodal sentiment analysis tasks.

[0139] Example 4

[0140] On the basis of Example 1 or Example 2 or Example 3, the specific process of the adaptive data processing module in step S4 matching the mixed data to be processed with the corresponding deep learning network model based on the fused feature vector is as follows:

[0141] S41: Establishing the mapping relationship between model and task, the adaptive data processing module analyzes the fused feature vector and calculates the variance, entropy and cross-correlation of the fused feature vector. h final , containing the associated information of image, time series, structured, and text modalities. Variance: measures the degree of fluctuation of the feature vector and reflects the complexity of the data. Example: If the variance of the "vehicle movement direction" dimension in the image feature is large, it means that the vehicle driving directions in the scene are diverse (such as an intersection), which may correspond to the "congestion prediction" or "traffic violation detection" tasks. Entropy: evaluates the uncertainty of feature distribution. Example: If the entropy value of text features is high (such as containing multiple regulatory clauses), it may correspond to the "regulatory intelligent question answering" task. Cross-correlation: calculates the correlation coefficient of features between modalities and identifies the dominant mode. Example: The correlation coefficient between image features and time series features r =0.9, indicating a strong spatiotemporal correlation, which may belong to the “congestion prediction” or “traffic incident detection” tasks.

[0142] S42: Determine the task type of the mixed data to be processed based on the analysis results, and select the preliminary network model based on the task type and the model-task mapping relationship. r =0.9, and the variance is higher than the threshold (e.g., large fluctuations in traffic volume), it is determined to be a congestion prediction task. r =0.8 (such as the violation image matching regulations), it is determined to be a violation detection task.

[0143] S43: When there are records of the same type of tasks in the historical dynamic optimization table, the matching score of the preliminary calculation model is calculated based on the reinforcement learning algorithm. Q ( s , a ), adjust the network model according to the matching score, matching score Q ( s , a ) is calculated as follows:

[0144] Q ( s , a )=(1− α ) Q ( s ′, a ′)+α ( r + γ max a′ Q ( s ′, a ′));

[0145] in, s is the current state, i.e. the current data type + task type + load index, a Select an action for the current state model, s ′ is the next state, a 'The next state may be an action, r is the reward value, α is the learning rate, max a′ Q ( s ′, a ′) is the next state s All possible actions under a The maximum Q value of ′ represents the agent’s estimate of the optimal decision for the future. Q ( s ′, a ′) is the next state Q value;

[0146] S44: Extract task priority labels from the fused features. High-priority data directly enters the hot queue of each network model instance pool, while medium and low-priority data enter the ordinary queue. Each network model processes data based on the queue.

[0147] The specific process of step S5 is as follows:

[0148] S51: Embed a monitoring module in the data processing process to collect real-time data processing volume, processing time, actual resources used, total available resources, correct prediction data, and total number of samples;

[0149] S52: Calculate the processing efficiency, resource usage, and processing accuracy during data processing. The calculation formula for processing efficiency is as follows:

[0150] Processing efficiency = amount of data processed / processing time;

[0151] The resource utilization rate is calculated as follows:

[0152] Resource utilization rate = actual used resources / total available resources × 100%;

[0153] Processing accuracy = correct prediction data / total number of samples;

[0154] S53: Aggregate indicators by fixed time window to generate state vector s t= [processing efficiency t, resource occupancy t, processing accuracy t], set the action space, and the action space matches the optimization direction of the model;

[0155] S54: Co-optimization of the original loss function based on the deep learning model and the reinforcement learning reward function,

[0156] Joint losses: L = L task ( θ )− β ⋅ r t ;

[0157] in, β To balance the coefficient, guide the model to optimize towards high rewards, L task is the task loss, which represents the loss function value of the deep learning model on a specific task, and represents the loss function value of the deep learning model on a specific task. θ represents the learnable parameters of the model, and the task loss is θ changes with the changes of r t is the reward signal, which comes from the environment feedback in reinforcement learning.

[0158] A server data processing system based on deep learning is used to implement the server data processing method based on deep learning, including a data acquisition module, a preprocessing module, a time-varying neural network topology structure, a feature extraction module, a feature fusion module, an adaptive data processing module, and a reinforcement learning module. The data acquisition module is connected to the preprocessing module, the preprocessing module is connected to the time-varying neural network topology structure, the feature extraction module is arranged in the time-varying neural network topology structure, the feature extraction module is connected to the feature fusion module, the feature fusion module is connected to the adaptive data processing module, and the adaptive data processing module is connected to the reinforcement learning module.

[0159] The data acquisition module is used to acquire mixed data to be processed. The preprocessing module is used to preprocess the mixed data to be processed, dividing the preprocessed mixed data into four subsets: structured data, image data, text data, and time series data. The time-varying neural network topology structure is used to input the four subsets into corresponding branches of the time-varying neural network topology structure for preliminary feature extraction. The time-varying neural network topology structure dynamically adjusts the network edge connection weights based on the input data, capturing the correlation features between different modal data features to obtain cross-modal correlation features. The feature fusion module is used to perform fusion processing based on the cross-modal correlation features to obtain a fused feature vector. The adaptive data processing module is configured to set multiple deep learning network models and match the mixed data to corresponding deep learning network models based on the fused feature vectors for processing. The reinforcement learning module is used to calculate the processing efficiency, resource utilization, and processing accuracy during the data processing process, establish a reward function for the reinforcement learning algorithm based on the processing efficiency, resource utilization, and processing accuracy, and jointly iteratively optimize the deep learning network models based on the reward function.

[0160] In summary, the deep learning-based server data processing method and system provided by the present invention utilizes a dual dynamic adjustment mechanism based on server load and data features to optimize the edge connection weights of the graph neural network in real time for multimodal feature fusion paths. This dynamic, adaptive data processing architecture integrates feature vector analytic variance, entropy, and cross-correlation coefficients with reinforcement learning scheduling to complete model selection in a very short time, improving efficiency compared to manual parameter adjustment. Furthermore, the combination of priority queues and a flexible resource scheduling mechanism significantly reduces latency in real-time data processing.

[0161] By employing targeted extraction methods for different modal data, feature extraction accuracy is effectively improved compared to general extraction methods. By combining server load and data characteristics, network edge connection weights are adjusted in real time. When the server is highly loaded, feature fusion paths are automatically optimized to reduce processing latency. Weights are also adjusted based on data complexity, enabling more accurate feature extraction of complex data and improving model generalization. By integrating server load information with modal correlation, key related features are prioritized when resources are limited, avoiding information loss due to insufficient resources and improving the integrity of extracted important features.

[0162] Based on the weighted fusion of the correlation strength matrix and the introduction of a residual structure, it fully integrates cross-modal information while retaining the original feature details, thereby improving the information integrity of the fused feature vector and effectively enhancing the accuracy of subsequent processing. By analyzing the fused feature vector to determine the task type and combining the model-task mapping relationship to select the model, compared with fixed model configurations, the task-model matching accuracy and processing efficiency are improved. Reinforcement learning is used to calculate the matching score and adjust the model, so that the model selection strategy is continuously optimized. After iteration, the average processing efficiency of the model is improved and resource waste is reduced. The reward function is constructed based on processing efficiency, resource utilization, and processing accuracy. Through collaborative optimization of the joint loss function, a balance is achieved among the three. Compared with single-objective optimization, the overall resource utilization of the server is improved.

Claims

1. A server data processing method based on deep learning, characterized in that: The following steps are involved: S1: The server obtains mixed data to be processed, preprocesses the mixed data to be processed, and divides the preprocessed mixed data to be processed into four subsets: structured data, image data, text data, and time series data; S2: constructing a time-varying neural network topology structure, inputting the four subsets into corresponding branches in the time-varying neural network topology structure for preliminary feature extraction, and dynamically adjusting the network edge connection weights according to the input data; S3: Capture the correlation features between different modal data features to obtain cross-modal correlation features, and perform fusion processing based on the cross-modal correlation features to obtain a fused feature vector; S4: constructing an adaptive data processing module, setting multiple deep learning network models in the adaptive data processing module, and the adaptive data processing module matching the mixed data to be processed with the corresponding deep learning network model based on the fusion feature vector for processing; S5: Calculate the processing efficiency, resource utilization, and processing accuracy during data processing, establish the reward function of the reinforcement learning algorithm based on the processing efficiency, resource utilization, and processing accuracy, and perform joint iterative optimization of the deep learning network model based on the reward function; S6: Based on the deep learning network model after joint iteration, the real-time mixed data to be processed is processed to obtain multimodal data processing results.

2. A server data processing method based on deep learning according to claim 1, characterized in that: The specific process of step S1 is as follows: S11: The server receives mixed data to be processed through a network interface or a storage device interface, performs preliminary verification on the integrity and format of the data, and verifies whether the data file is damaged or whether data fields are missing. If errors or incompleteness are found in the data, an error prompt is triggered and reception is stopped, requiring retransmission of the data. S12: performing data cleaning and format standardization on the mixed data to be processed, removing noise data, filling missing values, converting different types of data into a format suitable for processing by a deep learning network model, and adding type labels; S13: Divide the preprocessed mixed data into four subsets based on type labels: structured data, image data, text data, and time series data.

3. The server data processing method based on deep learning according to claim 1, characterized in that: The specific process of step S2 is as follows: S21: Map structured data subsets into high-dimensional feature vectors through an embedding layer, use a deformable convolutional feature extractor to extract multi-scale features from images, focus on key areas through a spatial attention mechanism to output a feature map sequence, adjust the mask ratio based on the text length and semantic complexity, capture long-range dependencies in the text through a multi-head self-attention mechanism, use the attention mechanism LSTM unit to process time series data, and dynamically allocate attention to different time steps through attention weights to capture long-term and short-term patterns in time series data; S22: The modal features of different dimensions output by the four subset branches are used as the initial node set of the deep learning network model, where each node represents the feature representation of a specific modality and the initial edge weight matrix; S23: Collects server hardware metrics and software metrics such as data processing delay and model inference speed in real time, and converts multi-dimensional load metrics into weight adjustment coefficients through a nonlinear mapping function. S24: Calculate the statistical characteristics of the input data of each branch, including variance, entropy, and KL divergence of feature distribution, evaluate the complexity and uncertainty of the data, calculate the correlation between different modalities through the cross-modal attention mechanism, and generate the attention weight matrix; S25: Fusion of data-driven attention weights with server load-adjusted weights; S26: Set a sliding time window and update the edge weight matrix at each time step. When new data arrives, use incremental learning to update the edge weights. S27: When the edge weight change exceeds the threshold, the topology structure is reorganized, adding or deleting node connections to form a new subnetwork structure.

4. The server data processing method based on deep learning according to claim 1, characterized in that: The specific process of step S3 is as follows: S31: Project modal features of different dimensions into a shared latent space, achieve dimension alignment through linear transformation, and enhance feature correlation within each modal feature; S32: Capture the associations between different modal features through a query-key-value pair mechanism, map server CPU utilization and memory occupancy indicators to attention weight adjustment coefficients, and integrate server load information with the correlation between modal features of different dimensions; S33: Design attention heads of different scales to capture fine-grained to coarse-grained correlation features, construct inter-modal correlation strength matrices, and quantify the mutual influence of different modalities; S34: Calculate the attention weights between the two modal features, construct the correlation strength matrix according to the attention weights, perform weighted fusion of the modal features based on the correlation strength matrix, and introduce a residual structure to retain the original feature information.

5. The server data processing method based on deep learning according to claim 4, characterized in that: The specific process of step S33 is as follows: S331: Convert the input features into feature tensors, and evenly split the feature tensors into a specified number of sub-tensors according to the channel dimension. Each sub-tensor corresponds to an independent attention head. S332: Assign different key-value pair dimensions to each attention head. Small dimensions capture fine-grained associations, while large dimensions capture coarse-grained associations. S333: Apply different types of positional encodings to attention heads of different scales to enhance the perception of local and global relationships; S334: For each modal feature, calculate its correlation strength matrix.

6. The server data processing method based on deep learning according to claim 4, characterized in that: In step S34, the attention weights between the two modal features are calculated, the correlation strength matrix is ​​calculated based on the attention weights, and the modal features are weighted fused based on the correlation strength matrix. At the same time, the residual structure is introduced to retain the original feature information. The specific process is as follows: S341: Each modal feature is linearly transformed to obtain a query vector, a key vector, and a value vector, and the attention score is calculated. The attention weight between each modal feature is obtained through the softmax function. S342: Construct an association strength matrix based on the attention weights between the two modal features. Each element in the matrix represents the modality. i Modal j The attention weight of S343: According to the correlation strength matrix M, the features of each modality are weighted and fused. The fused features are globally average pooled to obtain channel descriptors. The channel weights are calculated through two fully connected layers and an activation function. The channel weights are multiplied by the fused features channel by channel to obtain enhanced fused features.

7. The server data processing method based on deep learning according to claim 1, characterized in that: The specific process of the adaptive data processing module in step S4 matching the mixed data to be processed with the corresponding deep learning network model based on the fused feature vector is as follows: S41: establishing a model-task mapping relationship, and the adaptive data processing module analyzes the fused feature vector and calculates the variance, entropy and cross-correlation of the fused feature vector; S42: Determine the task type of the mixed data to be processed based on the analysis results, and select a preliminary network model based on the task type and the model-task mapping relationship; S43: When there are task records of the same type in the historical dynamic optimization table, a matching score of the preliminary calculation model is calculated based on the reinforcement learning algorithm, and the network model is adjusted according to the matching score; S44: Extract task priority labels from the fused features. High-priority data directly enters the hot queue of each network model instance pool, while medium and low-priority data enter the ordinary queue. Each network model processes data based on the queue.

8. The server data processing method based on deep learning according to claim 1, characterized in that: The specific process of step S5 is as follows: S51: Embed a monitoring module in the data processing process to collect real-time data processing volume, processing time, actual resources used, total available resources, correct prediction data, and total number of samples; S52: Calculate processing efficiency, resource occupancy, processing accuracy, and processing efficiency during data processing; S53: Aggregate indicators according to a fixed time window, generate a state vector, and set an action space, where the action space matches the optimization direction of the model; S54: Co-optimization of the original loss function based on the deep learning model and the reinforcement learning reward function, Joint losses: L = L task ( θ )− β ⋅ r t ; in, β To balance the coefficient, guide the model to optimize towards high rewards, L task is the task loss, which represents the loss function value of the deep learning model on a specific task. θ represents the learnable parameters of the model, and the task loss is θ changes with the changes of r t is the reward signal, which comes from the environment feedback in reinforcement learning.

9. A server data processing system based on deep learning, used to implement a server data processing method based on deep learning according to any one of claims 1 to 8, characterized in that: It includes a data acquisition module, a preprocessing module, a time-varying neural network topology structure, a feature extraction module, a feature fusion module, an adaptive data processing module, and a reinforcement learning module. The data acquisition module is connected to the preprocessing module, the preprocessing module is connected to the time-varying neural network topology structure, the feature extraction module is arranged in the time-varying neural network topology structure, the feature extraction module is connected to the feature fusion module, the feature fusion module is connected to the adaptive data processing module, and the adaptive data processing module is connected to the reinforcement learning module. The data acquisition module is used to acquire mixed data to be processed; The preprocessing module is used to preprocess the mixed data to be processed, and divide the preprocessed mixed data to be processed into four subsets: structured data, image data, text data and time series data; The time-varying neural network topology structure is used to input the four subsets into corresponding branches in the time-varying neural network topology structure for preliminary feature extraction. The time-varying neural network topology structure dynamically adjusts the network edge connection weights according to the input data to capture the correlation features between the features of different modal data to obtain cross-modal correlation features; The feature fusion module is used to perform fusion processing based on cross-modal correlation features to obtain a fused feature vector; The adaptive data processing module sets a plurality of deep learning network models, and matches the mixed data to be processed with the corresponding deep learning network models based on the fusion feature vector for processing; The reinforcement learning module is used to calculate the processing efficiency, resource utilization, and processing accuracy during the data processing process, establish a reward function for the reinforcement learning algorithm based on the processing efficiency, resource utilization, and processing accuracy, and perform joint iterative optimization on the deep learning network model based on the reward function.

Images