Method and system for automatically classifying mass data of data platform based on large model
By using a large-model-based data platform, a unified cross-modal representation is generated and information features are optimized. Combined with platform context information, a classification path selection strategy is generated, which solves the problems of information sharing and resource waste in multimodal data processing and achieves efficient and stable classification task execution.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ZHONGAN ZHISHANG (BEIJING) DIGITAL TECHNOLOGY CO LTD
- Filing Date
- 2026-03-04
- Publication Date
- 2026-05-19
AI Technical Summary
Existing technologies cannot effectively integrate multimodal data from different sensors, such as image data, audio data, and sensor data, resulting in the inability to effectively share and utilize information, which affects the accuracy and efficiency of classification and recognition.
A large-model-based data platform is adopted to generate a unified cross-modal representation through a cross-modal encoder, remove redundant information and retain task-related information, combine platform context information to generate a classification path selection strategy, and perform cost and risk prediction to dynamically optimize the classification path selection.
It achieves effective integration of multimodal data, improves the accuracy and efficiency of classification tasks, avoids resource waste and risk exceeding limits, and ensures the stability and adaptability of the system.
Smart Images
Figure CN122065259A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of intelligent data processing technology, and in particular to a method and system for automatic classification of massive data based on a large model data platform. Background Technology
[0002] Current multimodal data processing technologies face limitations in processing single-modal data and cannot effectively integrate data from different sensors, such as image, audio, and sensor data in video surveillance systems. Traditional techniques typically process each modality of data separately, resulting in ineffective information sharing and utilization, thus limiting the accuracy and efficiency of classification and recognition.
[0003] Currently, traditional technologies mostly rely on the limitations of single-modal data processing. Since a single modality cannot fully describe complex situations, it is easy to cause information loss. Summary of the Invention
[0004] To overcome the above shortcomings, this invention provides a method and system for automatic classification of massive data based on a large model data platform. It aims to improve the limitations of traditional technologies that mostly use single-modality data processing. Since a single modality cannot fully describe complex situations, it is easy to cause information loss.
[0005] In a first aspect, the present invention provides the following technical solution: an automatic classification method for massive data based on a large model data platform, comprising the following steps: S1. Receive multimodal data input, wherein the multimodal data includes text data, image data, audio data, and video data; S2. The multimodal data is processed by a cross-modal encoder to obtain a unified cross-modal representation; S3. Based on the cross-modal representation, generate cross-modal feature representations using an information optimization method, removing redundant information and retaining task-related information; S4. Input the cross-modal feature representation and platform context information into the policy network to generate a classification path selection policy. The platform context information includes computing resource status, task complexity, and latency constraints. S5. Based on the classification path selection strategy, select and execute the classification path to classify the multimodal data, and predict costs and risks during the execution process; S6. Based on the results of the cost and risk prediction, update the strategy network and constraints to achieve continuous optimization of the classification path.
[0006] By adopting the above technical solution, multimodal data input is received and a unified cross-modal representation is generated, thereby realizing the effective integration of different modal data. This improves the limitation of traditional technologies that mostly use single-modal data processing, which is prone to information loss because a single modality cannot fully describe complex situations.
[0007] Preferably, the cross-modal encoder includes: Different modal data are input into the corresponding modal encoders to generate representations of each modality; The modal representations are input into the cross-modal alignment layer; Based on the output of the cross-modal alignment layer, a unified cross-modal representation is obtained.
[0008] Preferably, the cross-modal feature representation includes: Receive the unified cross-modal representation; Remove redundant features from the cross-modal representation that are irrelevant to the classification task; Retain and extract feature information relevant to the classification task to generate an optimized cross-modal feature representation.
[0009] Preferably, the classification path selection strategy includes: Receive cross-modal feature representations and platform context information; Joint modeling of cross-modal feature representations with platform context information; Based on the results of the joint modeling, the candidate classification paths are scored; Generate the selection probability distribution of each candidate classification path and determine the classification path selection strategy.
[0010] Preferably, the multimodal data classification includes: Receive the classification path selection strategy and determine the classification path to be executed; The execution cost of the classification path is estimated, including latency and computational resource consumption; The tail risk of the classification path is predicted, including the risk of classification error or the risk of exceeding the time limit. Preferably, the cost and risk prediction includes: Based on the predicted execution costs and tail risks, the classification path is optimized and selected. Determine the classification path that satisfies the preset resource constraints and risk constraints, and execute the classification path.
[0011] Preferably, the policy network includes: Receive the execution results of the classification path; The parameters of the policy network are updated based on the execution results; Adjust the constraints based on the execution results to ensure that subsequent path selection meets resource and risk constraints.
[0012] Preferably, the constraints include: The parameter updates based on the policy network employ an online update mechanism. When new task data is input, the policy network is adaptively adjusted based on historical feedback information.
[0013] Preferably, the continuous optimization includes: Receive feedback information from the classification task, including classification accuracy, execution path latency, computational resources consumed, and tail risk; The classification path selection strategy is dynamically optimized based on the feedback information.
[0014] Secondly, this invention provides the following technical solution: a massive data automatic classification system based on a large model data platform, comprising the following modules: The data acquisition module is used to receive multimodal data input, including text data, image data, audio data, and video data; A cross-modal encoder module is used to process the multimodal data through a cross-modal encoder to obtain a unified cross-modal representation; The information optimization module is used for the cross-modal representation. It generates cross-modal feature representations through information optimization methods, removes redundant information, and retains information relevant to the task. The policy network module is used to input the cross-modal feature representation and platform context information into the policy network to generate a classification path selection policy. The platform context information includes computing resource status, task complexity, and latency constraints. The path execution module is used for the classification path selection strategy, selecting and executing classification paths, classifying the multimodal data, and predicting costs and risks during the execution process. The optimization and update module is used to update the policy network and constraints based on the results of the cost and risk predictions, so as to achieve continuous optimization of the classification path.
[0015] The present invention has the following beneficial effects: 1. In this invention, by receiving multimodal data input and generating a unified cross-modal representation, the effective integration of different modal data is achieved, thereby improving the limitation of traditional technologies that mostly use single-modal data processing. Since a single modality cannot fully describe complex situations, it is easy to cause information loss.
[0016] 2. In this invention, by generating cross-modal representations and optimizing information features, redundant information is removed and key features related to the task are retained, thereby improving the problem that most traditional methods fail to effectively remove irrelevant features. Since redundant features affect system efficiency and increase computational burden, they result in slow processing speed and high resource consumption.
[0017] 3. In this invention, a classification path selection strategy is generated based on platform context information, thereby optimizing path selection and ensuring the execution of the optimal path under resource and risk constraints. This improves upon the problem that most traditional methods fail to dynamically adjust path selection to adapt to different environments. Fixed path selection can lead to resource waste or task timeouts, affecting the system's execution efficiency and stability.
[0018] 4. In this invention, by realizing the dynamic optimization of the classification path selection strategy, and then adjusting the path selection strategy according to the feedback information, the execution effect of subsequent tasks is guaranteed, thereby improving the problem that traditional methods fail to update and adjust strategies in real time. Due to the lack of dynamic optimization, the system is prone to static decision-making, which affects the long-term performance and adaptability of the task. Attached Figure Description
[0019] Figure 1 This is a flowchart of the automatic classification method for massive data based on a large model data platform proposed in this invention. Figure 2 This is a flowchart of the cross-modal encoder module of the automatic classification method for massive data based on a large model data platform proposed in this invention; Figure 3 This is a flowchart of the strategy network module of the automatic classification method for massive data on a data platform based on a large model proposed in this invention. Figure 4 This is a flowchart of the classification path execution module of the automatic classification method for massive data on a data platform based on a large model proposed in this invention; Figure 5 This is a system architecture diagram of the massive data automatic classification system based on a large model proposed in this invention. Detailed Implementation
[0020] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0021] Example 1: In the first embodiment of the present invention, the present invention provides an automatic classification method for massive data based on a large model data platform, such as... Figures 1-4 As shown, it includes the following steps: S1. Receive multimodal data input, wherein the multimodal data includes text data, image data, audio data, and video data; Specifically, in the formal representation of multimodal data, text data is represented as a set of vectors. Each of them The data consists of initially encoded word or sentence vectors; image data is represented as a set of pixel matrices. Each of them The pixel features of an image are represented; audio data is represented as a set of feature sequences. Each of them This represents the features of an audio segment after time-frequency domain processing; video data is represented as a set of frame sequences. Each of them These are video frame features.
[0022] A unified input interface is used to receive the four types of modal data mentioned above and store them in their respective data buffers. The process is as follows: ;in, It represents the complete set of multimodal data inputs, including all modalities.
[0023] The result of step S1 is to output a data set that can be used by the cross-modal encoder. Its structured form is: ;in, For a single element of multimodal data, retain the original modality labeling information for subsequent differentiation steps.
[0024] Input: Raw data in four categories: text, images, audio, and video. Processing: Receive and format the data accordingly, forming... , , , Four sets. Output: Unify the four sets into... , as the input of the cross-modal encoder.
[0025] This ensures that data from different modalities are effectively received and distinguished when entering the system; forms a structured set of input data, laying the foundation for cross-modal alignment and unified representation; and avoids incompatibility issues in subsequent steps due to modal differences.
[0026] S2. The multimodal data is processed by a cross-modal encoder to obtain a unified cross-modal representation; Furthermore, the cross-modal encoder includes: Different modal data are input into the corresponding modal encoders to generate representations of each modality; The modal representations are input into the cross-modal alignment layer; Based on the output of the cross-modal alignment layer, a unified cross-modal representation is obtained.
[0027] Specifically, a dedicated modal encoder processes the data for each modality, and these processed representations are then fused through a cross-modal alignment layer to generate a unified representation for subsequent tasks. This operation effectively integrates information from different modalities, avoids processing difficulties caused by differences in data modalities, and enables the system to process and analyze various types of input data.
[0028] Modal encoder processing, the input multimodal data is: ;in For text data, For image data, For audio data, This is video data. The data for each modality is processed by its dedicated encoder to form a corresponding representation: ;in It is a dedicated encoder for each modality, whose function is to transform the data of each modality into a low-dimensional, dense feature representation. The representation of each modality is as follows: .
[0029] After obtaining the representations of each modality, the next step in the cross-modal alignment layer is to align these representations of different modalities to the same feature space. Let the alignment result output by the cross-modal alignment layer be: ;in It is a cross-modal alignment function that adjusts the features of different modalities through an alignment mechanism, enabling features from different modalities to share information in a unified representation space. In this way, features from different modalities are mapped to a unified space, eliminating differences between modalities and allowing them to interoperate in subsequent processing.
[0030] Generate a unified cross-modal representation, the result of which is processed by a cross-modal alignment layer. This is the required unified cross-modal representation. This unified representation contains key information from different modalities and can be used for subsequent tasks such as classification, retrieval, and generation. The output unified cross-modal representation. It can be used as input for subsequent modules, ensuring that information from different modalities is integrated into a valid representation for further use by the model.
[0031] enter: The data consists of text, image, audio, and video data. Each modal data type is encoded using a dedicated encoder to generate a modal representation. Processing: The representations of each modality are input separately to a cross-modal alignment layer for modal alignment, generating a unified cross-modal representation. Output: A unified cross-modal representation is obtained. This representation can be used as input for subsequent tasks.
[0032] S3. Based on the cross-modal representation, generate cross-modal feature representations using an information optimization method, removing redundant information and retaining task-related information; Furthermore, the cross-modal feature representation includes: Receive the unified cross-modal representation; Remove redundant features from the cross-modal representation that are irrelevant to the classification task; Retain and extract feature information relevant to the classification task to generate an optimized cross-modal feature representation.
[0033] Specifically, information optimization methods are used to process cross-modal representations, removing redundant information and retaining features useful for classification tasks, thereby improving the performance of subsequent classification or tasks. This operation ensures the conciseness and effectiveness of information, allowing subsequent classification tasks to focus on processing key information relevant to the target task, thus improving classification accuracy and efficiency.
[0034] Receive the unified cross-modal representation. Input: Receive the unified cross-modal representation generated in the previous step. This representation is a high-dimensional feature representation obtained by cross-modal encoding and alignment of input data from different modalities. The formula is: ;in These represent text, image, audio, and video data, respectively. It is a function for cross-modal alignment layers, and its output is... To unify cross-modal representation.
[0035] Redundant features are removed to obtain a unified cross-modal representation. Then, information optimization methods are used to process it, removing redundant information, i.e., unnecessary features irrelevant to the classification task. The formula is: ;in It is an information optimization function that uses algorithms to remove redundant and irrelevant features to obtain an optimized representation. .
[0036] Retain task-relevant features, especially key information needed for classification tasks. For text classification, retain semantically relevant text features; for image classification, retain important visual features in the image. The formula is: ;in It is a feature extraction function that extracts and retains information that is helpful for the classification task, generating an optimized cross-modal feature representation. .
[0037] Generate optimized cross-modal feature representations. This representation incorporates key information from different modalities and removes redundant and irrelevant features. Output: The generated optimized cross-modal feature representation. It can be used as input data for subsequent tasks.
[0038] Input: Receives a unified cross-modal representation This representation is an integrated result of multi-source data (text, image, audio, video) from different modalities, which has undergone cross-modal encoding and alignment. Processing: Redundant features are removed and features relevant to the classification task are retained through information optimization methods. Optimization algorithms (such as information bottlenecks and feature selection) are used during processing to ensure that task-irrelevant parts are removed and useful information is retained. Output: Generates an optimized cross-modal feature representation. This indicates that redundant information has been removed and that the key features required for the classification task have been included, which can be used as input for subsequent classification tasks.
[0039] Information Conciseness: By removing irrelevant features, processing burden and computational complexity are reduced, ensuring model processing efficiency. Task Relevance: Key features required for the classification task are retained, ensuring the accuracy and effectiveness of subsequent tasks. Efficient Processing: Optimized cross-modal representations can improve classification performance and ensure that information is not lost due to redundancy or noise in multimodal data processing.
[0040] S4. Input the cross-modal feature representation and platform context information into the policy network to generate a classification path selection policy. The platform context information includes computing resource status, task complexity, and latency constraints. Furthermore, the classification path selection strategy includes: Receive cross-modal feature representations and platform context information; Joint modeling of cross-modal feature representations with platform context information; Based on the results of the joint modeling, the candidate classification paths are scored; Generate the selection probability distribution of each candidate classification path and determine the classification path selection strategy.
[0041] Specifically, a classification path selection strategy is generated by jointly modeling cross-modal feature representations and platform context information within a policy network. This strategy not only considers multimodal information but also dynamically adjusts path selection based on platform state and task characteristics, thereby optimizing the execution of the classification task.
[0042] Receive cross-modal feature representation and platform context information. Input: Receive cross-modal feature representation generated in the previous step. and platform context information. Platform context information This includes computing resource status, task complexity, and latency constraints. The formula is: ;in For specific contextual information, This is the optimized cross-modal representation generated in the previous step.
[0043] Joint modeling, by jointly modeling cross-modal feature representations and platform context information, yields a joint feature representation. This indicates that the key information from multimodal data and the actual state of the platform are integrated. The formula is: ;in This is a joint modeling function responsible for fusing cross-modal feature representations and platform context information into a new feature representation. This is to facilitate subsequent route selection.
[0044] The candidate classification paths are scored based on the joint feature representation. A score is calculated for each candidate classification path to determine the priority of each path. The formula is: ;in It is a scoring function. Indicates the first The score of each candidate path, As candidate paths, It is a joint feature representation of the input.
[0045] Generate a probability distribution for the selection of classification paths, based on the calculated score of each candidate path. The softmax function is used to generate the probability distribution of path selection, representing the probability that each path will be selected. The formula is: ;in For the first The probability of selecting each candidate path Score the path. It represents the total number of candidate paths.
[0046] The classification path selection strategy is determined, and finally the path selection probability is used. Determine the classification path selection strategy and select the path with the highest probability for execution. The formula is: ;in For the final classification path, select the path with the highest selection probability.
[0047] Input: Receive cross-modal feature representation and platform context information This information provides complete input data for subsequent path selection. Processing: Through joint modeling, cross-modal feature representations and platform context information are fused into a new feature representation. Then, the score for each candidate classification path is calculated, and a probability distribution for path selection is generated. Output: Based on the path selection probability distribution, the final classification path selection strategy is determined, that is, the path with the highest probability is selected. Implement the classification.
[0048] Multimodal information integration: Data from different modalities and the actual state of the platform are integrated into a unified representation. This avoids classification path selection errors caused by modal differences or platform constraints. The path selection is dynamic: it dynamically adjusts the classification path selection based on platform status and task characteristics, ensuring that each selected path best meets the current task requirements and platform capabilities. The classification path is optimized: through a scoring and probability distribution mechanism, the system can rationally select the optimal path based on the scores of different candidate paths, improving the efficiency and accuracy of the classification task.
[0049] S5. Based on the classification path selection strategy, select and execute the classification path to classify the multimodal data, and predict costs and risks during the execution process; Furthermore, the multimodal data classification includes: Receive the classification path selection strategy and determine the classification path to be executed; The execution cost of the classification path is estimated, including latency and computational resource consumption; The tail risk of the classification path is predicted, including the risk of classification error or the risk of exceeding the time limit.
[0050] The cost and risk forecasts include: Based on the predicted execution costs and tail risks, the classification path is optimized and selected. Determine the classification path that satisfies the preset resource constraints and risk constraints, and execute the classification path.
[0051] Specifically, by predicting the execution cost and tail risk of the classification path, the system can select the optimal path for classification execution, thereby ensuring the accuracy and efficiency of multimodal data classification and avoiding resource waste and risk exceeding limits during the execution process.
[0052] Receive the classification path selection strategy and determine the classification path to be executed. Input: Receive the classification path selection strategy, which is based on the path selection result generated in step S4 above. The formula is: ;in It's the path selection probability; the system chooses the path with the highest probability. The process involves categorizing the data. This step determines the path to be executed based on the strategies developed in the previous steps.
[0053] Execution cost estimation estimates the execution cost of the selected classification path, primarily including latency and computational resource consumption. The formula is: ,in Indicates execution cost, It is a function for calculating execution cost. The selected path. Delay and computational resource consumption It can be estimated using the following formula: ;in and These represent estimation functions for latency and resource consumption, respectively.
[0054] Tail risk prediction forecasts the tail risk of the selected path, primarily including classification misclassification risk and latency exceeding limits. The formula is: ;in It is tail risk. It is a risk prediction function. The selected path. Risk of classification error. and the risk of excessive latency Prediction can be made using the following formula: ;in and These represent the risk prediction functions for classification errors and excessive latency, respectively.
[0055] Path optimization selection based on cost and risk prediction, according to execution cost and tail risk Based on the prediction results, optimize path selection. The formula is: ;in It is a weighting factor used to balance cost and risk. The classification path selected for final optimization.
[0056] Determine the classification paths that satisfy resource and risk constraints, and select the optimal classification path that meets these constraints based on these constraints. The formula is: ;in and These are the maximum resource and maximum risk constraints, respectively. This step ensures that the final selected path is within the resource and risk constraints.
[0057] Execute the classification path, and finally select the classification path. Then, the actual classification task is performed. The output of this step is the final classification result. : ;in The output of the classification task.
[0058] Input: The input consists of the classification path selection strategy, and estimates of execution cost and tail risk. Process: Based on the path selection strategy, determine the path to be executed and estimate its execution cost and tail risk. Then, based on cost and risk predictions, perform path optimization to ensure the final selected path meets resource and risk constraints. Output: Output the final selected classification path. And execute the path to classify the data.
[0059] Efficient Path Selection: By predicting execution costs and risks, the optimal classification path is selected, avoiding resource waste and excessive risk. Stable Task Execution: By predicting and adjusting tail risks, the system can avoid classification errors and excessive latency, improving the reliability of task execution. Resource and Risk Control: By meeting preset resource and risk constraints, the system ensures that the execution of classification tasks meets the platform's performance requirements, guaranteeing the accuracy and stability of classification results.
[0060] S6. Based on the results of the cost and risk prediction, update the strategy network and constraints to achieve continuous optimization of the classification path.
[0061] Furthermore, the policy network includes: Receive the execution results of the classification path; The parameters of the policy network are updated based on the execution results; Adjust the constraints based on the execution results to ensure that subsequent path selection meets resource and risk constraints.
[0062] The constraints include: The parameter updates based on the policy network employ an online update mechanism. When new task data is input, the policy network is adaptively adjusted based on historical feedback information.
[0063] The continuous optimization includes: Receive feedback information from the classification task, including classification accuracy, execution path latency, computational resources consumed, and tail risk; The classification path selection strategy is dynamically optimized based on the feedback information.
[0064] Specifically, by updating the policy network and adjusting the constraints, the classification path is continuously optimized, ensuring that the system adapts itself based on feedback from actual execution, thereby improving the accuracy, stability, and efficiency of the classification task.
[0065] Receive the execution results of the classification path. Input: Receive the results after the classification path is executed, including classification accuracy, execution path latency, resource consumption, and tail risk. The formula is: ;in It is the classification accuracy. It is the delay of the execution path. It's the computing resources consumed. It is a tail risk.
[0066] Update the parameters of the policy network. Based on the classification path execution results, update the parameters in the policy network, and adjust the weights of the policy network through backpropagation or other optimization algorithms. The formula is: ;in These are the parameters of the current policy network. These are the updated policy network parameters. It's the learning rate. It is the gradient of the loss function with respect to the parameters.
[0067] Adjust constraints based on execution results, such as adjusting latency constraints and resource consumption limits, to ensure subsequent path selection better reflects the actual situation. The formula is: ;in For the adjusted resource constraints, The execution results are used to dynamically adjust constraints, ensuring that path selection meets new constraint requirements.
[0068] An online update mechanism is employed to ensure that the parameters of the policy network are updated promptly when processing new tasks, thereby better adapting to different task scenarios. (Still for:) ;in It is the step size updated online. It is based on parameter changes updated in real time.
[0069] The strategy network is adaptively adjusted based on historical feedback information, particularly classification accuracy, latency, resource consumption, and tail risk, to improve performance on future classification tasks. The formula is: ;in It is an adaptive adjustment function. It is the execution result. These are the adjusted policy network parameters.
[0070] The classification path selection strategy is dynamically optimized based on feedback information from the classification task, continuously improving the accuracy of the selected path. The formula is: ;in It is a path score. It is the probability of path selection. This is the normalization factor for the path score.
[0071] The process involves receiving and processing feedback information, including classification accuracy, execution path latency, computational resource consumption, and tail risk. The formula is: ;in For feedback information set.
[0072] Dynamic optimization of the path selection strategy, based on feedback information, dynamically optimizes the classification path selection strategy to continuously improve the accuracy of path selection. The formula is: ;in The final selected classification path must satisfy both resource constraints and risk constraints.
[0073] Input: Classification path execution results: including feedback information such as classification accuracy, execution latency, resource consumption, and tail risk. Policy network parameters: policy network parameters adjusted and updated based on historical feedback information.
[0074] Processing: Update the policy network: Update the parameters of the policy network based on the classification path execution results. Adjust constraints: Adjust latency and resource consumption constraints based on the execution results. Online update: Employ an online learning mechanism to ensure real-time updates to the policy network. Adaptive adjustment: Adaptively adjust the policy network based on historical feedback information. Optimize path selection strategy: Dynamically optimize the path selection strategy based on execution result feedback.
[0075] Output: Optimized policy network: including updated network parameters. Adjusted constraints: new resource and risk constraints. Final selected classification path: the classification path that satisfies the resource and risk constraints.
[0076] Real-time response: The system can adjust its strategy in real time based on feedback from classification tasks, improving the adaptability and accuracy of task execution. Resource and risk control: By adjusting constraints, the system ensures that classification tasks are performed within controllable resource and risk limits. Adaptive optimization: Through online updates and the fusion of historical feedback information, the policy network can continuously self-adjust, thereby improving the long-term performance of classification path selection.
[0077] Example 2: In smart city monitoring systems, processing and accurately classifying multimodal data faces several technical challenges. First, processing and fusing different modalities is difficult, requiring effective cross-modal encoders to reduce information loss and enhance information sharing. Second, dynamically optimizing classification path selection under limited resources and risks to ensure efficient classification without exceeding resource or risk constraints is a critical issue. Furthermore, real-time prediction of classification path execution costs and tail risks, and optimization during execution to ensure system stability and efficiency, are also essential technical challenges. To address these issues, this invention provides an automatic classification system for massive datasets based on a large-model data platform, the structure of which is as follows: Figure 5 As shown. The specific implementation process of this system is as follows: The data acquisition module is used to receive multimodal data input, including text data, image data, audio data, and video data; A cross-modal encoder module is used to process the multimodal data through a cross-modal encoder to obtain a unified cross-modal representation; The information optimization module is used for the cross-modal representation. It generates cross-modal feature representations through information optimization methods, removes redundant information, and retains information relevant to the task. The policy network module is used to input the cross-modal feature representation and platform context information into the policy network to generate a classification path selection policy. The platform context information includes computing resource status, task complexity, and latency constraints. The path execution module is used for the classification path selection strategy, selecting and executing classification paths, classifying the multimodal data, and predicting costs and risks during the execution process. The optimization and update module is used to update the policy network and constraints based on the results of the cost and risk predictions, so as to achieve continuous optimization of the classification path.
[0078] Specifically, the data acquisition module is responsible for receiving multimodal data from multiple sensors, including text, images, audio, and video data, and inputting them into the system to provide raw data support for subsequent processing.
[0079] The cross-modal encoder module processes the input data using dedicated encoders for different modalities, converting the features of different modalities into a unified cross-modal representation to eliminate differences between modalities and enable subsequent processing to be compatible with multiple data formats.
[0080] The information optimization module optimizes the generated cross-modal representation, removing redundant information and retaining key features relevant to the task, thereby improving the efficiency and accuracy of the classification task.
[0081] The strategy network module generates classification path selection strategies based on cross-modal feature representations and platform context information, ensuring optimal path selection and meeting the system's resource and risk requirements.
[0082] The path execution module performs tasks based on the classification path selection strategy and predicts the cost and tail risk of the classification path in real time, providing data feedback for subsequent optimization.
[0083] The optimization and update module updates the strategy network and constraints based on cost and risk predictions of the execution results, thereby continuously optimizing the classification path and ensuring that the system can adapt to changing task and platform conditions, and continuously improve classification performance and efficiency.
[0084] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. An automatic classification method for massive data based on a large model data platform, characterized in that: Includes the following steps: S1. Receive multimodal data input, wherein the multimodal data includes text data, image data, audio data, and video data; S2. The multimodal data is processed by a cross-modal encoder to obtain a unified cross-modal representation; S3. Based on the cross-modal representation, generate cross-modal feature representations using an information optimization method, removing redundant information and retaining task-related information; S4. Input the cross-modal feature representation and platform context information into the policy network to generate a classification path selection policy. The platform context information includes computing resource status, task complexity, and latency constraints. S5. Based on the classification path selection strategy, select and execute the classification path to classify the multimodal data, and predict costs and risks during the execution process; S6. Based on the results of the cost and risk prediction, update the strategy network and constraints to achieve continuous optimization of the classification path.
2. The automatic classification method for massive data based on a large model data platform according to claim 1, characterized in that, The cross-modal encoder includes: Different modal data are input into the corresponding modal encoders to generate representations of each modality; The modal representations are input into the cross-modal alignment layer; Based on the output of the cross-modal alignment layer, a unified cross-modal representation is obtained.
3. The automatic classification method for massive data based on a large model data platform according to claim 1, characterized in that, The cross-modal feature representation includes: Receive the unified cross-modal representation; Remove redundant features from the cross-modal representation that are irrelevant to the classification task; Retain and extract feature information relevant to the classification task to generate an optimized cross-modal feature representation.
4. The automatic classification method for massive data based on a large model data platform according to claim 1, characterized in that, The classification path selection strategy includes: Receive cross-modal feature representations and platform context information; Joint modeling of cross-modal feature representations with platform context information; Based on the results of the joint modeling, the candidate classification paths are scored; Generate the selection probability distribution of each candidate classification path and determine the classification path selection strategy.
5. The automatic classification method for massive data based on a large model data platform according to claim 1, characterized in that, The multimodal data classification includes: Receive the classification path selection strategy and determine the classification path to be executed; The execution cost of the classification path is estimated, including latency and computational resource consumption; The tail risk of the classification path is predicted, including the risk of classification error or the risk of exceeding the time limit.
6. The automatic classification method for massive data based on a large model data platform according to claim 1, characterized in that, The cost and risk forecasts include: Based on the predicted execution costs and tail risks, the classification path is optimized and selected. Determine the classification path that satisfies the preset resource constraints and risk constraints, and execute the classification path.
7. The automatic classification method for massive data based on a large model data platform according to claim 1, characterized in that, The policy network includes: Receive the execution results of the classification path; The parameters of the policy network are updated based on the execution results; Adjust the constraints based on the execution results to ensure that subsequent path selection meets resource and risk constraints.
8. The automatic classification method for massive data based on a large model data platform according to claim 1, characterized in that, The constraints include: The parameter updates based on the policy network employ an online update mechanism. When new task data is input, the policy network is adaptively adjusted based on historical feedback information.
9. The automatic classification method for massive data based on a large model data platform according to claim 1, characterized in that, The continuous optimization includes: Receive feedback information from the classification task, including classification accuracy, execution path latency, computational resources consumed, and tail risk; The classification path selection strategy is dynamically optimized based on the feedback information.
10. A massive data automatic classification system based on a large model data platform, characterized in that: The automatic classification method for massive data based on a large model data platform, as described in any one of claims 1-9, includes the following modules: The data acquisition module is used to receive multimodal data input, including text data, image data, audio data, and video data; A cross-modal encoder module is used to process the multimodal data through a cross-modal encoder to obtain a unified cross-modal representation; The information optimization module is used for the cross-modal representation. It generates cross-modal feature representations through information optimization methods, removes redundant information, and retains information relevant to the task. The policy network module is used to input the cross-modal feature representation and platform context information into the policy network to generate a classification path selection policy. The platform context information includes computing resource status, task complexity, and latency constraints. The path execution module is used for the classification path selection strategy, selecting and executing classification paths, classifying the multimodal data, and predicting costs and risks during the execution process. The optimization and update module is used to update the policy network and constraints based on the results of the cost and risk predictions, so as to achieve continuous optimization of the classification path.