An AI server intelligent data analysis method based on edge-cloud collaboration

Abstract

This invention discloses an intelligent data analysis method for an AI server based on edge-cloud collaboration, which relates to the field of edge-cloud computing collaborative technology. Edge devices are deployed at the edge to collect raw data for each task in real time. The data is preprocessed to extract task difficulty characteristics and form a task difficulty feature sequence. Based on the task difficulty feature sequence and a pre-trained AI data analysis model, the task difficulty of the collected task data is evaluated in combination with the current resource status, and the feasibility of the edge performing the task is determined. By deploying devices at the edge and preprocessing data, the invention effectively offloads some computing tasks to the edge, significantly reducing the computing burden of the cloud AI server. The edge can independently complete simple tasks, while complex tasks are uploaded to the cloud on demand, avoiding excessive centralized use of cloud resources, thereby reducing the energy consumption and operating costs of the cloud server and improving the energy efficiency of the overall system.

Description

Technical Field

[0001] The present invention relates to the field of edge-cloud computing collaborative technology, and specifically to an AI server intelligent data analysis method based on edge-cloud collaboration. Background Art

[0002] With the rapid development of the Internet and the Internet of Things, the amount of data generated is increasing day by day. Traditional data processing methods can no longer meet the needs of efficient and real-time data analysis. Various industries, especially finance, e-commerce, medical care, manufacturing and other fields, are facing the problem of huge and complex data volumes. In recent years, the field of artificial intelligence has made many major breakthroughs, providing strong technical support for intelligent data analysis. AI intelligent data analysis is equipped with high-performance processors, large-capacity memory and high-speed storage devices, which can quickly process massive amounts of data, help enterprises to extract valuable information from massive data, and provide a scientific basis for corporate decision-making.

[0003] In the existing technology, AI servers need to process massive amounts of data and have extremely high requirements for computing resources, resulting in huge energy consumption and increased operating costs. Therefore, how to offload some computing tasks to edge devices through edge-cloud collaborative technology to reduce the computing burden of cloud-based AI servers, thereby reducing the computing pressure and energy consumption of cloud-based AI servers, is the problem to be solved by the present invention. To this end, an AI server intelligent data analysis method based on edge-cloud collaboration is proposed. Summary of the Invention

[0004] The purpose of the present invention is to provide an AI server intelligent data analysis method based on edge-cloud collaboration to solve the problems raised in the above background technology.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] An AI server intelligent data analysis method based on edge-cloud collaboration includes the following steps:

[0007] S1. Deploy edge devices at the edge to collect the raw data of each task in real time, pre-process the data, extract task difficulty features, and form a task difficulty feature sequence;

[0008] S2: Based on the task difficulty feature sequence and pre-trained AI data analysis model, combined with the current resource status, the task difficulty of the collected task data is evaluated and the feasibility of performing the task at the edge is determined;

[0009] S3. Based on the task difficulty assessment results, the tasks are divided into cloud tasks and edge tasks, and different tasks are allocated and managed;

[0010] S4. Develop an offloading strategy based on edge computing capabilities, network status, and cloud load.

[0011] S5: The edge receives the task, uses local computing resources to quickly process it, obtains preliminary results, and uploads them to the cloud.

[0012] S6. The cloud receives the task processing results uploaded by the edge, analyzes the received offload tasks, and then integrates the task processing results of the edge and cloud to generate a final data analysis report.

[0013] A further improvement of the technical solution of the present invention is that: S1 specifically includes:

[0014] Based on the application scenario requirements, plan the type, quantity, and deployment location of edge devices required at the edge end, select edge devices with corresponding computing, storage, and communication capabilities, and deploy the edge devices at the edge end, ensure that they are properly connected to the system network, initialize edge device parameters, configure data collection tasks and frequency, and put them in a standby state;

[0015] Deployed edge devices continuously collect raw data generated by each task according to preset rules and frequencies. Data sources include various sensors, device interfaces, or other data sources. During the collection process, they monitor data quality and mark missing or abnormal data.

[0016] Preprocess the collected raw data on the edge device. Preprocessing includes data formatting, data cleaning, and data normalization. Data formatting unifies the data structure and format. Data cleaning removes noise, duplicate data, and unreasonable data points. Data normalization adjusts the data to a unified dimensional range. Feature analysis is performed on the preprocessed data to extract task difficulty features that reflect the difficulty of the task, including data volume, data update frequency, data value density, and computing resource requirements.

[0017] Integrate the extracted task difficulty features, and set the reference value of each task difficulty feature according to the preset edge and cloud task division standards, and then form a task difficulty feature sequence, which is stored in the local storage system of the edge device.

[0018] A further improvement of the technical solution of the present invention is that: S2 specifically includes:

[0019] Read the task difficulty feature sequence from the local storage of the edge device, load the pre-trained AI data analysis model, and obtain the current resource status of the edge end in real time, including CPU usage, remaining memory, and storage space;

[0020] Input the task difficulty features in the task difficulty feature sequence into the AI ​​data analysis model, analyze and process the input task difficulty features, and output a task difficulty assessment result for the collected task data, wherein the AI ​​data analysis model receives the input task difficulty features and outputs a task difficulty score, which is a score of 1-10. The task difficulty is divided into different task difficulty levels, namely simple tasks, medium difficulty tasks, and complex tasks, and the scoring range of each task difficulty level is determined;

[0021] Based on the task difficulty assessment results, combined with the characteristics of different types of tasks and historical data, the feasibility of performing tasks on the edge is determined.

[0022] A further improvement of the technical solution of the present invention is that the calculation process of the task difficulty score is:

[0023] Based on the task difficulty feature sequence, the total volume of task data is calculated to obtain the data size, the update frequency of task data is determined to represent the dynamic nature of the data, the proportion of valid information in the data is calculated to obtain the data value density, the degree of computing resource demand of the task is determined, and the reference value of each task difficulty feature is determined;

[0024] Calculate the ratio of the characteristic value of each task difficulty feature to its reference value, and square each ratio;

[0025] The squared ratios are weighted and summed, where the weight of each task difficulty feature is equal, which is 1 / 4. The square root of the weighted sum is then calculated and multiplied by 10 to obtain the final task difficulty score.

[0026] A further improvement of the technical solution of the present invention is that: S3 specifically includes:

[0027] Based on the task difficulty assessment results, combined with the resource characteristics and processing power differences between the cloud and edge, a standard for classifying tasks into cloud tasks and edge tasks is developed. This standard takes into account the task's requirements for computing and storage resources, as well as its real-time and security requirements. Simple tasks are classified as edge tasks, and complex tasks are classified as cloud tasks. For tasks of medium difficulty, tasks with high computing requirements, non-real-time requirements, and relatively low sensitivity to data security are classified as cloud tasks, while tasks with low computing requirements, high real-time requirements, and the need for fast local processing are classified as edge tasks.

[0028] All pending tasks are classified according to the established classification criteria. Tasks that meet the characteristics of cloud tasks are grouped into the cloud task set, while tasks that meet the characteristics of edge tasks are grouped into the edge task set. Edge tasks emphasize local processing and fast response to reduce data transmission delays.

[0029] For the divided cloud tasks and edge tasks, a task management system is established, and a task allocation strategy is formulated to uniformly manage and monitor cloud tasks and edge tasks.

[0030] A further improvement of the technical solution of the present invention is that the process of formulating the task allocation strategy is as follows:

[0031] Design and build a task management system with task classification, resource scheduling, and monitoring management capabilities, supporting collaborative work between the cloud and edge, ensuring that tasks are assigned to the appropriate computing nodes based on classification results;

[0032] Based on the task classification results, a detailed task allocation strategy is formulated. According to the formulated task allocation strategy, cloud tasks and edge tasks are assigned to the corresponding computing nodes. Cloud tasks are sent to the cloud server for centralized processing, and edge tasks are assigned to edge devices for local processing. At the same time, the execution of tasks is monitored in real time to detect and handle abnormal situations in task execution.

[0033] Conduct unified monitoring of cloud and edge tasks, collect task execution status, resource usage, and performance indicators in real time, and dynamically adjust the task allocation strategy for each task based on monitoring data to optimize resource utilization efficiency. For tasks that execute slowly, reallocate resources based on actual conditions.

[0034] A further improvement of the technical solution of the present invention is that: S4 specifically includes:

[0035] Real-time monitoring of edge computing power, network status, and cloud load to obtain the overall resource status of the current system. Ensure the accuracy and real-time nature of monitoring data to promptly reflect changes in system resources. Evaluate the resource requirements of each pending task and, based on the difficulty of the task, determine the resources required to execute the task at the edge or in the cloud, ensuring that the evaluation results accurately reflect the actual resource requirements of the task.

[0036] Develop dynamic task offloading strategies based on edge computing power, network status, and cloud load. When edge resources are insufficient or task complexity exceeds its processing capacity, tasks are offloaded to the cloud first.

[0037] According to the established offloading strategy, eligible tasks are offloaded from the edge to the cloud. During the task offloading process, the execution status and resource usage of the tasks are monitored in real time, and the offloading strategy is dynamically adjusted according to changes in system resources. For offloaded tasks, resources are reallocated according to the actual load conditions in the cloud to ensure that the tasks can be executed efficiently.

[0038] A further improvement of the technical solution of the present invention is that the process of task offloading is:

[0039] Based on the established offloading strategy, tasks that meet the offloading conditions are screened out. Before starting the offloading operation, a comprehensive analysis of the tasks is conducted to clarify relevant information about the tasks, including task type, data size, and processing logic. At the same time, a communication channel with the cloud is established to ensure the reliability of data transmission, and the protocols and interfaces required for task offloading are prepared to ensure the standardization and compatibility of the offloading process. Then, the task offloading process is officially started, and qualified tasks are transferred from the edge to the cloud.

[0040] During the task offloading process, the execution status and resource usage of the task are monitored in real time. The execution status of the task is monitored to track the dynamics of task offloading, and the offloading strategy is dynamically adjusted according to the monitored changes in system resources.

[0041] For tasks that have been offloaded to the cloud, the resource requirements of the tasks are regularly re-evaluated based on the actual load of the cloud to reallocate resources. If the cloud load changes, the resources required for the task are reallocated based on the current resource status to ensure the execution efficiency of the task in the cloud. At the same time, the resource allocation strategy is optimized based on the execution status of the task and the efficiency of resource utilization to further improve the overall performance and resource utilization of the system.

[0042] A further improvement of the technical solution of the present invention is that: S5 specifically includes:

[0043] The edge receives the assigned tasks, parses them, and automatically allocates local computing, storage, and network resources based on the task's resource requirements and the current state of the edge device to ensure that the task has sufficient resource support. It also records detailed task information and resource allocation.

[0044] After task parsing is complete, the edge performs a comprehensive assessment of the locally available computing resources, checking CPU usage, remaining computing power, memory usage, available capacity, and the read / write speed and remaining space of storage devices. Based on task requirements and local resource availability, resources are allocated, reserving computing, memory, and storage space for task processing. At the same time, the algorithm libraries and model files required for task processing are loaded to ensure that the local environment is equipped to handle the task.

[0045] The edge uses prepared local resources and algorithm models to quickly process tasks. During the processing, it monitors the execution progress and resource usage of tasks in real time to ensure that the tasks are carried out as expected. It then obtains preliminary results of the tasks and formats them to meet the requirements of subsequent upload and cloud processing.

[0046] The edge encrypts the generated preliminary results to ensure the security and confidentiality of the data during transmission, and uploads the encrypted preliminary results to the cloud. After the upload is completed, the edge sends feedback information to the task initiator to inform that the preliminary processing of the task has been completed and uploaded to the cloud. At the same time, the edge records relevant information about the task processing, including processing time, resource consumption, etc., for subsequent performance analysis and optimization.

[0047] A further improvement of the technical solution of the present invention is that: S6 specifically includes:

[0048] The cloud receives the task processing results uploaded by the edge, ensures the integrity and accuracy of the data, decrypts and verifies the format of the received data, and stores it in the cloud's distributed storage system for subsequent analysis and processing. At the same time, it records the task upload time, data volume, and source information;

[0049] The cloud analyzes the received offload tasks, extracts valuable information and patterns from the data, analyzes the execution of the tasks, evaluates the effectiveness and accuracy of the task results, and provides a basis for integrating the results;

[0050] Integrate the task processing results uploaded by the edge with the analysis results of the offloaded tasks in the cloud. Use a unified data format and standard to clean, convert, and match data from different sources to ensure data consistency and compatibility. Then analyze the integrated data, compare the results of different processing links, identify differences and commonalities, and deeply analyze the impact of each step of task processing on the final result.

[0051] Based on the results of the comprehensive analysis, the cloud automatically generates a final data analysis report, covering the overall situation of task processing, key findings, potential value and related suggestions. After the data analysis report is generated, it will be fed back to the task initiator. At the same time, the data analysis report and related data will be stored in the cloud database for subsequent query, tracing and further analysis.

[0052] Due to the adoption of the above technical solution, the present invention has the following technical advancements compared to the prior art:

[0053] The present invention provides an AI server intelligent data analysis method based on edge-cloud collaboration. By deploying devices at the edge and preprocessing data, some computing tasks are effectively offloaded to the edge, significantly reducing the computing burden of the cloud AI server. The edge can complete simple tasks independently, and complex tasks are uploaded to the cloud on demand, avoiding excessive centralized use of cloud resources, thereby reducing the energy consumption and operating costs of the cloud server and improving the energy efficiency of the overall system.

[0054] The present invention provides an AI server intelligent data analysis method based on edge-cloud collaboration. It combines the computing power of the edge, network status and cloud load to formulate an offloading strategy to ensure that the task is executed in the most suitable environment. When the edge resources are insufficient or the task is complex, the task is offloaded to the cloud. Otherwise, it is retained for processing at the edge. This flexible resource allocation mechanism avoids idleness and waste of resources, improves the overall utilization of system resources, and achieves optimal resource configuration. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments described in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0056] Figure 1 Schematic diagram of the workflow of the present invention.

[0057] Figure 2 Schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0059] Example 1, as Figure 1 、 Figure 2 As shown, the present invention provides an AI server intelligent data analysis method based on edge-cloud collaboration, comprising the following steps:

[0060] S1. Deploy edge devices at the edge to collect the raw data of each task in real time, pre-process the data, extract the task difficulty features, and form a task difficulty feature sequence. Based on the application scenario requirements, plan the type, quantity and deployment location of the edge devices required at the edge, select edge devices with corresponding computing, storage and communication capabilities, and deploy the edge devices at the edge to ensure that they are connected to the system network normally, initialize the edge device parameters, configure the data collection tasks and frequency, and put them in standby state. The deployed edge devices continuously collect the raw data generated by each task according to the preset rules and frequency. The data sources include various sensors, device interfaces or other data sources, etc., and monitor the data quality during the collection process, mark missing and abnormal data, and pre-process the collected raw data on the edge devices. The pre-processing includes data formatting, data cleaning and data normalization. Among them, the data is unified through data formatting. Structure and format, through data cleaning, remove noise, duplicate data and unreasonable data points, through data normalization, adjust the data to a unified dimensional range, and perform feature analysis on the preprocessed data to extract task difficulty features that reflect the difficulty of the task, including data volume, data update frequency, data value density and computing resource requirements. The data volume is the total volume of the task data, that is, the file size of the task data, the data update frequency is the update frequency of the task data, that is, the dynamic nature of the data, the data value density is the proportion of effective information in the data, and measures the importance of data to the task, measured by the ratio of the effective data volume to the total data volume. The computing resource requirement is the degree of demand for computing resources for the task. The extracted task difficulty features are integrated, and according to the preset edge and cloud task division standards, the reference value of each task difficulty feature is set, thereby forming a task difficulty feature sequence and storing it in the local storage system of the edge device;

[0061] S2. Based on the task difficulty feature sequence and the pre-trained AI data analysis model, the task difficulty of the collected task data is evaluated in combination with the current resource status, and the feasibility of the edge end performing the task is judged. By evaluating the task requirements, the edge end can determine whether it can complete the task independently. The task difficulty feature sequence is read from the local storage of the edge device. At the same time, the pre-trained AI data analysis model is loaded, and the current resource status of the edge end is obtained in real time, including CPU usage, memory remaining, and storage space. Among them, a large amount of task sample data is collected, including difficulty features of different types of tasks, and each task sample is labeled. The annotation content includes the actual task difficulty score of the task and the computing resources required to complete the task. Feature extraction is performed on the collected task sample data to extract features related to task difficulty. The random forest model is selected as the basic architecture to train the AI ​​data analysis model. The parameters of the selected random forest model are initialized. The task difficulty feature data in the training set is input into the model. The predicted task difficulty evaluation result is obtained through the calculation of the model. The loss function value is calculated based on the predicted result and the actual labeled task difficulty evaluation label. The gradient is calculated based on the loss function value through the back propagation algorithm, and the optimization algorithm is used. Update the model parameters to minimize the loss function value, repeat the process of forward propagation, loss function calculation, backpropagation and parameter update until the training stop condition is met, use the test set to evaluate the trained model, calculate the model evaluation index, and tune the model based on the model evaluation results. The tuning method includes adjusting the model parameters and increasing or decreasing the number of layers and neurons in the model. After multiple tuning and evaluations, a trained AI data analysis model is obtained. Finally, the trained AI data analysis model is deployed on the edge device so that it can receive task difficulty feature data in real time and perform task difficulty assessment. The task difficulty features in the task difficulty feature sequence are input into the AI ​​data analysis model, the input task difficulty features are analyzed and processed, and the task difficulty assessment results of the collected task data are output. The AI ​​data analysis model receives the input task difficulty features and outputs a task difficulty score. The task difficulty score is 1-10 points. The task difficulty is divided into different task difficulty levels, namely simple tasks, medium difficulty tasks and complex tasks, and the score range of each task difficulty level is determined. Based on the task difficulty assessment results, combined with the characteristics of different types of tasks and historical data, the feasibility of performing the task on the edge is judged;

[0062] In addition, the task difficulty score is calculated as follows:

[0063] Based on the task difficulty feature sequence, the total volume of the task data is calculated to obtain the data size, the update frequency of the task data is determined to represent the dynamic nature of the data, the proportion of valid information in the data is calculated to obtain the data value density, the degree of computing resource demand of the task is determined, and the reference value of each task difficulty feature is determined. The ratio of the characteristic value of each task difficulty feature to its reference value is calculated, and each ratio is squared. The squared ratios are weighted and summed, where the weight of each task difficulty feature is equal, namely 1 / 4. The square root of the weighted sum is then calculated and multiplied by 10 to obtain the final task difficulty score.

[0064] The calculation expression of task difficulty score is:

[0065] ;

[0066] Where, Rate the difficulty of the task on a scale of 1 to 10, with higher values ​​indicating greater difficulty. is the data size, indicating the total volume of task data, It is a reference value for the data size, used to standardize the data size, and is determined based on historical data. is the data update frequency, indicating the dynamic nature of task data. It is a reference value for data update frequency, used to standardize data update frequency, determined based on historical data. is the data value density, which indicates the proportion of effective information in the data. It is a reference value for data value density, used to standardize data value density, and is determined based on historical data. is the computing resource demand, which indicates the degree of computing resource demand of the task. It is a reference value for computing resource requirements, used to standardize computing resource requirements, and is determined based on historical data. The scoring range for simple tasks is , the scoring range for medium difficulty tasks is , the scoring range of complex tasks is ;

[0067] S3. Based on the task difficulty assessment results, tasks are divided into cloud tasks and edge tasks, and different tasks are allocated and managed. Based on the task difficulty assessment results, combined with the resource characteristics and processing power differences between the cloud and edge, a standard for dividing tasks into cloud tasks and edge tasks is established. The requirements for computing and storage resources, as well as the real-time and security requirements of the tasks are comprehensively considered. Simple tasks are divided into edge tasks, and complex tasks are divided into cloud tasks. For tasks of medium difficulty, tasks with high computing requirements, non-real-time requirements, and relatively low sensitivity to data security are divided into cloud tasks. Tasks with low computing requirements, high real-time requirements, and the need for fast local processing are divided into edge tasks. According to the established classification standards, all pending tasks are classified. Tasks that meet the characteristics of cloud tasks are assigned to the cloud task set. Cloud tasks require strong computing power and large-scale data storage support and can be processed centrally in the cloud. Tasks that meet the characteristics of edge tasks are assigned to the edge task set. Edge tasks emphasize local processing, fast response, and reduced data transmission delay. For the divided cloud and edge tasks, a task management system is established, and a task allocation strategy is formulated to uniformly manage and monitor cloud and edge tasks.

[0068] In addition, the task allocation strategy is formulated as follows:

[0069] Design and build a task management system with task classification, resource scheduling and monitoring management functions, support the collaborative work of the cloud and edge, ensure that tasks are assigned to the corresponding computing nodes according to the classification results, and formulate detailed task allocation strategies based on the task classification results. For cloud tasks, consider the computing resources and storage capabilities of the cloud to ensure that tasks can be efficiently executed on suitable servers. For edge tasks, focus on local processing capabilities and give priority to allocating them to edge devices with sufficient resources. According to the formulated task allocation strategy, cloud tasks and edge tasks are assigned to the corresponding computing nodes. Cloud tasks are sent to cloud servers for centralized processing, and edge tasks are assigned to edge devices for local processing. At the same time, the execution of tasks is monitored in real time, and abnormal situations in task execution are discovered and handled. Cloud and edge tasks are uniformly monitored, and the execution status, resource usage and performance indicators of tasks are collected in real time. Based on the monitoring data, the task allocation strategy of each task is dynamically adjusted to optimize resource utilization efficiency. For tasks that execute slowly, resources are reallocated according to actual conditions.

[0070] S4. Develop an offloading strategy based on edge computing capabilities, network status, and cloud load. If edge resources are insufficient or the task is complex, offload the task to the cloud. This fully utilizes the resource advantages of both the edge and cloud, avoids resource waste, ensures efficient task execution, and reduces cloud pressure.

[0071] S5: The edge receives the task, uses local computing resources to quickly process it, obtains preliminary results, and uploads them to the cloud.

[0072] S6. The cloud receives the task processing results uploaded by the edge, analyzes the received offload tasks, and then integrates the task processing results of the edge and cloud to generate a final data analysis report.

[0073] Example 2, as Figure 1 、 Figure 2 As shown, based on Example 1, the present invention provides a technical solution: preferably, S4 specifically includes:

[0074] Monitor the computing power, network status and cloud load of the edge in real time to obtain the overall resource status of the current system, ensure the accuracy and real-time nature of the monitoring data, so as to timely reflect the changes in system resources, and evaluate the resource requirements of each pending task. Combined with the difficulty characteristics of the task, determine the resources required for the task to be executed at the edge or in the cloud, and ensure that the evaluation results can accurately reflect the actual resource requirements of the task. According to the computing power, network status and cloud load of the edge, formulate a dynamic task offloading strategy. When the edge resources are insufficient or the complexity of the task exceeds its processing capacity, the task is offloaded to the cloud first. At the same time, consider the impact of the network status on data transmission to avoid the decrease in task execution efficiency due to network delay or insufficient bandwidth, and ensure that the offloading strategy can fully utilize resources while reducing the pressure on the cloud. According to the formulated offloading strategy, eligible tasks are offloaded from the edge to the cloud. During the task offloading process, monitor the execution status and resource usage of the task in real time, and dynamically adjust the offloading strategy according to the changes in system resources. For offloaded tasks, reallocate resources according to the actual load of the cloud to ensure that the task can be executed efficiently.

[0075] In addition, the process of task offloading is:

[0076] According to the established offloading strategy, tasks that meet the offloading conditions are screened out, and before starting the offloading operation, the tasks are comprehensively analyzed to clarify the relevant information of the tasks, including task type, data scale and processing logic. At the same time, a communication channel with the cloud is established to ensure the reliability of data transmission, prepare the protocols and interfaces required for task offloading, ensure the standardization and compatibility of the offloading process, and then formally start the task offloading process to transmit qualified tasks from the edge to the cloud. During the task offloading process, the execution status and resource usage of the tasks are monitored in real time, and the execution status of the tasks is paid attention to to track the dynamics of task offloading. The offloading strategy is dynamically adjusted according to the monitored changes in system resources. Among them, if the edge resources are suddenly relieved during the offloading process, and the cloud If the load is high, the number of offloaded tasks can be appropriately reduced or some offload operations can be suspended, and some tasks can be retained for processing on the edge. Conversely, if the edge resources are continuously tight and the cloud has sufficient processing capacity, the task offloading speed can be accelerated. By flexibly adjusting the offloading strategy, it is ensured that system resources are reasonably utilized to avoid resource waste or task execution obstruction. For tasks that have been offloaded to the cloud, the resource requirements of the tasks are regularly re-evaluated according to the actual load of the cloud to reallocate resources. If the cloud load changes, the resources required for the task are reallocated according to the current resource status to ensure the execution efficiency of the task in the cloud. At the same time, the resource allocation strategy is optimized according to the execution status of the task and the efficiency of resource utilization to further improve the overall performance and resource utilization of the system.

[0077] S5 specifically includes:

[0078] The edge receives the assigned tasks, analyzes the tasks, and automatically allocates local computing, storage, and network resources based on the resource requirements of the tasks and the current status of the edge devices to ensure that the tasks can obtain sufficient resource support. At the same time, the detailed information of the tasks and the resource allocation are recorded. After the task analysis is completed, the edge conducts a comprehensive evaluation of the locally available computing resources, checks the CPU usage, remaining computing power, memory occupancy, available capacity, and the read and write speed and remaining space of the storage device, and allocates resources based on the task requirements and local resource conditions, reserving computing, memory, and storage space for task processing. At the same time, the algorithm library and model files required for task processing are loaded to ensure that the local environment has the conditions to process the tasks. The edge side quickly processes the task with the prepared local resources and algorithm models, and monitors the execution progress and resource usage of the task in real time during the processing to ensure that the task is carried out as expected, and then obtains the preliminary results of the task, formats the preliminary results to meet the requirements of subsequent upload and cloud processing. The edge side encrypts the generated preliminary results to ensure the security and confidentiality of the data during transmission, and uploads the encrypted preliminary results to the cloud. After the upload is completed, the edge side sends feedback information to the task initiator, informing that the preliminary processing of the task has been completed and uploads it to the cloud. At the same time, the edge side records the relevant information of the task processing, including processing time, resource consumption, etc., for subsequent performance analysis and optimization;

[0079] S6 specifically includes:

[0080] The cloud receives the task processing results uploaded by the edge, ensures the integrity and accuracy of the data, decrypts and verifies the format of the received data, and stores it in the distributed storage system of the cloud for subsequent analysis and processing. At the same time, it records the upload time, data volume and source information of the task, and verifies the format of the data to ensure that it meets the preset specifications. If the verification passes, the data is stored in a temporary buffer. If the verification fails, a retransmission request is sent to the edge to request the data to be uploaded again. The cloud analyzes the received offloaded tasks, extracts valuable information and patterns from the data, analyzes the execution of the tasks, evaluates the effectiveness and accuracy of the task results, and provides a basis for integrating the results. The task processing results are integrated with the cloud's analysis results of the offloaded tasks. Using a unified data format and standard, data from different sources are cleaned, converted, and matched to ensure data consistency and compatibility. The integrated data is then analyzed, and the results of different processing links are compared to identify differences and commonalities. The impact of each link in the task processing on the final result is deeply analyzed. Based on the results of the comprehensive analysis, the cloud automatically generates a final data analysis report, covering the overall situation of the task processing, key findings, potential value, and related suggestions. After the data analysis report is generated, it is fed back to the task initiator. At the same time, the data analysis report and related data are stored in the cloud database for subsequent query, tracing, and further analysis.

[0081] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. An AI server intelligent data analysis method based on edge-cloud collaboration, characterized in that: The following steps are involved: S1. Deploy edge devices at the edge to collect raw data of each task in real time, preprocess the data, extract task difficulty features, and form a task difficulty feature sequence. S1 specifically includes: Based on the application scenario requirements, plan the type, quantity, and deployment location of edge devices required at the edge, deploy the edge devices at the edge, initialize the edge device parameters, configure the data collection tasks and frequency, and put them in a standby state; Deployed edge devices continuously collect raw data generated by each task according to preset rules and frequencies. During the collection process, they monitor data quality and mark missing or abnormal data. Preprocess the collected raw data on the edge device. This includes data formatting, data cleaning, and data normalization. Feature analysis is then performed on the preprocessed data to extract task difficulty features, including data volume, data update frequency, data value density, and computing resource requirements. The proportion of valid information in the data is calculated to obtain the data value density. Integrate the extracted task difficulty features and set reference values ​​for each task difficulty feature according to the preset edge and cloud task division standards, thereby forming a task difficulty feature sequence and storing it in the local storage system of the edge device; S2: Based on the task difficulty feature sequence and pre-trained AI data analysis model, combined with the current resource status, the task difficulty of the collected task data is evaluated and the feasibility of performing the task at the edge is determined; S3. Based on the task difficulty assessment results, the tasks are divided into cloud tasks and edge tasks, and different tasks are allocated and managed; S4. Develop an offloading strategy based on edge computing capabilities, network status, and cloud load. S4 specifically includes: Monitor edge computing power, network status, and cloud load in real time to obtain the overall resource status of the current system. This system then assesses the resource requirements of each pending task and, based on the task's difficulty, determines the resources required to execute the task on the edge or in the cloud. Develop dynamic task offloading strategies based on edge computing power, network status, and cloud load. When edge resources are insufficient or task complexity exceeds its processing capacity, tasks are offloaded to the cloud first. According to the established offloading strategy, eligible tasks are offloaded from the edge to the cloud. During the task offloading process, the execution status and resource usage of the tasks are monitored in real time. The offloading strategy is dynamically adjusted according to changes in system resources. For offloaded tasks, resources are reallocated based on the actual load on the cloud. S5: The edge receives the task, uses local computing resources to quickly process it, obtains preliminary results, and uploads them to the cloud. S6. The cloud receives the task processing results uploaded by the edge, analyzes the received offload tasks, and then integrates the task processing results of the edge and cloud to generate a final data analysis report.

2. The AI ​​server intelligent data analysis method based on edge-cloud collaboration according to claim 1, characterized in that: The S2 specifically includes: Read the task difficulty feature sequence from the local storage of the edge device, load the pre-trained AI data analysis model, and obtain the current resource status of the edge end in real time, including CPU usage, remaining memory, and storage space; Input the task difficulty features in the task difficulty feature sequence into the AI ​​data analysis model, analyze and process the input task difficulty features, and output a task difficulty assessment result for the collected task data, wherein the AI ​​data analysis model receives the input task difficulty features and outputs a task difficulty score, which is a score of 1-10. The task difficulty is divided into different task difficulty levels, namely simple tasks, medium difficulty tasks, and complex tasks, and the scoring range of each task difficulty level is determined; Based on the task difficulty assessment results, combined with the characteristics of different types of tasks and historical data, the feasibility of performing tasks on the edge is determined.

3. The AI ​​server intelligent data analysis method based on edge-cloud collaboration according to claim 2 is characterized by: The calculation process of the task difficulty score is: Based on the task difficulty feature sequence, the total volume of task data is calculated to obtain the data size, determine the update frequency of task data, represent the dynamic nature of the data, determine the degree of computing resource demand of the task, and determine the reference value of each task difficulty feature; Calculate the ratio of the characteristic value of each task difficulty feature to its reference value, and square each ratio; The squared ratios are weighted and summed, where the weight of each task difficulty feature is equal, which is 1 / 4. The square root of the weighted sum is then calculated and multiplied by 10 to obtain the final task difficulty score.

4. The AI ​​server intelligent data analysis method based on edge-cloud collaboration according to claim 2, characterized in that: The S3 specifically includes: Based on the task difficulty assessment results, combined with the resource characteristics and processing power differences between the cloud and edge, a standard for classifying tasks into cloud tasks and edge tasks is developed. This standard takes into account the task's requirements for computing and storage resources, as well as its real-time and security requirements. Simple tasks are classified as edge tasks, and complex tasks are classified as cloud tasks. For tasks of medium difficulty, tasks with high computing requirements, non-real-time requirements, and relatively low sensitivity to data security are classified as cloud tasks, while tasks with low computing requirements, high real-time requirements, and the need for fast local processing are classified as edge tasks. According to the established classification criteria, all pending tasks are classified. Tasks that meet the characteristics of cloud tasks are grouped into the cloud task set, and tasks that meet the characteristics of edge tasks are grouped into the edge task set. For the divided cloud tasks and edge tasks, a task management system is established, and a task allocation strategy is formulated to uniformly manage and monitor cloud tasks and edge tasks.

5. The AI ​​server intelligent data analysis method based on edge-cloud collaboration according to claim 4 is characterized in that: The formulation process of the task allocation strategy is as follows: Design and build a task management system with task classification, resource scheduling, and monitoring management capabilities, supporting collaborative work between the cloud and edge. Based on the task classification results, a detailed task allocation strategy is formulated. According to the formulated task allocation strategy, cloud tasks and edge tasks are assigned to corresponding computing nodes. Cloud tasks are sent to cloud servers for centralized processing, and edge tasks are assigned to edge devices for local processing. Conduct unified monitoring of cloud and edge tasks, collect task execution status, resource usage, and performance indicators in real time, dynamically adjust the task allocation strategy for each task based on monitoring data, and reallocate resources for slow-executing tasks based on actual conditions.

6. The AI ​​server intelligent data analysis method based on edge-cloud collaboration according to claim 1, characterized in that: The process of task offloading is as follows: Based on the established offloading strategy, tasks that meet the offloading conditions are screened out. Before initiating the offloading operation, a comprehensive task analysis is conducted to clarify relevant task information, including task type, data size, and processing logic. At the same time, a communication channel with the cloud is established, and the protocols and interfaces required for task offloading are prepared. The task offloading process is then officially initiated, and eligible tasks are transferred from the edge to the cloud. During the task offloading process, the execution status and resource usage of the task are monitored in real time. The execution status of the task is monitored to track the dynamics of task offloading, and the offloading strategy is dynamically adjusted according to the monitored changes in system resources. For tasks that have been offloaded to the cloud, the resource requirements of the tasks are regularly re-evaluated based on the actual load of the cloud to perform resource reallocation. If the cloud load changes, the resources required for the task are reallocated based on the current resource status. At the same time, the resource allocation strategy is optimized based on the task execution status and resource utilization efficiency.

7. The AI ​​server intelligent data analysis method based on edge-cloud collaboration according to claim 1, characterized in that: The S5 specifically includes: The edge receives the assigned tasks, parses them, and automatically allocates local computing, storage, and network resources based on the task's resource requirements and the current state of the edge device. It also records detailed task information and resource allocation. After task parsing is complete, the edge performs a comprehensive assessment of locally available computing resources, checking CPU usage, remaining computing power, memory usage, available capacity, and the read / write speed and remaining space of storage devices. Based on task requirements and local resource availability, it allocates resources, reserving computing, memory, and storage space for task processing. At the same time, it loads the algorithm libraries and model files required for task processing. The edge uses prepared local resources and algorithm models to quickly process tasks. During the processing, it monitors the execution progress and resource usage of tasks in real time, and then obtains preliminary results of the tasks. The preliminary results are formatted to meet the requirements of uploading and cloud processing. The edge encrypts the generated preliminary results and uploads the encrypted preliminary results to the cloud. After the upload is completed, the edge sends feedback information to the task initiator, informing that the preliminary processing of the task has been completed and uploaded to the cloud. At the same time, the edge records the relevant information of the task processing.

8. The AI ​​server intelligent data analysis method based on edge-cloud collaboration according to claim 7, characterized in that: The S6 specifically includes: The cloud receives the task processing results uploaded by the edge, decrypts and verifies the format of the received data, and stores it in the cloud's distributed storage system. At the same time, it records the task's upload time, data volume, and source information. The cloud analyzes the received offload tasks, extracts valuable information and patterns from the data, analyzes the execution of the tasks, and evaluates the effectiveness and accuracy of the task results; Integrate the task processing results uploaded by the edge with the analysis results of the offloaded tasks in the cloud. Use a unified data format and standard to clean, convert, and match data from different sources. Then analyze the integrated data, compare the results of different processing links, identify differences and commonalities, and deeply analyze the impact of each step of task processing on the final result. Based on the results of the comprehensive analysis, the cloud automatically generates a final data analysis report, covering the overall situation of task processing, key findings, potential value and related suggestions. After the data analysis report is generated, it will be fed back to the task initiator, and the data analysis report and related data will be stored in the cloud database.

Images