Storage system configuration monitoring system based on smart energy cloud

Abstract

The present invention discloses a storage system configuration monitoring system based on a smart energy cloud, which belongs to the field of data transmission processing and monitoring technology. The present invention can obtain the specific numerical value of the computing resource demand and the specific change multiple of the data volume after executing the corresponding task through the execution task type information of the edge data segment in the computing server; when calculating and allocating the edge data segment, the edge data segment can be accurately and reasonably calculated and allocated based on the specific numerical value of the computing resource demand, effectively avoiding the excessive load of individual computing servers, and ensuring the safety of system operation to a certain extent; when storing and allocating the calculation result data of the edge data segment, the edge data segment can be accurately and reasonably stored and allocated based on the original data volume and the specific change multiple of the data volume after executing the corresponding task, thereby realizing the reasonable and efficient allocation of storage space.

Description

Technical Field

[0001] The present invention relates to the field of data transmission processing and monitoring technology, and in particular to a storage system configuration monitoring system based on a smart energy cloud. Background Art

[0002] Smart energy is the product of the deep integration of next-generation information technology and energy systems. Its core goal is to achieve efficient, clean, and sustainable energy utilization through intelligent means. The cloud storage system of a smart energy system primarily consists of servers, storage devices, and network equipment. Servers are primarily computing servers, responsible for handling various control logic, data requests, and management tasks of the storage system. They run storage management software and file system services, coordinate data flow between storage devices, and communicate with clients or other cloud services. Storage devices typically consist of multiple hard drives, using storage technology to distribute data across multiple drives to provide highly reliable and high-performance data storage. These can be traditional hard disk arrays (HDDs), solid-state drive arrays (SSDs), or a hybrid of the two. Network equipment primarily includes switches and routers.

[0003] When the edge devices of the smart energy system generate relevant data, they are transmitted to the computing server through the network for calculation and processing. The computing server then stores and manages the calculated data and allocates the calculated data to the storage device for storage.

[0004] How to better process edge device data based on the configuration monitoring information of the cloud storage system and thus achieve classified management of computing servers and storage devices is an urgent problem to be solved. To this end, a storage system configuration monitoring system based on the smart energy cloud is proposed. Summary of the Invention

[0005] The technical problem to be solved by the present invention is: how to better process edge device data based on the configuration monitoring information of the cloud storage system, and then realize the classified management of computing servers and storage devices, and provide a storage system configuration monitoring system based on the smart energy cloud.

[0006] The present invention solves the above technical problems through the following technical solutions, which include an edge data acquisition module, a data preprocessing module, a data transmission module, a server monitoring module, a storage device monitoring module, and a data distribution management module;

[0007] The edge data acquisition module is used to acquire edge data segments generated by different edge devices;

[0008] The data preprocessing module is used to read the original data volume of the acquired edge data segments, obtain the computing resource requirements, and perform data segment encoding processing to obtain encoded data packets, and send each data packet to the data transmission module;

[0009] The data transmission module is used to send the data packets corresponding to the edge data segments to the data distribution management module in parallel;

[0010] The server monitoring module is used to obtain the performance indicators of each computing server and calculate the available computing resources of each computing server;

[0011] The storage device monitoring module is used to obtain the available capacity of each hard disk through storage management software;

[0012] The data allocation management module is used to obtain the data packet corresponding to each edge data segment, and read the computing resource requirement code of each edge data segment from the data packet, and allocate each edge data segment to the corresponding computing server for calculation based on the specific value of the computing resource requirement of each edge data segment; and is used to read the original data volume identifier and data volume change coefficient code of the edge data segment from the corresponding data packet after the calculation of any computing server is completed, and store the calculation result data of the edge data segment in the corresponding hard disk based on the original data volume identifier and data volume change coefficient code of each edge data segment.

[0013] Furthermore, the edge data acquisition module includes a first data acquisition unit, a second data acquisition unit and a third data acquisition unit; the first data acquisition unit is used to acquire edge data segments for performing aggregation tasks in the computing server; the second data acquisition unit is used to acquire edge data segments for performing model output tasks in the computing server; and the third data acquisition unit is used to acquire edge data segments for performing feature engineering tasks in the computing server.

[0014] Furthermore, the data preprocessing module includes a data volume reading unit, a computing resource requirement acquisition unit and a data segment encoding unit; the data volume reading unit is used to read the original data volume information of each edge data segment; the computing resource requirement acquisition unit is used to search and compare in a preset computing resource requirement database according to the original data volume information and execution task type information of each edge data segment, and obtain the computing resource requirement of each edge data segment; the data segment encoding unit is used to add a computing resource requirement code to the head of each edge data segment according to the computing resource requirement of each edge data segment, and to add an original data volume identifier and a data volume change coefficient code to the tail of each edge data segment according to the execution task type information of each edge data segment, thereby obtaining the encoded data packet of each edge data segment, wherein the original data volume identifier is in front and the data volume change coefficient code is in the back.

[0015] Furthermore, the original data volume is identified as the specific number of bytes of each edge data segment, the computing resource requirement is encoded as the specific numerical value of the computing resource requirement, and the data volume change coefficient is encoded as the specific change multiple of the data volume of each edge data segment after executing the corresponding task.

[0016] Furthermore, the specific processing process of the computing resource demand acquisition unit is as follows:

[0017] S11: For a single edge data segment, obtain the original data volume information and execution task type information obtained by the data volume reading unit. The execution task type information includes executing aggregation tasks, executing model output tasks, and executing feature engineering tasks.

[0018] S12: performing normalization processing on the original data volume information to obtain a normalized value of the original data volume information of the edge data segment;

[0019] S13: Search and compare the preset task calculation complexity coefficient library according to the execution task type information to obtain the task calculation complexity coefficient β of the edge data segment i , calculate the complexity coefficient β for the task i Perform normalization processing to obtain the task calculation difficulty coefficient β of the edge data segment i The normalized value of , where i represents the number of the edge data segment, the task computational complexity coefficient of executing the aggregation task is less than the task computational complexity coefficient of executing the feature engineering task, and the task computational complexity coefficient of executing the feature engineering task is less than the task computational complexity coefficient of executing the model output task;

[0020] S14: Calculate the product of the normalized value of the original data volume information of the edge data segment and the normalized value of the task calculation complexity coefficient to obtain the computing resource requirement score, recorded as Ps i ;

[0021] S15: Score Ps based on the computing resource requirements of the edge data segment i Search and compare in the preset computing resource requirement database to obtain the computing resource requirement CRn of each edge data segment i .

[0022] Furthermore, the task computational complexity coefficient library stores the correspondence between the execution task type information and the task computational complexity coefficient; the computational resource requirement database stores the correspondence between the computational resource requirement score range and the computational resource requirement.

[0023] Furthermore, in the data segment encoding unit, a search and comparison is performed in a preset data volume change coefficient encoding library according to the execution task type information of each edge data segment to obtain the correspondence between the data volume change coefficient encodings corresponding to each execution task type information, wherein, when the execution task type information is to execute an aggregation task, the data volume change coefficient encoding is γ1, when the execution task type information is to execute a model output task, the data volume change coefficient encoding is γ2, and when the execution task type information is to execute a feature engineering task, the data volume change coefficient encoding is γ3, γ1<γ2<γ3.

[0024] Furthermore, the specific processing process of the server monitoring module is as follows:

[0025] S21: Obtain the performance indicators of each computing server through the server management software, including CPU usage U cpuj , memory usage U Memoryj ; where j represents the jth computing server;

[0026] S22: Calculate the available computing resources CRa of each computing server according to the following formula: j :

[0027] CRa j =W1*U cpuj +W2*U Memoryj -CR0

[0028] Among them, W1 and W2 are the weight ratios corresponding to CPU utilization and memory utilization, and CR0 is the reserved computing resource safety amount.

[0029] Furthermore, in the data distribution management module, the specific process of distributing each edge data segment to the corresponding computing server for computing is as follows:

[0030] S301: Reading the computing resource requirement code of each edge data segment from the data packet, thereby obtaining the specific value of the computing resource requirement of each edge data segment, and sorting the edge data segments in descending order according to the specific value of the computing resource requirement;

[0031] S302: Obtain the available computing resources of each computing server from the server monitoring module, eliminate computing servers with available computing resources less than the maximum value according to the specific value of the computing resource demand, and then sort the remaining computing servers in descending order according to the size of the available computing resources;

[0032] S303: Allocate each edge data segment to the remaining computing servers for calculation in sequence according to the specific numerical value of the computing resource requirement, until all edge data segments are calculated and allocated; wherein, the edge data segment with a larger specific numerical value of the computing resource requirement is allocated to the remaining computing servers with a larger amount of available computing resources.

[0033] Furthermore, in the data allocation management module, the specific process of storing the calculation result data of the edge data segment into the corresponding hard disk is as follows:

[0034] S311: After any computing server completes the calculation, the original data volume identifier and data volume change coefficient code of the currently calculated edge data segment are read from the corresponding data packet to obtain the specific number of bytes of the edge data segment and the specific change multiple of the data volume;

[0035] S312: Calculate the product of the specific number of bytes of the edge data segment and the specific change multiple of the data volume, that is, obtain the calculation result data volume of the currently calculated edge data segment;

[0036] S313: Obtain the available capacity of each hard disk from the storage device monitoring module. Based on the specific number of bytes of the calculated result data volume of the currently calculated edge data segment, remove the hard disks whose available capacity is less than the specific number of bytes of the calculated result data volume. Then, sort the remaining hard disks in descending order based on available capacity.

[0037] S314: storing the calculation result data of the edge data segment that has been calculated currently in the remaining hard disk with the maximum available capacity;

[0038] S315: After the calculation is completed on each computing server, the processing from step S311 to step S314 is performed until the edge data segments in all computing servers are calculated.

[0039] Compared with the existing technology, the present invention has the following advantages: the storage system configuration monitoring system based on the smart energy cloud obtains the specific value of the computing resource demand and the specific change multiple of the data volume after executing the corresponding task through the execution task type information of the edge data segment in the computing server; when calculating and allocating the edge data segment, the server monitoring data is combined with the specific value of the computing resource demand to accurately and reasonably allocate the edge data segment, effectively avoiding the excessive load of individual computing servers, and ensuring the operation safety of the system to a certain extent; when storing and allocating the calculation result data of the edge data segment, the storage device monitoring data is combined with the original data volume and the specific change multiple of the data volume after executing the corresponding task to accurately and reasonably allocate the edge data segment, realizing the reasonable and efficient allocation of storage space. In addition, the present invention does not need to read the calculation result data of any edge data segment after the calculation is completed to obtain the data volume information. It can obtain a relatively accurate calculation result data volume only through the specific number of bytes of the edge data segment and the specific change multiple of the data volume, and directly store it, ensuring the security of the calculation result data. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 This is a schematic block diagram of the structure of a storage system configuration monitoring system based on a smart energy cloud in an embodiment of the present invention;

[0041] Figure 2 Schematic diagram of edge data segment execution task types in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] The following is a detailed description of an embodiment of the present invention. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process. However, the protection scope of the present invention is not limited to the following embodiment.

[0043] like Figure 1 As shown, this embodiment provides a technical solution: a storage system configuration monitoring system based on a smart energy cloud, comprising the following modules: an edge data acquisition module, a data preprocessing module, a data transmission module, a server monitoring module, a storage device monitoring module, and a data allocation management module;

[0044] In this embodiment, the edge data acquisition module is used to acquire edge data segments generated by different edge devices;

[0045] As more specific, the edge data acquisition module includes a first data acquisition unit, a second data acquisition unit and a third data acquisition unit; the first data acquisition unit is used to acquire edge data segments for performing aggregation tasks in the computing server; the second data acquisition unit is used to acquire edge data segments for performing model output tasks in the computing server; the third data acquisition unit is used to acquire edge data segments for performing feature engineering tasks in the computing server; the edge data segment execution task type is shown in FIG. Figure 2 .

[0046] To be more specific, in this embodiment, the edge data acquisition module acquires three edge data segments at a time, namely, the edge data segment for executing the aggregation task in the computing server, the edge data segment for executing the model output task in the computing server, and the edge data segment for executing the feature engineering task in the computing server.

[0047] It should be noted that aggregation tasks generally involve operations such as summing and averaging data, resulting in a much smaller amount of data than the original data. Model output tasks typically involve cloud-based deep learning models obtaining classification labels, regression predictions, and other data. Each piece of data corresponds to a calculation result, and the amount of data resulting from the calculation is equal to the original data. Feature engineering tasks generally involve acquiring new features, new fields, temporary tables, and other features from the original data through feature engineering. The amount of data resulting from the calculation is much larger than the original data. The execution task type for edge data segments is determined by their data source. Different edge devices generate edge data segments, and the corresponding types of tasks are executed on the compute server.

[0048] In this embodiment, the data preprocessing module is used to read the original data volume of the acquired edge data segments, obtain the computing resource requirements, and perform data segment encoding processing to obtain encoded data packets, and send the data packets corresponding to each edge data segment to the data transmission module;

[0049] To be more specific, the data preprocessing module includes a data volume reading unit, a computing resource requirement acquisition unit and a data segment encoding unit; the data volume reading unit is used to read the original data volume information of each edge data segment; the computing resource requirement acquisition unit is used to search and compare in a preset computing resource requirement database according to the original data volume information and execution task type information of each edge data segment, and obtain the computing resource requirement of each edge data segment; the data segment encoding unit is used to add a computing resource requirement code to the head of each edge data segment according to the computing resource requirement of each edge data segment, and to add an original data volume identifier and a data volume change coefficient code to the tail of each edge data segment according to the execution task type information of each edge data segment, thereby obtaining the encoded data packet of each edge data segment, wherein the original data volume identifier is in front and the data volume change coefficient code is in the back.

[0050] To be more specific, the original data volume is identified as the specific number of bytes of each edge data segment, the computing resource requirement is encoded as the specific numerical value of the computing resource requirement, and the data volume change coefficient is encoded as the specific change multiple of the data volume of each edge data segment after executing the corresponding task.

[0051] More specifically, the specific processing process of the computing resource demand acquisition unit is as follows:

[0052] S11: For a single edge data segment, obtain the original data volume information and execution task type information obtained by the data volume reading unit. The execution task type information includes executing aggregation tasks, executing model output tasks, and executing feature engineering tasks.

[0053] S12: performing normalization processing on the original data volume information to obtain a normalized value of the original data volume information of the edge data segment;

[0054] S13: Search and compare the preset task calculation complexity coefficient library according to the execution task type information to obtain the task calculation complexity coefficient β of the edge data segment i , calculate the complexity coefficient β for the task i Perform normalization processing to obtain the task calculation difficulty coefficient β of the edge data segment i The normalized value of , where i represents the number of the edge data segment, the task computational complexity coefficient of executing the aggregation task is less than the task computational complexity coefficient of executing the feature engineering task, and the task computational complexity coefficient of executing the feature engineering task is less than the task computational complexity coefficient of executing the model output task;

[0055] S14: Calculate the product of the normalized value of the original data volume information of the edge data segment and the normalized value of the task calculation complexity coefficient to obtain the computing resource requirement score, recorded as Ps i ;

[0056] S15: Score Ps based on the computing resource requirements of the edge data segment i Search and compare in the preset computing resource requirement database to obtain the computing resource requirement CRn of each edge data segment i .

[0057] It should be noted that in step S13, the task calculation complexity coefficient library stores the correspondence between the execution task type information and the task calculation complexity coefficient. The task calculation complexity coefficient in the database is determined according to the number of calculation process steps and intermediate data of each execution task type.

[0058] Specifically, in this embodiment, when the execution task type information is to execute an aggregation task, the task calculation complexity coefficient is 100; when the execution task type information is to execute a model output task, the task calculation complexity coefficient is 1000; when the execution task type information is to execute a feature engineering task, the task calculation complexity coefficient is 500.

[0059] More specifically, in the data segment encoding unit, a search and comparison is performed in a preset data volume change coefficient encoding library according to the execution task type information of each edge data segment to obtain the correspondence between the data volume change coefficient encodings corresponding to each execution task type information, wherein, when the execution task type information is to execute an aggregation task, the data volume change coefficient encoding is γ1, when the execution task type information is to execute a model output task, the data volume change coefficient encoding is γ2, and when the execution task type information is to execute a feature engineering task, the data volume change coefficient encoding is γ3, γ1<γ2<γ3.

[0060] It should be noted that, in step S15, the computing resource requirement database stores the corresponding relationship between the computing resource requirement score range and the computing resource requirement.

[0061] In this embodiment, the data transmission module is used to send data packets corresponding to each edge data segment to the data distribution management module in parallel.

[0062] It should be noted that the data transmission module sends the data packets corresponding to the edge data segments to the data distribution management module in parallel via the wireless network.

[0063] In this embodiment, the server monitoring module is used to obtain the performance indicators of each computing server through the server management software, and calculate the available computing resources of each computing server according to the performance indicators;

[0064] More specifically, the specific processing process of the server monitoring module is as follows:

[0065] S21: Obtain the performance indicators of each computing server through the server management software, including CPU usage U cpuj , memory usage U Memoryj ; where j represents the jth computing server;

[0066] S22: Calculate the available computing resources CRa of each computing server according to the following formula: j :

[0067] CRa j =W1*U cpuj +W2*U Memoryj -CR0

[0068] Among them, W1 and W2 are the weight ratios corresponding to CPU utilization and memory utilization, and CR0 is the reserved computing resource safety amount, which is used to ensure that the computing server does not reach the load threshold.

[0069] In this embodiment, the storage device monitoring module is used to obtain the available capacity of each hard disk through storage management software;

[0070] More specifically, in the storage device monitoring module, the available capacity of each hard disk is recorded as RL k , where k represents the kth hard disk.

[0071] In this embodiment, the data allocation management module is used to obtain the data packet corresponding to each edge data segment, and read the computing resource requirement code of each edge data segment from the data packet. According to the specific value of the computing resource requirement of each edge data segment, each edge data segment is allocated to the corresponding computing server for calculation. After the calculation of any computing server is completed, the module is used to read the original data volume identifier and data volume change coefficient code of the edge data segment from the corresponding data packet. According to the original data volume identifier and data volume change coefficient code of each edge data segment, the module stores the calculation result data of the edge data segment in the corresponding hard disk for storage.

[0072] More specifically, in the data distribution management module, the specific process of distributing each edge data segment to the corresponding computing server for computing is as follows:

[0073] S31: Reading the computing resource requirement code of each edge data segment from the data packet, thereby obtaining the specific value of the computing resource requirement of each edge data segment, and arranging the edge data segments in descending order according to the specific value of the computing resource requirement;

[0074] S32: Obtain the available computing resources of each computing server from the server monitoring module, eliminate computing servers whose available computing resources are less than the maximum value according to the specific value of the computing resource demand, and then sort the remaining computing servers in descending order according to the size of the available computing resources;

[0075] S33: Allocate each edge data segment to the remaining computing servers for computation in order according to the specific numerical value of the computing resource requirement, until all edge data segments have been computed and allocated; wherein, edge data segments with larger specific numerical values ​​of computing resource requirements are allocated to the remaining computing servers with larger available computing resources;

[0076] It should be noted that when performing calculation and allocation work for non-first edge data segments, the calculation servers that have been allocated edge data segments will not be allocated again and will be deleted from the remaining calculation servers.

[0077] More specifically, in the data allocation management module, the specific process of storing the calculation result data of the edge data segment into the corresponding hard disk is as follows:

[0078] S41: After any computing server completes the calculation, the original data volume identifier and the data volume change coefficient code of the currently calculated edge data segment are read from the corresponding data packet to obtain the specific number of bytes of the edge data segment and the specific change multiple of the data volume. The present invention does not need to read the calculation result data of any edge data segment after the calculation is completed to obtain the data volume information. A relatively accurate calculation result data volume can be obtained only by the specific number of bytes of the edge data segment and the specific change multiple of the data volume, and directly stored, thereby ensuring the security of the calculation result data.

[0079] S42: Calculate the product of the specific number of bytes of the edge data segment and the specific change multiple of the data volume, that is, obtain the calculation result data volume of the edge data segment that has been calculated currently;

[0080] S43: Obtain the available capacity of each hard disk from the storage device monitoring module, and based on the specific number of bytes of the calculated result data volume of the currently calculated edge data segment, remove the hard disks whose available capacity is less than the specific number of bytes of the calculated result data volume, and then sort the remaining hard disks in descending order based on the available capacity;

[0081] S44: storing the calculation result data of the edge data segment that has been calculated currently in the remaining hard disk with the maximum available capacity;

[0082] S45: After each computing server completes the calculation, the processing of steps S41 to S44 is performed until the edge data segments in all computing servers are calculated;

[0083] It should be noted that, when performing storage allocation for non-leading edge data segments, the hard disks to which calculation result data for edge data segments have been allocated may be allocated repeatedly.

[0084] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. The storage system configuration monitoring system based on the smart energy cloud is characterized by: include: Edge data acquisition module, data preprocessing module, data transmission module, server monitoring module, storage device monitoring module and data distribution management module; The edge data acquisition module is used to acquire edge data segments generated by different edge devices; The data preprocessing module is used to read the original data volume of the acquired edge data segments, obtain the computing resource requirements, and perform data segment encoding processing to obtain encoded data packets, and send each data packet to the data transmission module; The data transmission module is used to send the data packets corresponding to the edge data segments to the data distribution management module in parallel; The server monitoring module is used to obtain the performance indicators of each computing server and calculate the available computing resources of each computing server; The storage device monitoring module is used to obtain the available capacity of each hard disk through storage management software; The data allocation management module is used to obtain the data packet corresponding to each edge data segment, and read the computing resource requirement code of each edge data segment from the data packet. According to the specific value of the computing resource requirement of each edge data segment, each edge data segment is allocated to the corresponding computing server for calculation; and after the calculation of any computing server is completed, it is used to read the original data volume identifier and data volume change coefficient code of the edge data segment from the corresponding data packet, and according to the original data volume identifier and data volume change coefficient code of each edge data segment, the calculation result data of the edge data segment is stored in the corresponding hard disk for storage; The data preprocessing module includes a data volume reading unit, a computing resource requirement acquisition unit and a data segment encoding unit; the data volume reading unit is used to read the original data volume information of each edge data segment; the computing resource requirement acquisition unit is used to search and compare in a preset computing resource requirement database according to the original data volume information and execution task type information of each edge data segment, and obtain the computing resource requirement of each edge data segment; the data segment encoding unit is used to add a computing resource requirement code to the head of each edge data segment according to the computing resource requirement of each edge data segment, and to add an original data volume identifier and a data volume change coefficient code to the tail of each edge data segment according to the execution task type information of each edge data segment, thereby obtaining the encoded data packet of each edge data segment, wherein the original data volume identifier is in front and the data volume change coefficient code is in the back; The specific processing process of the computing resource demand acquisition unit is as follows: S11: For a single edge data segment, obtain the original data volume information and execution task type information obtained by the data volume reading unit. The execution task type information includes executing aggregation tasks, executing model output tasks, and executing feature engineering tasks. S12: performing normalization processing on the original data volume information to obtain a normalized value of the original data volume information of the edge data segment; S13: Search and compare the preset task calculation complexity coefficient library according to the execution task type information to obtain the task calculation complexity coefficient β of the edge data segment i , calculate the complexity coefficient β for the task i Perform normalization processing to obtain the task calculation difficulty coefficient β of the edge data segment i The normalized value of , where i represents the number of the edge data segment, the task calculation complexity coefficient of executing the aggregation task is less than the task calculation complexity coefficient of executing the feature engineering task, and the task calculation complexity coefficient of executing the feature engineering task is less than the task calculation complexity coefficient of executing the model output task; the task calculation complexity coefficient library stores the correspondence between the execution task type information and the task calculation complexity coefficient; S14: Calculate the product of the normalized value of the original data volume information of the edge data segment and the normalized value of the task calculation complexity coefficient to obtain the computing resource requirement score, recorded as Ps i ; S15: Score Ps based on the computing resource requirements of the edge data segment i Search and compare in the preset computing resource requirement database to obtain the computing resource requirement CRn of each edge data segment i , wherein the computing resource demand database stores the corresponding relationship between the computing resource demand score range and the computing resource demand; In the data segment encoding unit, a search and comparison is performed in a preset data volume change coefficient encoding library according to the execution task type information of each edge data segment to obtain the correspondence between the data volume change coefficient encodings corresponding to each execution task type information, wherein, when the execution task type information is to execute an aggregation task, the data volume change coefficient encoding is γ1, when the execution task type information is to execute a model output task, the data volume change coefficient encoding is γ2, and when the execution task type information is to execute a feature engineering task, the data volume change coefficient encoding is γ3, γ1<γ2<γ3.

2. The storage system configuration monitoring system based on the smart energy cloud according to claim 1 is characterized in that: The edge data acquisition module includes a first data acquisition unit, a second data acquisition unit and a third data acquisition unit; the first data acquisition unit is used to acquire edge data segments for performing aggregation tasks in the computing server; the second data acquisition unit is used to acquire edge data segments for performing model output tasks in the computing server; the third data acquisition unit is used to acquire edge data segments for performing feature engineering tasks in the computing server.

3. The storage system configuration monitoring system based on the smart energy cloud according to claim 1 is characterized in that: The original data volume is identified as the specific number of bytes of each edge data segment, the computing resource requirement is encoded as the specific value of the computing resource requirement, and the data volume change coefficient is encoded as the specific change multiple of the data volume of each edge data segment after executing the corresponding task.

4. The storage system configuration monitoring system based on the smart energy cloud according to claim 1 is characterized in that: The specific processing process of the server monitoring module is as follows: S21: Obtain the performance indicators of each computing server through the server management software, including CPU usage U cpuj , memory usage U Memoryj ; where j represents the jth computing server; S22: Calculate the available computing resources CRa of each computing server according to the following formula: j : CRa j =W1*U cpuj +W2*U Memoryj -CR0 Among them, W1 and W2 are the weight ratios corresponding to CPU utilization and memory utilization, and CR0 is the reserved computing resource safety amount.

5. The storage system configuration monitoring system based on the smart energy cloud according to claim 1 is characterized in that: In the data distribution management module, the specific process of allocating each edge data segment to the corresponding computing server for calculation is as follows: S301: Reading the computing resource requirement code of each edge data segment from the data packet, thereby obtaining the specific value of the computing resource requirement of each edge data segment, and sorting the edge data segments in descending order according to the specific value of the computing resource requirement; S302: Obtain the available computing resources of each computing server from the server monitoring module, eliminate computing servers with available computing resources less than the maximum value according to the specific value of the computing resource demand, and then sort the remaining computing servers in descending order according to the size of the available computing resources; S303: Allocate each edge data segment to the remaining computing servers for calculation in sequence according to the specific numerical value of the computing resource requirement, until all edge data segments are calculated and allocated; wherein, the edge data segment with a larger specific numerical value of the computing resource requirement is allocated to the remaining computing servers with a larger amount of available computing resources.

6. The storage system configuration monitoring system based on the smart energy cloud according to claim 5 is characterized in that: In the data allocation management module, the specific process of storing the calculation result data of the edge data segment into the corresponding hard disk is as follows: S311: After any computing server completes the calculation, the original data volume identifier and data volume change coefficient code of the currently calculated edge data segment are read from the corresponding data packet to obtain the specific number of bytes of the edge data segment and the specific change multiple of the data volume; S312: Calculate the product of the specific number of bytes of the edge data segment and the specific change multiple of the data volume, that is, obtain the calculation result data volume of the currently calculated edge data segment; S313: Obtain the available capacity of each hard disk from the storage device monitoring module. Based on the specific number of bytes of the calculated result data volume of the currently calculated edge data segment, remove the hard disks whose available capacity is less than the specific number of bytes of the calculated result data volume. Then, sort the remaining hard disks in descending order based on available capacity. S314: storing the calculation result data of the edge data segment that has been calculated currently in the remaining hard disk with the maximum available capacity; S315: After the calculation is completed on each computing server, the processing from step S311 to step S314 is performed until the edge data segments in all computing servers are calculated.

Images