Mine cloud cross-layer computing resource self-optimization scheduling method

Abstract

The present application relates to the technical field of computing resource scheduling, and discloses a mine cloud and fog cross-layer computing resource self-optimization scheduling method, which is used to solve the problem of cooperative judgment of whether the cross-layer path is shorter than the fog end and meets the remaining time limit of the task. The scheduler receives a mine computing task, screens the fog end candidate nodes and the cloud end candidate nodes which meet the task resource demand of the available resources of the nodes and the task category of the task execution container, obtains the fog end completion time, the cloud end completion time, the inter-layer transmission time and the task remaining time limit, forms a cross-layer double surplus value, and determines the execution location and the target node. The target node allocates computing resources, the task execution container processes input data and returns mine business results, and after completion, the fog end completion time is updated according to the execution location, or the inter-layer transmission time and the cloud end completion time are updated, and the processor share and the memory amount are released. The present application is used for cross-layer scheduling of mine computing tasks.

Description

Technical Field

[0001] The present invention relates to the technical field of computing resource scheduling, and more specifically, it relates to a mine cloud-fog cross-layer computing resource self-optimization scheduling method. Background Art

[0002] Video surveillance, environmental monitoring, equipment status analysis and coordinated control in mine production systems continuously generate mine computing tasks. Such tasks are usually first delivered by the mine task source to adjacent fog nodes through the mine industrial network. The fog nodes can run the tasks on the site side, or send input data to cloud nodes for processing via the existing uplink. Fog nodes and cloud nodes are generally deployed with task execution containers corresponding to task categories, and the scheduler completes task distribution according to the resource directory, which records the processor, memory, runnable services, network address and online status of the nodes.

[0003] In actual operation, the input data volume, arrival time and latest completion time of mine computing tasks are not fixed, and the execution queue, processor occupancy, memory occupancy and container operation status of fog nodes and cloud nodes will also change. Fog nodes are close to mine task sources, which can reduce the transmission process of input data in the uplink, but the allocable processor share and memory capacity are usually limited; cloud nodes can provide another layer of computing resources, but before the task enters the cloud node, it needs to be forwarded by the fog node and transmitted through the uplink, and the time required to form mine service results is jointly affected by the inter-layer transmission time, cloud queuing and container execution process.

[0004] Common schedulers select target nodes based on node load, node available resources, estimated execution time, queuing status or latest completion time. When the above data are judged separately, the scheduler may issue the task because the cloud node currently has more resources, but the time required for inter-layer transmission and cloud execution is not shorter than the completion time on the fog side; it may also judge that the cross-layer path is faster, but does not check the remaining time limit of the task in combination with the current moment, and the task may still arrive at the cloud node later than the latest completion time. For tasks with large input data volume such as video streams and equipment vibration data, the available bandwidth of the uplink and transmission fluctuations will further amplify such deviations.

[0005] Furthermore, if the scheduler uses fixed estimates or historical averages that are not updated according to task categories for a long time, deviations between the expected results and the current operating status are likely to occur after node restarts, container queue changes, and link fluctuations. If task delivery fails or a node goes offline, and the scheduling queue, resource directory, and task number are not synchronously restored, issues such as unreleased resource records, duplicate task delivery, or inconsistent results may occur. Therefore, task scheduling between the mining end, fog end, and cloud end needs to comprehensively consider node online status, task execution container adaptation, available node resources, fog end completion time, cloud end completion time, inter-layer transmission time, and remaining task time limits within the existing computing channels. Computing resources should be allocated and mining business results returned to the target node. The fog end completion time should be updated according to the execution location, or the inter-layer transmission time and cloud end completion time should be updated, and processor shares and memory should be released to ensure that the selection of the execution location corresponds to the current node and link operating status. Summary of the Invention

[0006] This invention provides a self-optimizing scheduling method for cross-layer computing resources in mine clouds and fog, which solves the technical problems mentioned in the background art.

[0007] To achieve the above-mentioned objectives, the present invention provides the following technical solution.

[0008] A self-optimizing scheduling method for cross-layer computing resources in mine cloud and fog computing, wherein the scheduler executes in an architecture in which mine task sources, fog nodes, and cloud nodes are sequentially connected: Receive mining computing tasks carrying task type and task resource requirements, write them to the scheduling queue, and obtain the remaining time limit of the task. Filter online, task execution containers that correspond to the task category and whose available resources meet the task resource requirements of the fog and cloud candidate nodes; Get the most recent fog completion time, cloud completion time, and inter-layer transmission time for the task category; accumulate the inter-layer transmission time and cloud completion time as the cross-layer path reference time; subtract the cross-layer path reference time from the fog completion time to obtain the local savings; subtract the cross-layer path reference time from the remaining task time limit to obtain the remaining time limit; and use the smaller of the local savings and the remaining time limit as the cross-layer double surplus value. When both fog candidate nodes and cloud candidate nodes coexist, if the cross-layer double redundancy value is positive, the execution location is the cloud; if the cross-layer double redundancy value is non-positive, the execution location is the fog. If there are only fog endpoint candidate nodes, the execution location is the fog endpoint; if there are only cloud endpoint candidate nodes, the execution location is the cloud endpoint. If there are no candidates, the mining computing task remains in the scheduling queue. Mining computation tasks are issued to the target node at the execution location according to the existing layer selection rules; The target node allocates computing resources that meet the task resource requirements to the task execution container, and the task execution container executes the mine computing task and generates a mine service result; When the mine computing task is completed at the fog end, the fog-end completion time consumption is updated, and when the mine computing task is completed at the cloud end, the inter-layer transmission time consumption and the cloud-end completion time consumption are updated.

[0009] Compared with the prior art, the present invention has the following substantial features and significant progress: The present invention relies on the mine task source, fog node, cloud node, task execution container and existing uplink to complete task dispatch. Compared with the methods of making selection based on node load, estimated execution time consumption or latest completion time respectively, the scheduler first filters fog-end candidate nodes and cloud-end candidate nodes according to the online status, the correspondence between the task execution container and the task category, and the available node resources, and then forms a cross-layer double margin value based on the fog-end completion time consumption, cloud-end completion time consumption, inter-layer transmission time consumption and remaining task time limit, so that the determination of the execution position takes into account both whether cross-layer execution saves time and whether the task can be completed within the remaining task time limit; the target node allocates computing resources and the task execution container processes input data, the processor share and memory amount are released after the mine service result is confirmed to be delivered, the fog-end completion time consumption is updated when the task is completed at the fog end, and the inter-layer transmission time consumption and the cloud-end completion time consumption are updated when the task is completed at the cloud end. This helps reduce time-unprofitable cross-layer transmission and invalid task delivery to nodes with insufficient available resources, lowers the possibility that mine computing tasks miss the latest completion time, and enables the scheduling basis for subsequent tasks of the same category to be updated along with the node operation and link transmission status. Brief Description of the Drawings

[0010] Figure 1 is a schematic diagram of the mine cloud-fog cross-layer computing resource self-optimization scheduling architecture and mechanism of the present invention. Detailed Description of the Embodiments

[0011] The following description is made with reference to the accompanying drawings of the specification. It should be understood that the following embodiments are used to illustrate the present invention, not to limit the present invention; those skilled in the art can make equivalent replacements or combinations for the specific implementations.

[0012] This embodiment relates to a mine cloud-fog cross-layer computing resource self-optimization scheduling method, which is applicable to mine computing tasks generated by video monitoring, environmental monitoring, equipment state analysis and linkage control in a mine production system. The mine computing tasks are generated by a mine task source, which may be an existing sensor, camera device, equipment controller or monitoring terminal. The mine task source is connected to adjacent fog nodes through a mine industrial network, and the fog nodes are connected to cloud nodes through existing uplinks in the mine industrial network.

[0013] As shown in Figure 1As shown, the scheduler 200, the scheduling queue 300, the resource directory 400, the fog node 500, the cloud node 600, and the task execution containers 800 and 900 jointly participate in the cross-layer scheduling of mining computing tasks. The scheduler 200 receives mining computing tasks sent by the mining task source 100, writes the mining computing tasks into the scheduling queue 300, and obtains the online status of the fog node 500 and cloud node 600, the task category corresponding to the task execution container, and the available resources of the node from the resource directory 400. The fog node 500 and cloud node 600 respectively deploy task execution containers; task execution container 800 is located inside the fog node 500, and task execution container 900 is located inside the cloud node 600. Both are used to process input data and form mining business results.

[0014] Specifically, the scheduler receives mining computation tasks carrying task categories and resource requirements, and obtains the remaining time limit for the task; the scheduler filters candidate nodes in the fog terminal and candidate nodes in the cloud; the scheduler obtains the most recent fog terminal completion time, cloud terminal completion time, and inter-layer transmission time for the task category; the scheduler forms cross-layer path reference time, local savings, remaining time limit, and cross-layer double redundancy values; when fog terminal candidate nodes and cloud terminal candidate nodes coexist, the scheduler determines the execution location based on the cross-layer double redundancy values; the scheduler issues mining computation tasks to the target node at the execution location according to the existing intra-layer selection rules; the target node allocates computing resources to the task execution container; the task execution container forms the mining business result; after the mining computation task is completed, the scheduler updates the fog terminal completion time, or updates the inter-layer transmission time and cloud terminal completion time according to the execution location.

[0015] This process separates candidate node screening and execution location determination. Candidate node screening confirms whether fog nodes or cloud nodes possess the basic conditions to execute mining computation tasks, while execution location determination determines whether to cross layers and enter the cloud when both fog and cloud candidate nodes coexist. Through this division of labor, the scheduler will not send mining computation tasks to the cloud simply because cloud nodes have abundant resources, nor will it ignore the inter-layer transmission time in the existing uplink and the remaining time limit of the current mining computation task simply because the cloud completion time is short.

[0016] Furthermore, the implementation methods for receiving and scheduling mining computing tasks are as follows: After generating monitoring data, image data, equipment status data, or linkage calculation data, the mine task source forms a mine computing task. A mine computing task carries at least a task category and task resource requirements. The task category indicates the existing business program category to which the mine computing task belongs, such as video recognition, gas data processing, equipment vibration analysis, or linkage instruction calculation. The task resource requirements indicate the computing resources that the target node needs to provide for the mine computing task to run. Task resource requirements include task computing power requirements and task memory requirements. The task computing power requirements limit the processor share that the target node should provide for the task execution container, or the amount of allocable processors under the same metering caliber in the resource catalog. The task memory requirements limit the amount of memory that the target node should provide for the task execution container.

[0017] In one implementation, the mining computation task also carries a latest completion time. The latest completion time is time information generated by the mining task source based on existing mining business processing requirements, indicating the latest point in time when the mining business result should be formed and returned. When the scheduler receives a mining computation task, it records the reception time of the task and reads the current time. To avoid deviations in the calculation of the remaining task time limit due to different device clocks or different time formats, the scheduler converts the latest completion time and the current time to the same time base before obtaining the remaining task time limit.

[0018] The remaining time limit for the task is obtained by subtracting the current time from the latest completion time. The mathematical expression for the remaining time limit is: subscript This represents the mining computation task currently awaiting scheduling. The remaining time limit for the task is neither a historical average nor a fixed empirical value, but rather obtained by the scheduler when making scheduling decisions for the current mining computation task, based on the latest completion time and the current time. Its calculation formula is: ,in, Indicates the remaining time limit for the task, measured in time. This indicates the latest completion time of the current mining calculation task, taken from the mining calculation task; This indicates the current time read by the scheduler, taken from the existing time service of the system where the scheduler resides or the mining computing platform.

[0019] The scheduler writes the received mining computation tasks into the scheduling queue. The scheduling queue has 300 tasks. Figure 1The table shows mining computation tasks that have not yet determined a target node or currently have no fog endpoint or cloud endpoint candidate nodes. The scheduler arranges the mining computation tasks in the scheduling queue according to the order of their reception time; for mining computation tasks with the same reception time, they are arranged according to the order of their latest completion time. The scheduler then sequentially retrieves the mining computation tasks according to this arrangement to filter for fog endpoint and cloud endpoint candidate nodes. This arrangement ensures that mining computation tasks arriving earlier enter the scheduling process first, and in the case of the same reception time, prioritizes processing mining computation tasks with an earlier completion time.

[0020] The purpose of the scheduling queue is to decouple task reception from resource scheduling. When the scheduler fails to find any candidate nodes in the fog or cloud at a given time, the mining computing task will not be discarded or forcibly assigned to a node with insufficient resources; instead, it will remain in the scheduling queue. After the resource catalog is updated, the scheduler retrieves the mining computing tasks from the scheduling queue and then performs the selection of candidate nodes in the fog and cloud. The scheduling queue can be implemented using existing message queues, database queues, or task queues within the mining computing platform.

[0021] Furthermore, the implementation methods for selecting candidate nodes in the fog and cloud are as follows: The resource catalog records the online status of fog nodes and cloud nodes, the task categories corresponding to task execution containers, and the available resources of each node. For example... Figure 1 As shown, fog node 500 and cloud node 600 provide node status and available resources to resource directory 400. Resource directory 400 then provides online status, task categories corresponding to task execution containers, and available resources to node 200. Available resources in the resource directory include available resources for fog nodes and available resources for cloud nodes. Available resources for fog nodes include idle computing power and idle memory on the fog edge, while available resources for cloud nodes include idle computing power and idle memory on the cloud.

[0022] When selecting candidate fog nodes, the scheduler checks each fog node to see if it is online, whether the task execution container on that fog node corresponds to the task category of the mining computation task, and whether the fog node's idle computing power and idle memory meet the task's resource requirements. Fog nodes with idle computing power and idle memory not less than the task's computing power requirement and idle memory not less than the task's memory requirement are identified as candidate fog nodes. The mathematical representation of fog node idle computing power is as follows: The mathematical representation of the task's computing power requirements is as follows: The mathematical representation of fog-end free memory is as follows: The mathematical representation of the task's memory requirements is as follows: The resource conditions for fog candidate nodes can be written as: ,in, Indicates the idle computing power at the fog terminal. This indicates the computing power required for the task.

[0023] ,in, Indicates free memory at the fog end. This indicates the task's memory requirements.

[0024] When selecting candidate cloud nodes, the scheduler checks each node's online status, whether the task execution container on that node corresponds to the task category of the mining computation task, and whether the available computing power and memory in the cloud meet the task's resource requirements. Cloud nodes with available computing power and memory not less than the task's computing power and memory requirements are identified as candidate cloud nodes. The mathematical representation of available computing power in the cloud is as follows: The mathematical representation of free memory in the cloud is: The resource conditions for candidate nodes in the cloud can be written as: ,in, Indicates idle computing power in the cloud. This indicates the computing power required for the task.

[0025] ,in, Indicates free memory in the cloud. This indicates the task's memory requirements.

[0026] The determination of the task category corresponding to the task execution container also includes container version identifier consistency verification, health checks, and consistency confirmation of input data format and mining business result format. The scheduler reads the container version identifiers of the task execution containers on fog nodes and cloud nodes and performs consistency verification on the container version identifiers. The container version identifier can be provided by the task execution container image tag, the version field in the container configuration file, or the existing service registration field of the mining computing platform. The scheduler does not change the business algorithm of the task execution container itself, but only confirms that the task execution containers on fog nodes and cloud nodes for the same task category can process the same format of input data and form the same format of mining business results.

[0027] Health checks can be performed by checking the existing process status of the task execution container, starting probes, or service interface probes. A health check indicates that the task execution container can be started or invoked by the target node and can receive input data. Consistency between the input data format and the mining business result format means that the input data fields, file address referencing methods, task number identification methods, and mining business result return fields for the same task category are consistent across fog nodes and cloud nodes. Only fog nodes and cloud nodes with consistent container version identifiers, passing the health check, and consistent input data and mining business result formats are identified as fog endpoint candidate nodes and cloud endpoint candidate nodes, respectively.

[0028] When performing container version identifier consistency verification, the scheduler compares the task category, task execution container identifier, and container version identifier registered in the resource directory. For task execution containers used for the same task category in fog nodes and cloud nodes, their container version identifier fields should be consistent, and this field should match the field registered for the corresponding task category in the resource directory. If any container version identifier field is missing, inconsistent, or does not match the registered field for the task category, the scheduler will not designate the corresponding node as a fog node candidate or a cloud node candidate.

[0029] During health checks, the health check must include at least the following: the task execution container process is running; the task execution container's service interface returns a success status within the existing timeout period; and the task execution container can receive input data in a format corresponding to the task category. The health check fails if the service interface times out, returns a failure status, the task execution container process is not running, or the task execution container cannot receive input data in a format corresponding to the task category. The input data format must be consistent with the mining business result format and must include at least the task number field, input data address field, execution location field, and mining business result field, all of which must be recognizable by the task execution containers on both fog nodes and cloud nodes. If any field cannot be recognized by either layer of task execution containers, the scheduler will not designate the corresponding node as a fog node candidate or a cloud node candidate.

[0030] When there are no executable nodes in either layer, the scheduler keeps the mining computation task in the scheduling queue and waits for an update in the resource directory. When only fog candidate nodes are available, the execution location is the fog node; when only cloud candidate nodes are available, the execution location is the cloud node. When both fog and cloud candidate nodes exist, the scheduler continues to generate cross-layer double redundancy values ​​and determine the execution location.

[0031] Furthermore, the implementation methods for obtaining on-site time and generating cross-layer dual redundancy values ​​are as follows: The scheduler retrieves the most recent fog-end completion time, cloud-end completion time, and inter-layer transmission time for each task category. Fog-end completion time is the actual time taken for a mining computation task of the same category to complete from entering the execution queue on the fog node to generating a mining business result. Cloud-end completion time is the actual time taken for a mining computation task of the same category to complete from entering the execution queue on the cloud node to generating a mining business result. Inter-layer transmission time is the actual time taken for a mining computation task of the same category to complete from the time the fog node sends input data to the cloud node receiving the input data. All three timestamps are derived from the mining computation platform's existing timestamp records of the task execution and input data transmission processes.

[0032] The mathematical expression for the time taken to complete the fog end is as follows: When the previous mining computation task of the same task category is executed on the fog node, the scheduler records the time when the mining computation task enters the execution queue and the time when the mining business result is formed, and the difference between the two is the fog node completion time. The mathematical representation of the cloud completion time is as follows: When the previous mining computation task of the same task category is executed on a cloud node, the scheduler records the time when the mining computation task enters the execution queue and the time when the mining business result is formed, and the difference between the two gives the cloud completion time. The mathematical representation of inter-layer transmission time is as follows: When the previous mining computation task of the same task category is executed in the cloud, the scheduler records the time when the fog node sends the input data and the time when the cloud node fully receives the input data, and obtains the inter-layer transmission time from the difference between the two.

[0033] Before obtaining the cross-layer path reference time, local savings, remaining time limit, and cross-layer double redundancy values, the scheduler converts the fog terminal completion time, cloud completion time, inter-layer transmission time, and remaining task time limit to the same time unit. This time unit can be seconds, milliseconds, or the unified time unit already present on the mining computing platform.

[0034] When there is no valid record for any of the following for a certain task category: fog endpoint completion time, cloud endpoint completion time, and inter-layer transmission time, the scheduler does not generate cross-layer double redundancy values. In this case, the scheduler first executes the task at the nearest fog endpoint and stores the sample; after the cloud obtains the corresponding record through normal mining computing tasks or existing test tasks, it then enables cross-layer double redundancy value generation for subsequent mining computing tasks of the same task category.

[0035] A valid record must meet at least the following conditions: consistent task category, complete start and end timestamps, non-negative completion time calculated from the start and end timestamps for the fog terminal, cloud terminal, or inter-layer transmission, and the corresponding mining computation task must have already generated mining business results. If any condition is not met, the scheduler will treat it as a missing valid record. When the cloud obtains the corresponding record through an existing test task, the existing test task will still enter the scheduling queue according to the task number, task category, input data address, task resource requirements, and latest completion time, and occupy the computing resources corresponding to the task resource requirements; among these, the existing test task is only used to obtain the cloud terminal completion time and inter-layer transmission time.

[0036] The scheduler accumulates the inter-layer transmission time and the cloud completion time to form the cross-layer path reference time. The mathematical expression for the cross-layer path reference time is as follows: Cross-layer path reference time represents the reference time required for a mining computation task to be executed in the cloud, from the moment input data is sent from the fog node to the moment the cloud node fully receives the input data, and then the task execution container on the cloud node forms the mining business result. The formula for calculating cross-layer path reference time is: ,in, This indicates the reference time for cross-layer paths, measured in time. It represents the time taken for inter-layer transmission, with the unit being time. It is taken from the actual time taken for the most recent input data sent by the fog node to the complete reception of the input data by the cloud node in mining computing tasks of the same task category. It represents the time taken to complete the task in the cloud, measured in time, and is taken from the actual time taken for mining computing tasks of the same task category to be executed from entering the execution queue on the cloud node to forming the mining business result.

[0037] The scheduler calculates the local savings by subtracting the cross-layer path reference time from the completion time at the fog end. The mathematical representation of the local savings is as follows: A positive local saving value indicates that the reference time for cross-layer paths is less than the completion time at the fog endpoint, meaning that using the cloud execution location saves time compared to the fog endpoint execution location. The calculation formula is: ,in, This represents local savings, measured in units of time. This represents the time taken to complete the fog task, measured in time. It is taken from the actual time taken for a mining computation task of the same task category to be executed from the time it entered the execution queue to the time it generated the mining business result at the fog node. This indicates the reference time for cross-layer paths, measured in time.

[0038] Combining the expanded form of cross-layer path reference time, the local savings can also be written as: ,in, Indicates local savings; Indicates the time taken to complete the fogging process; Indicates the time taken for inter-layer transmission; This indicates the time taken to complete the task in the cloud; all items use the same unit of time.

[0039] The scheduler subtracts the cross-layer path reference time from the remaining task time to obtain the remaining time limit. The mathematical expression for the remaining time limit is: A positive remaining time limit indicates that the reference time for the cross-layer path is less than the remaining time limit of the task, meaning that even after using the cloud execution location, the mining business results can still be generated within the remaining time limit of the current mining computation task. The calculation formula is: ,in, It represents the amount of time remaining, with the dimension being time; Indicates the remaining time limit for the task, measured in time. This indicates the reference time for cross-layer paths, measured in time.

[0040] Combining the expanded form of cross-layer path reference time, the remaining time limit can also be written as: ,in, Indicates the amount remaining before the deadline; Indicates the remaining time limit for the task; Indicates the time taken for inter-layer transmission; This indicates the time taken to complete the task in the cloud; all items use the same unit of time.

[0041] The scheduler uses the smaller of the local savings and the remaining time limit as the cross-layer double redundancy value. The mathematical representation of the cross-layer double redundancy value is as follows: The formula for calculating the cross-layer double redundancy value is: ,in, This represents the cross-layer double redundancy value, with the dimension being time; Indicates local savings; Indicates the amount of time remaining.

[0042] Combining the expansion forms of local savings and time-limited remaining amounts, the cross-level double surplus value can also be written as: ,in, Indicates the cross-layer double redundancy value; Indicates the time taken to complete the fogging process; Indicates the time taken for inter-layer transmission; Indicates the time taken for the cloud to complete the task; Indicates the remaining time limit for the task; all items use the same time unit.

[0043] The cross-layer double redundancy value expresses two conditions simultaneously through a smaller value operation. A positive local saving indicates that the reference time for the cross-layer path corresponding to the cloud execution location is less than the completion time on the fog server. A positive time limit remaining indicates that the reference time for the cross-layer path corresponding to the cloud execution location is less than the remaining task time limit. The cross-layer double redundancy value is positive only if both local saving and time limit remaining are positive; if either is negative, the cross-layer double redundancy value is negative. This relationship can be written as: ,in, Indicates the cross-layer double redundancy value; Indicates local savings; Indicates the amount remaining before the deadline; Represents logical AND relation.

[0044] Furthermore, the implementation methods for determining the execution location, target node, and issuing mining computation tasks are as follows: After obtaining candidate nodes in the fog, candidate nodes in the cloud, and cross-layer double redundancy values, the scheduler determines the execution location. The mathematical representation of the execution location is as follows: The execution location indicates the run layer selected by the scheduler for the current mining computation task. When both fog-end candidate nodes and cloud-end candidate nodes coexist, a positive cross-layer double redundancy value indicates the execution location is in the cloud; a non-positive cross-layer double redundancy value indicates the execution location is in the fog-end. The formula for determining the execution location is: ,in, Indicates the execution location; Indicates the cross-layer double redundancy value; This indicates that the execution location is in the cloud; This indicates that the execution location is the fog end.

[0045] When only fog endpoint candidate nodes are available, the execution location is the fog endpoint. When only cloud endpoint candidate nodes are available, the execution location is the cloud endpoint. When there are no fog endpoint or cloud endpoint candidate nodes, the mining computation task remains in the scheduling queue. After determining the execution location, the scheduler only sends the mining computation task to one target node along the determined execution location, and does not send the same mining computation task to both the fog endpoint and the cloud endpoint simultaneously.

[0046] The scheduling period is defined as the time range within which the scheduler performs a single screening of candidate nodes in the fog and cloud environments for the queue to be scheduled. Within the same scheduling period, there may be multiple mining computation tasks with positive cross-layer double redundancy values ​​and executed in the cloud. For these mining computation tasks, the scheduler arranges them in descending order of cross-layer double redundancy values; mining computation tasks with the same cross-layer double redundancy values ​​are arranged in the order of their reception time. The scheduler then sequentially completes target node determination, computational resource allocation, and mining computation task distribution according to the arranged order.

[0047] Within the same scheduling cycle, after the scheduler completes the target node determination and computing resource allocation for each mining computing task, it immediately deducts the corresponding cloud idle computing power and cloud idle memory, or the corresponding fog terminal idle computing power and fog terminal idle memory, from the resource catalog. When subsequent mining computing tasks continue to determine target nodes and allocate computing resources, the available resources of the deducted nodes will be used. If the available resources of the deducted nodes no longer meet the task's resource requirements, the mining computing task will be processed again in the order of fog terminal candidate node and cloud candidate node selection, execution location determination, and target node determination.

[0048] The target node is the specific fog node or cloud node in the execution location that ultimately receives the mining computing task. When the execution location is at the fog end, the target node is determined from the fog end candidate nodes according to the existing intra-layer selection rules; when the execution location is at the cloud end, the target node is determined from the cloud candidate nodes according to the existing intra-layer selection rules. The existing intra-layer selection rules are the rules already in place on the mining computing platform before the execution of this method, which are already used to select specific nodes within the same execution location.

[0049] The existing layer selection rule reads the node identifier, available resources, and online status of each candidate node within the execution location and outputs a target node identifier. After obtaining the target node identifier, the scheduler re-verifies the target node's online status, the task category corresponding to the task execution container, and the node's available resources. If the verification passes, the fog node or cloud node corresponding to that target node identifier is determined as the target node. When the existing layer selection rule returns multiple target node identifiers, the scheduler selects the first target node identifier according to the order of node registration in the resource catalog. If the existing layer selection rule returns no result, the returned target node identifier does not exist, or the re-verification fails, the mining computing task remains in the scheduling queue, waiting for the resource catalog to be updated before re-executing the fog and cloud candidate node selection.

[0050] Fog nodes connected to the mining task source are designated as source-side fog nodes. When the execution location is at the fog end, the source-side fog node can act as a target node to process input data. When the execution location is at the cloud, it is used to obtain input data based on the input data address and forward it to the target node via the existing uplink.

[0051] The scheduler configures task numbers and input data addresses for mining computing tasks. The task number identifies the mining computing task and its corresponding execution instance, while the input data address indicates the location where the target node or source-side fog node obtains the input data. The scheduler sends the task number, execution location, and container start command to the target node. When the execution location is at the fog endpoint, the scheduler sends the input data address to the target node, which then retrieves the input data based on the address. When the execution location is at the cloud endpoint, the scheduler sends the input data address to the source-side fog node, which retrieves the input data based on the address and forwards it to the target node via the existing uplink.

[0052] The input data address must be resolvable by either the target node or the source-side fog node as the location where the input data is stored, and must correspond to the task number. If the target node or the source-side fog node cannot retrieve the input data based on the input data address, it returns this information to the scheduler. The scheduler does not start the corresponding task execution container and writes the mining computation tasks that have not yet returned mining business results to the mining task source back to the scheduling queue. The container startup instruction must include at least the task number, task type, execution location, input data address, task computing power requirements, and task memory requirements, enabling the target node to complete the task execution container startup and computing resource allocation based on the same instruction.

[0053] After receiving the task number, execution location, and container start command, the target node allocates computing resources to the task execution container to meet the task's resource requirements. Specifically, the target node allocates a processor share to the task execution container to meet the task's computing power requirements and a memory amount to meet the task's memory requirements. The processor share and task computing power requirements use the same computing power measurement metric as the resource catalog, and the memory amount and task memory requirements use the same memory measurement metric as the resource catalog. The target node starts the task execution container according to the container start command, and the task execution container is identified by its task number as the execution instance.

[0054] Furthermore, the implementation methods for task execution, mine operation result return, fault handling, and resource release are as follows: After the task execution container starts, it processes the input data and generates mining business results. For mining computation tasks in the video recognition category, the mining business results can be the status of personnel or equipment; for mining computation tasks in the environmental data processing category, the mining business results can be the sensor data processing results; for mining computation tasks in the equipment vibration analysis category, the mining business results can be the equipment operating status; and for mining computation tasks in the linkage command calculation category, the mining business results can be command data that the existing control system can recognize.

[0055] The task execution container returns the task number, execution location, and mining operation results to the mining task source. For example... Figure 1As shown, the mining business results generated by task execution container 800 are returned to the mining task source 100 via fog node 500 and mining industrial network 700; the mining business results generated by task execution container 900 are returned to the mining task source 100 via cloud node 600, existing uplink, fog node 500, and mining industrial network 700. The mining task source identifies the mining computing task to which the mining business results belong based on the task number and confirms receipt of the mining business results.

[0056] The confirmation of mining operations results must include at least the task number and confirmation status. When the confirmation status indicates successful reception, the scheduler triggers the target node to release processor shares and memory, and updates the corresponding fog endpoint idle computing power, fog endpoint idle memory, cloud endpoint idle computing power, or cloud endpoint idle memory in the resource catalog. If the confirmation status is not returned or does not indicate successful reception, the scheduler does not perform the update after resource release, but continues to monitor the return status of mining operations results and task heartbeats.

[0057] The scheduler monitors the startup status of task execution containers and the task heartbeats of mining computing tasks. The startup status indicates whether the task execution container has started according to the container startup instructions, and the task heartbeat indicates whether the mining computing task is still in the execution process within the task execution container. The predetermined heartbeat period can adopt the existing task operation monitoring period of the mining computing platform. The scheduler only reads this period and determines whether a task heartbeat has been received within the predetermined heartbeat period.

[0058] The scheduled heartbeat cycle is read from the existing task execution monitoring configuration of the mining computing platform. The task heartbeat is indexed by the task number and includes at least the task number and the heartbeat sending time. If the scheduler does not receive a task heartbeat for the corresponding task number within a scheduled heartbeat cycle, it determines that a task heartbeat has not been received and restarts the task execution container once on the target node. If, after restarting, a task heartbeat for the corresponding task number is still not received in the next scheduled heartbeat cycle, the scheduler proceeds to release processor share and memory on the target node, exclude offline target nodes, and write the mining computing task back to the scheduling queue.

[0059] When the startup status indicates a startup failure, or if no task heartbeat is received within the predetermined heartbeat period, the scheduler restarts the task execution container once on the target node. If the task still fails to start after restarting or still does not receive a task heartbeat while the target node is online, the target node releases its processor share and memory. When the target node is offline, the scheduler excludes it from either the fog-side candidate node or the cloud-side candidate node. The scheduler writes mining computation tasks that have not yet returned mining business results to the mining task source back to the scheduling queue, re-executes the fog-side and cloud-side candidate node screening, cross-layer dual-residual value generation, execution location determination, and target node determination, and retains the task number.

[0060] The task numbering is retained so that the mining task source can identify the correspondence between the rearranged mining business results and the original mining computation tasks. For mining computation tasks that have already returned mining business results to the mining task source, the scheduler will neither migrate the mining computation task nor reissue it. For mining computation tasks that have not yet returned mining business results, after being written back to the pending scheduling queue, a new assessment is made based on the current resource directory, the current remaining time limit of the task, and the most recent valid completion time of the fog terminal, cloud terminal, and inter-layer transmission.

[0061] After the mining task source confirms receipt of the mining business results, the target node releases its processor share and memory. When the execution location is in the fog, the scheduler updates the target node's fog-end idle computing power and fog-end idle memory to reflect the release of processor share and memory. When the execution location is in the cloud, the scheduler updates the target node's cloud-end idle computing power and cloud-end idle memory to reflect the release of processor share and memory.

[0062] When a mining computation task is completed at the fog node, the scheduler updates the fog node completion time. When a mining computation task is completed at the cloud node, the scheduler updates the inter-layer transmission time and the cloud completion time. During the update, the scheduler records the actual path and timestamp according to the task category. For mining computation tasks completed at the fog node, the fog node completion time is obtained from the actual time it takes for the mining computation task to enter the execution queue at the fog node and form the mining business result. For mining computation tasks completed at the cloud node, the inter-layer transmission time is obtained from the actual time it takes for the fog node to send input data and for the cloud node to fully receive the input data, and the cloud completion time is obtained from the actual time it takes for the mining computation task to enter the execution queue at the cloud node and form the mining business result.

[0063] When a record exceeds the predetermined monitoring period or the corresponding node restarts, the scheduler marks the record as invalid, retrieves the actual sample again, and then enables cross-layer dual-residual value generation. The predetermined monitoring period can be provided by the existing operation and maintenance configuration of the mining computing platform, used to limit the availability of the most recent valid record after changes in node operating status and link transmission conditions. This process avoids the scheduler from using expired fog terminal completion time, cloud completion time, or inter-layer transmission time for extended periods.

[0064] The scheduled monitoring cycle is read from the existing record validity configuration of the mining computing platform. The completion time of the fog terminal, the completion time of the cloud terminal, and the inter-layer transmission time for the same task category are recorded using the completion time of the corresponding mining computing task. When the scheduler reads the most recent valid record, it first checks whether the task category, record status, start and end timestamp integrity, and record time exceed the scheduled monitoring cycle; if the check fails, the scheduler treats it as a missing valid record and does not use the record to generate cross-layer double redundancy values.

[0065] As an optional implementation, the mining task source continuously generates mining computing tasks of the same task category. Both fog nodes and cloud nodes are online, the container version identifiers of the task execution containers are consistent, health checks are passed, and the input data format is consistent with the mining business result format. The idle computing power, idle memory, idle computing power, and idle memory on the fog end, cloud end, and cloud end in the resource catalog all meet the task resource requirements. The scheduler can obtain the most recent fog end completion time, cloud end completion time, and inter-layer transfer time for this task category, and can obtain the remaining time limit for the current mining computing task. At this time, the scheduler generates a cross-layer double redundancy value according to the formula. If the cross-layer double redundancy value is positive, the scheduler selects the cloud; if the cross-layer double redundancy value is non-positive, the scheduler selects the fog end. After the task execution container forms the mining business result, the scheduler updates the corresponding application time record according to the execution location.

[0066] As an optional implementation, the fog-end completion time for a certain task category is relatively long, while the sum of the cloud completion time and inter-layer transmission time is relatively short, and the remaining time limit for the current mining computation task is still greater than the cross-layer path reference time. The scheduler calculates that the local savings are positive, and the remaining time limit is also positive; therefore, the cross-layer double surplus value is positive. This result indicates that the current mining computation task entering the cloud not only saves time relative to the fog-end execution location but also does not exceed the remaining task time limit. The scheduler determines the execution location as the cloud. The source-side fog node obtains the input data based on the input data address and forwards it to the target node in the cloud via the existing uplink. The task execution container in the cloud processes the input data and returns the mining business result.

[0067] As an optional implementation, two scenarios may occur: The first scenario is that the cross-layer path reference time is less than the fog-end completion time, but not less than the remaining task time limit. In this case, the local savings are positive, the remaining time limit is non-positive, and the cross-layer double redundancy is also non-positive. The second scenario is that the cross-layer path reference time is less than the remaining task time limit, but not less than the fog-end completion time. In this case, the local savings are non-positive, the remaining time limit is positive, and the cross-layer double redundancy is still non-positive. In both scenarios, the scheduler determines the execution location as the fog end to avoid cross-layer transmissions with no time gain or to avoid selecting cross-layer paths that, although faster, cannot meet the remaining task time limit.

[0068] As an optional implementation, if the idle computing power on the fog endpoint is less than the task's computing power requirement, or the idle memory on the fog endpoint is less than the task's memory requirement, then the corresponding fog node will not be included in the fog endpoint candidate node list. If the idle computing power on the cloud is less than the task's computing power requirement, or the idle memory on the cloud is less than the task's memory requirement, then the corresponding cloud node will not be included in the cloud candidate node list. When only one layer has candidate nodes, the scheduler directly determines the execution location as that layer. When neither fog endpoint candidate nodes nor cloud endpoint candidate nodes exist, the mining computing task remains in the scheduling queue, and the fog endpoint candidate nodes and cloud endpoint candidate nodes are re-selected after the resource catalog is updated.

[0069] As an optional implementation, when a task category enters the scheduling system for the first time, the scheduler cannot simultaneously obtain the most recent fog endpoint completion time, cloud endpoint completion time, and inter-layer transmission time for that task category. In this case, the scheduler does not generate cross-layer double redundancy values, but instead executes the task based on the nearest fog endpoint and stores the sample. Subsequently, after the cloud endpoint completion time and inter-layer transmission time for that task category are obtained through normal mining computation tasks or existing test tasks, the scheduler then enables cross-layer double redundancy value generation for subsequent mining computation tasks of that task category.

[0070] As an optional implementation, the target node has identified and received the task number, execution location, and container start instruction, but the task execution container's start status indicates a start failure, or the scheduler has not received a task heartbeat within a predetermined heartbeat period. The scheduler restarts the task execution container once on the target node. If it still fails to start after restarting or still does not receive a task heartbeat and the target node is online, the target node releases its processor share and memory. If the target node is offline, the scheduler excludes the target node from the fog candidate nodes or cloud candidate nodes. For mining computing tasks that have not yet returned mining business results to the mining task source, the scheduler writes them back to the scheduling queue and re-executes the scheduling process using the task number.

[0071] It should be noted that this implementation method can be deployed in the existing hierarchical computing environment of the mine. The mine task source, fog nodes, cloud nodes, scheduler, scheduling queue, resource catalog, task execution container, and mine industrial network can all be hosted by existing hardware and software of the mine computing platform. The scheduler can be deployed on either fog nodes or cloud nodes, as long as it can read the status of nodes at both layers and the on-site time records, and can distribute mining computing tasks to both layers. The scheduling queue can be provided by existing task queues or message queues, and the resource catalog can be provided by existing resource registration services or database records.

[0072] The task execution container can be a one-time container or a long-running container. A one-time container stops and releases its processor share and memory after the mining business result is confirmed and delivered; a long-running container retains its process after the mining business result is confirmed and delivered, only returning the processor share and memory occupied by the current mining computing task. The task execution container runs existing mining business programs; this implementation does not change the business processing logic of the mining business programs for input data.

[0073] The computations performed by the scheduler can be completed by the scheduling service process in a regular server, industrial computer, or mining computing platform.

[0074] The mining industrial network connects mining task sources, fog nodes, and cloud nodes. When the execution location is at the fog node, input data is sent from the mining task source to the task execution container at the fog node via the mining industrial network. When the execution location is at the cloud node, input data is sent from the mining task source to the source-side fog node via the mining industrial network, and then forwarded from the source-side fog node to the task execution container at the cloud node via the existing uplink. Mining business results are returned to the mining task source according to the task number and execution location.

[0075] Although embodiments of this application have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of this application, and all such modifications and variations fall within the scope defined by the appended claims.

Claims

1. A self-optimizing scheduling method for cross-layer computational resources in mine clouds and fog, characterized in that, The scheduler executes in an architecture where mining task sources, fog nodes, and cloud nodes are connected sequentially: Receive mining computing tasks carrying task type and task resource requirements, write them to the scheduling queue, and obtain the remaining time limit of the task. Filter online, task execution containers that correspond to the task category and whose available resources meet the task resource requirements of the fog and cloud candidate nodes; Get the most recent fog completion time, cloud completion time, and inter-layer transmission time for the task category; accumulate the inter-layer transmission time and cloud completion time as the cross-layer path reference time; subtract the cross-layer path reference time from the fog completion time to obtain the local savings; subtract the cross-layer path reference time from the remaining task time limit to obtain the remaining time limit; and use the smaller of the local savings and the remaining time limit as the cross-layer double surplus value. When both fog candidate nodes and cloud candidate nodes coexist, if the cross-layer double redundancy value is positive, the execution location is the cloud; if the cross-layer double redundancy value is non-positive, the execution location is the fog. If there are only fog endpoint candidate nodes, the execution location is the fog endpoint; if there are only cloud endpoint candidate nodes, the execution location is the cloud endpoint. If there are no candidates, the mining computing task remains in the scheduling queue. Mining computation tasks are issued to the target node at the execution location according to the existing layer selection rules; The target node allocates computing resources to the task execution container to meet the task resource requirements. The task execution container executes the mining computing tasks and generates mining business results. When a mining computation task is completed at the fog terminal, the fog terminal completion time is updated; when it is completed at the cloud terminal, the inter-layer transmission time and the cloud terminal completion time are updated.

2. The self-optimization scheduling method for cross-layer computational resources in mine cloud and fog as described in claim 1, characterized in that, The mining computation tasks sent by the mining task source also carry the latest completion time; the scheduler records the reception time of the mining computation tasks and reads the current time, and subtracts the current time from the latest completion time to obtain the remaining time limit of the task; the scheduler arranges the mining computation tasks in the queue to be scheduled according to the order of reception time, and the mining computation tasks with the same reception time are arranged according to the order of latest completion time, and the scheduler obtains the mining computation tasks in the order of arrangement to screen the fog terminal candidate nodes and the cloud terminal candidate nodes.

3. The self-optimization scheduling method for cross-layer computational resources in mine cloud and fog as described in claim 1, characterized in that, The fog node completion time is the actual time taken for mining computation tasks of the same task category to be executed from the time they entered the execution queue to the time they formed the mining business result at the fog node. The time taken to complete the task in the cloud is the actual time taken for a mining computing task of the same task category to be executed from the time it enters the execution queue on the cloud node to the time it generates a mining business result. Interlayer transmission time is the actual time it takes for a mining computing task of the same task category to receive input data completely from the cloud node after the fog node last sent the input data.

4. The self-optimization scheduling method for cross-layer computational resources in mine cloud and fog as described in claim 1, characterized in that, Task resource requirements include task computing power requirements and task memory requirements. Fog node available resources include fog endpoint idle computing power and fog endpoint idle memory. Cloud node available resources include cloud endpoint idle computing power and cloud endpoint idle memory. Fog nodes with fog endpoint idle computing power not less than the task computing power requirements and fog endpoint idle memory not less than the task memory requirements are identified as fog endpoint candidate nodes. Cloud nodes with cloud endpoint idle computing power not less than the task computing power requirements and cloud endpoint idle memory not less than the task memory requirements are identified as cloud endpoint candidate nodes.

5. A method for self-optimizing scheduling of cross-layer computational resources in mine cloud and fog as described in claim 2, characterized in that, Before obtaining the remaining time limit for the task, the latest completion time and the current time are converted to the same time base; before obtaining the cross-layer path reference time, local savings, remaining time limit and cross-layer double surplus value, the fog terminal completion time, cloud terminal completion time, inter-layer transmission time and remaining task time limit are converted to the same time unit.

6. The self-optimization scheduling method for cross-layer computational resources in mine clouds and fog according to claim 2, characterized in that, The scheduling period is defined as the time range within which the scheduler performs a screening of candidate nodes in the fog end and the cloud end of the queue to be scheduled. Within the same scheduling period, multiple mining computing tasks with positive cross-layer double redundancy values ​​and execution locations in the cloud end are arranged in descending order of cross-layer double redundancy values. Mining computing tasks with the same cross-layer double redundancy values ​​are arranged in the order of their reception time. The target node determination, computing resource allocation, and mining computing task distribution are completed in the order of arrangement.

7. The self-optimization scheduling method for cross-layer computational resources in mine cloud and fog as described in claim 1, characterized in that, The scheduler reads the container version identifiers of the task execution containers on the fog nodes and cloud nodes, performs consistency checks on the container version identifiers, and performs health checks on the task execution containers. Only fog nodes and cloud nodes with consistent container version identifiers, pass the health checks, and have consistent input data formats and mining business result formats are identified as fog node candidates and cloud node candidates, respectively.

8. A self-optimizing scheduling method for cross-layer computational resources in mine cloud and fog as described in claim 4, characterized in that, Fog nodes connected to the mine task source are designated as source-side fog nodes. Task numbers and input data addresses are configured for the mine computing tasks. The scheduler sends task numbers, execution locations, and container startup instructions to the target node. When the execution location is at the fog edge, the scheduler sends the input data address to the target node, which then retrieves the input data. When the execution location is at the cloud, the scheduler sends the input data address to the source-side fog node, which retrieves the input data and forwards it to the target node via the existing uplink. The target node allocates a processor share sufficient to meet the task's computing power requirements and a memory amount sufficient to meet the task's memory requirements to the task execution container, and starts the task execution container according to the container startup instructions. The task execution container identifies the execution instance with the task number, processes the input data, and returns the task number, execution location, and mine business results to the mine task source.

9. A self-optimizing scheduling method for cross-layer computational resources in mine cloud and fog as described in claim 8, characterized in that, The scheduler monitors the startup status of the task execution container and the task heartbeat of the mining computing task; the startup status indicates that if the startup fails or if no task heartbeat is received within the predetermined heartbeat period, the task execution container will be restarted once on the target node. If the target node fails to start after a restart or still does not receive a task heartbeat while the target node is online, the target node releases its processor share and memory. When the target node is offline, the scheduler excludes the target node from either the fog candidate node or the cloud candidate node. The scheduler writes the mining computation tasks that have not yet returned mining business results to the mining task source back to the scheduling queue, re-executes the selection of candidate nodes at the fog end and the cloud end, the generation of cross-layer double surplus values, the determination of execution location and the determination of target node, and uses the task number again.

10. A self-optimizing scheduling method for cross-layer computational resources in mine clouds and fog according to claim 8, characterized in that, After the mining task source confirms receipt of the mining business results, the target node releases its processor share and memory. When the execution location is at the fog endpoint, the scheduler updates the target node's fog endpoint idle computing power and fog endpoint idle memory to reflect the release of processor share and memory. When the execution location is at the cloud endpoint, the scheduler updates the target node's cloud endpoint idle computing power and cloud endpoint idle memory to reflect the release of processor share and memory.

Images