A device capacity expansion and contraction control method, system, device and storage medium

Abstract

The application discloses a device expansion and contraction control method, system and device, and a storage medium. The method comprises the following steps: obtaining first information of each device; determining a first state of each device; determining a second state of each task; if there is a first task with the second state being waiting for scheduling, and there is a first device set with the first state being idle, scheduling the first device to execute the first task according to an expansion strategy, updating the state of the first task to be in scheduling, and updating the state of the first device to be in expansion; or, if there is a second device set with the first state being in work, performing a contraction operation on the second device according to a contraction strategy, updating the state of the second task to be waiting for scheduling, and updating the state of the second device to be in contraction; and if the second device completes the contraction operation, updating the state of the second device to be idle. The application can realize the real-time performance of the expansion and contraction operation, improve the resource utilization, and can be widely applied to the expansion and contraction technical field.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of capacity expansion and contraction, and particularly relates to a device capacity expansion and contraction control method, system and device and storage medium. BACKGROUND

[0002] In the normal operation process of a drilling ship, a large number of sensor devices, camera devices, infrared devices and other professional devices are usually required to participate in work. These devices usually have certain storage and computing capabilities and are distributed in different operation areas, such as a large number of devices arranged on a drilling rack, a workbench and the like. In the case of typhoons, sea waves and other extreme conditions at sea, the devices are more likely to be damaged than on land and have a shorter service life. Therefore, the devices on the drilling ship are usually designed with redundancy. In daily work, the devices need to be expanded and contracted to ensure the orderly development of work and maximize the use of resources. In the related art, the operation status of the devices is mainly observed manually, and then the drilling operation devices (such as sensors, cameras and the like) are manually scheduled, which cannot evaluate the performance of the devices and cannot achieve timely expansion and contraction, resulting in low resource utilization. SUMMARY

[0003] The present application aims to at least partly solve one of the problems in the prior art.

[0004] To this end, the present application aims to provide a quick and practical device capacity expansion and contraction control method, system, device and storage medium.

[0005] To achieve the above technical purpose, the technical solution adopted by the embodiments of the present application comprises:

[0006] On the one hand, the embodiments of the present application provide a device capacity expansion and contraction control method, comprising the following steps:

[0007] The device scaling control method of this invention includes: acquiring first information of each device; the first information including CPU utilization and memory utilization; determining a first state of each device; the first state including working, idle, offline, scaling up, and scaling down; determining a second state of each task; the second state including waiting to be scheduled, scheduling, executing, and completed; if there is a first task whose second state is waiting to be scheduled, and there is a first set of devices whose first state is idle, scheduling the first device to execute the first task according to a scaling strategy, updating the state of the first task to scheduling, and updating the state of the first device to scaling up; the first set of devices includes the first... The device, wherein the expansion strategy includes a strategy of selecting the first device from the first device set based on the first information; the first device is used to represent a device in an idle state; or, if there is a second device set whose first state is working, the second device is scaled down according to the scaling-down strategy, and the state of the second task is updated to waiting for scheduling, and the state of the second device is updated to scaling down; if the second device completes the scaling-down operation, the state of the second device is updated to idle; the second device set includes the second device, the second task is a task scheduled to the second device, and the scaling-down strategy is used to represent whether to perform a scaling-down operation on the second device based on the first information of the second device. This application embodiment improves the accuracy of evaluation by determining device information, device state, and task state, and evaluating devices and tasks based on real-time information; and by performing expansion and scaling-down scheduling based on device information, it can achieve real-time expansion and scaling-down operations and improve resource utilization.

[0008] In addition, the device expansion / reduction control method according to the above embodiments of the present invention may also have the following additional technical features:

[0009] Furthermore, in the device scaling control method of this embodiment, the step of scheduling the first device to execute the first task according to the scaling strategy includes:

[0010] Obtain the first information of each first device in the first device set;

[0011] Based on the first information, determine the weight of each first device;

[0012] The devices are sorted according to their weights from largest to smallest or smallest to largest, and the first device in the sorted order is determined to be the first device that needs to be expanded.

[0013] Furthermore, in one embodiment of the present invention, the first information of the first device includes a first CPU utilization rate and a first memory utilization rate; the method determines the first weight of the first device through the following steps:

[0014] The first probability is determined by using the first value as the base and the second value as the exponent; the second value is related to the first CPU utilization rate.

[0015] Using the first value as the base and the third value as the exponent, a second probability is determined; the third value is related to the first memory usage rate.

[0016] The first probability and the second probability are averaged to determine the first weight.

[0017] Furthermore, in one embodiment of the present invention, the method further includes the following steps:

[0018] Get the first timestamp of the last expansion or contraction operation;

[0019] If the time between the current moment and the first timestamp is less than the first threshold, no expansion or contraction operation will be performed.

[0020] Furthermore, in one embodiment of the present invention, the method further includes:

[0021] If the second task is completed, update the status of the second device to "idle" and update the status of the second task to "completed".

[0022] Alternatively, if the second device completes the power-on or restart operation, update the status of the second device to idle.

[0023] Furthermore, in one embodiment of the present invention, the method further includes:

[0024] Receive the first information of the current moment sent by the second device and store it in the database;

[0025] Obtain the first information of the second device at the previous moment from the database. If the time between the previous moment and the current moment is greater than a second threshold, update the status of the second device to offline, update the status of the task being performed by the second device to waiting for scheduling, and issue a first fault warning message.

[0026] Furthermore, in one embodiment of the present invention, the method further includes:

[0027] Record the second information of the second device; the second device is used to identify the newly added device, and the second information includes network address information and device address information;

[0028] Receive the third information sent by the second device when it is powered on; if the third information is inconsistent with the second information, issue a second fault warning message.

[0029] On the other hand, embodiments of the present invention propose a control system for scaling up or down equipment, comprising:

[0030] The first module is used to obtain first information from each device; the first information includes CPU utilization and memory utilization.

[0031] The second module is used to determine the first state of each device; the first state includes working, idle, offline, expanding capacity, and shrinking capacity.

[0032] The third module is used to determine the second state of each task; the second state includes waiting for scheduling, scheduling, executing, and completed.

[0033] The fourth module is used to schedule the first task to execute the first task according to a capacity expansion strategy if there is a first task in the second state of waiting to be scheduled and there is a first set of devices in the first state of being idle, and to update the status of the first task to be scheduled and update the status of the first device to be expanded; the first set of devices includes the first device, and the capacity expansion strategy includes a strategy of selecting the first device from the first set of devices according to the first information; the first device is used to represent a device in the state of being idle;

[0034] The fifth module is used to, if there is a second device set in the first state of being in operation, perform a scaling-down operation on the second device according to the scaling-down strategy, update the status of the second task to waiting for scheduling, and update the status of the second device to scaling down; if the second device completes the scaling-down operation, update the status of the second device to being idle; the second device set includes the second device, the second task is a task scheduled to the second device, and the scaling-down strategy is used to characterize whether to perform a scaling-down operation on the second device based on the first information of the second device.

[0035] On the other hand, embodiments of the present invention provide a control device for expanding or shrinking equipment, comprising:

[0036] At least one processor;

[0037] At least one memory for storing at least one program;

[0038] When the at least one program is executed by the at least one processor, the at least one processor implements the above-described device scaling control method.

[0039] On the other hand, embodiments of the present invention provide a storage medium storing a processor-executable program, which, when executed by a processor, is used to implement the above-described device scaling control method.

[0040] The scaling control method proposed in this invention includes the following steps: acquiring first information for each device; the first information includes CPU utilization and memory utilization; determining a first state for each device; the first state includes working, idle, offline, scaling up, and scaling down; determining a second state for each task; the second state includes waiting to be scheduled, scheduling, executing, and completed; if there is a first task whose second state is waiting to be scheduled, and there is a first set of devices whose first state is idle, scheduling the first device to execute the first task according to the scaling strategy, updating the state of the first task to scheduling, and updating the state of the first device to scaling up; the first set of devices includes the first device... The expansion strategy includes a strategy of selecting the first device from the first device set based on the first information; the first device represents a device in an idle state; or, if there is a second device set whose first state is working, the second device is scaled down according to the scaling-down strategy, and the state of the second task is updated to waiting for scheduling, and the state of the second device is updated to scaling down; if the second device completes the scaling-down operation, the state of the second device is updated to idle; the second device set includes the second device, the second task is a task scheduled to the second device, and the scaling-down strategy represents whether to perform a scaling-down operation on the second device based on the first information of the second device. This application embodiment improves the accuracy of evaluation by determining device information, device state, and task state, and evaluating devices and tasks based on real-time information; and by performing expansion and scaling-down scheduling based on device information, it can achieve real-time expansion and scaling-down operations and improve resource utilization. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following description is provided with accompanying drawings of the relevant technical solutions in the embodiments of the present invention or the prior art. It should be understood that the accompanying drawings described below are only for the purpose of clearly illustrating some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0042] Figure 1 A flowchart illustrating an embodiment of the device expansion / reduction control method provided by the present invention;

[0043] Figure 2 A schematic flowchart of another embodiment of the device expansion / reduction control method provided by the present invention;

[0044] Figure 3 A schematic flowchart of one embodiment of the capacity expansion process provided by the present invention;

[0045] Figure 4A schematic diagram of one embodiment of device state switching provided by the present invention;

[0046] Figure 5 A schematic flowchart of another embodiment of the capacity expansion process provided by the present invention;

[0047] Figure 6 A schematic diagram of an embodiment of task state switching provided by the present invention;

[0048] Figure 7 A schematic flowchart of one embodiment of the state change process provided by the present invention;

[0049] Figure 8 A schematic flowchart of one embodiment of the volume reduction process provided by the present invention;

[0050] Figure 9 A schematic diagram of one embodiment of the device scaling control system provided by the present invention;

[0051] Figure 10 This is a schematic diagram of one embodiment of the control device for expanding and shrinking the capacity of the equipment provided by the present invention. Detailed Implementation

[0052] The embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. The step numbers in the following embodiments are set only for ease of explanation, and there is no limitation on the order between the steps. The execution order of each step in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0053] A drilling vessel is a ship specifically designed for drilling operations on seabed geological structures, primarily used for marine geological exploration.

[0054] The Transmission Control Protocol (TCP) is a connection-oriented, reliable, byte-stream-based transport layer communication protocol. TCP is designed to adapt to layered protocol hierarchies that support multiple network applications. Pairs of processes in host computers connected to different but interconnected computer networks rely on TCP for reliable communication services. TCP assumes it can obtain simple, potentially unreliable datagram services from lower-level protocols. In principle, TCP should be able to operate over a variety of communication systems, from hardwired connections to packet-switched or circuit-switched networks.

[0055] UDP (User Datagram Protocol) is a connectionless transport layer protocol that provides a simple, unreliable, transaction-oriented message delivery service.

[0056] A MAC address (Media Access Control Address) is a unique network identifier for each device on a network. This address, called the MAC address or network card address, is written into the hardware by the network device manufacturer during production. A MAC address is 48 bits (6 bytes), typically represented as 12 hexadecimal numbers separated by colons, such as 08:00:20:0A:8C:6D. As shown in the diagram, the first 3 bytes represent the OUI (Organizationally Unique Identifier), a code assigned by the IEEE registration authority to different manufacturers to distinguish them. The last 3 bytes are assigned by the manufacturer.

[0057] Capacity expansion refers to the process of increasing capacity or scale. For services provided by an application, as user demand increases, existing devices or applications may become overloaded and unable to provide the expected quality of service. Therefore, capacity expansion is necessary to ensure service quality. Capacity expansion can be simply understood as adding devices and applications.

[0058] Downsizing refers to the process of reducing capacity or scale. For services provided by an application, as user demand decreases, existing devices or applications experience low load. To conserve resources and reduce energy consumption, downsizing is necessary to improve resource utilization. Simply put, downsizing means reducing the number of devices and applications.

[0059] Background of Drilling Equipment Expansion and Reduction: During normal operations on a drilling vessel, a large number of specialized equipment, such as sensors, cameras, and infrared devices, are typically required. These devices usually have storage and computing capabilities and are widely distributed across various operational equipment, such as the drilling rig and work platform. This is done for two reasons: first, to obtain operational information from multiple angles; and second, for mutual backup, ensuring that the failure of one device does not disrupt the operation of others. This differs from land-based operations, where extreme conditions such as typhoons and high waves make equipment more susceptible to damage and have shorter lifespans. Furthermore, offshore operations are typically very demanding, making replacement and repair impractical, as is the case in drilling scenarios. Therefore, a large number of devices (with redundancy) are usually deployed in advance. Additionally, offshore operations are characterized by short, high-frequency work, such as working continuously for a day after determining the drilling site. Considering the scarce and limited power supply resources on drilling vessels, using all equipment (sensors, cameras, infrared devices, etc.) for every operation would consume enormous amounts of power.

[0060] Current technical solutions primarily rely on manual observation of equipment operation and manual scheduling of drilling equipment (such as sensors and cameras). This approach fails to assess equipment performance or enable timely scaling up or down. Specifically, relying on manual observation and scheduling of drilling equipment (such as sensors and cameras) is inefficient and lacks automation. Real-time equipment performance assessment is impossible. For example, if equipment experiences a sudden malfunction (manifested as high load), existing methods rely on observing whether the equipment is functioning normally, but this could be detected much earlier using indicators such as CPU utilization. Timely scaling up or down is also impossible; manual scheduling is required after equipment malfunctions, preventing proactive scheduling before malfunctions occur. Furthermore, this leads to significant waste of resources such as electricity, which are scarce for ships navigating the oceans.

[0061] Understandably, the ideal scenario is to set a fixed number of devices for each task, but considering the varying intensity of different tasks and the performance of the equipment, manually maintaining the number of devices becomes very difficult. Therefore, a solution is needed that can dynamically evaluate equipment performance and automatically add or remove devices to ensure task quality.

[0062] The control method and system for expanding and shrinking equipment according to embodiments of the present invention will be described in detail below with reference to the accompanying drawings. First, the control method for expanding and shrinking equipment according to embodiments of the present invention will be described with reference to the accompanying drawings.

[0063] Reference Figure 1This invention provides a method for controlling device scaling up and down. This method can be applied to terminals, servers, or software running on either terminal or server. Terminals can be tablets, laptops, desktop computers, etc., but are not limited to these. Servers can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The method for controlling device scaling up and down mainly includes the following steps:

[0064] S100: Obtain the first information of each device; the first information includes CPU utilization and memory utilization.

[0065] S200: Determine the first state of each device; the first state includes working, idle, offline, expanding capacity, and shrinking capacity.

[0066] S300: Determine the second state of each task; the second state includes waiting for scheduling, scheduling, executing, and completed.

[0067] S400: If there is a first task in the second state of waiting to be scheduled, and there is a first set of devices in the first state of being idle, schedule the first device to execute the first task according to the expansion strategy, and update the status of the first task to be scheduled, and update the status of the first device to be expanded; the first set of devices includes the first device, and the expansion strategy includes the strategy of selecting the first device from the first set of devices according to the first information; the first device is used to represent the device in the state of being idle;

[0068] S500: If there is a second set of devices in the first state of being in operation, perform a scaling-down operation on the second devices according to the scaling-down strategy, update the status of the second task to waiting for scheduling, and update the status of the second devices to scaling down; if the second device completes the scaling-down operation, update the status of the second device to being idle; the second set of devices includes the second devices, the second task is the task scheduled to the second devices, and the scaling-down strategy is used to characterize whether to perform a scaling-down operation on the second devices based on the first information of the second devices.

[0069] In some possible implementations, refer to Figure 2 As shown, the control method provided in this application embodiment is mainly used to achieve the following aspects:

[0070] 1) Drilling equipment actively registers its information with the scheduling server;

[0071] 2) The scheduling server evaluates the performance information of drilling equipment in real time based on CPU utilization and remaining memory.

[0072] 3) The scheduling server dynamically expands and shrinks its capacity based on the task information to ensure that the task is completed as expected.

[0073] Specifically, refer to Figure 2 The illustrated embodiment follows the process as follows:

[0074] Step 1: The device registers its information with the scheduling server. Whenever a new device is added, the ship's scheduling server records its information in the database. The ship's network is actually a local area network (LAN), configured with a network gateway to control internet access. Additionally, new devices configure the scheduling server's IP address upon startup, allowing the device to verify the server's information. Each time a device powers on (e.g., during a restart), it proactively registers its information with the scheduling server, including its MAC address, LAN IP address, and geographical location. Upon receiving the registration request, the scheduling server checks and compares the MAC address with the information in the database. If the information matches, the server replies with a successful registration; otherwise, it notifies maintenance personnel that the device may be misconfigured or malfunctioning.

[0075] Step 2: The scheduling server broadcasts information to collect current device operating data. The scheduling server periodically broadcasts data to all devices using the UDP protocol. This primarily notifies devices to proactively register their information or requests real-time operating data. To prevent proactive registration requests from being ignored in extreme cases (such as a system crash during registration), the broadcast frequency is configurable, typically once per minute.

[0076] Step 3: Devices proactively respond to broadcasts or periodically reply with their current operating information. When a device receives a broadcast message, it checks if it has successfully registered. If not, it proactively requests registration information from the scheduling server. Otherwise, it proactively reports its operating information, including the average CPU utilization over the past 30 seconds and the current memory utilization (configurable, default is 30 seconds). After successful registration, the device proactively sends real-time operating information to the scheduling server via UDP protocol every 1 second (configurable). In general, the scheduling server holds real-time operating information for all online devices. By comparing device operating information with database information, the scheduling server can identify offline devices. Generally, if no device operating information is received within 3 minutes (configurable), the server suspects a device malfunction and proactively notifies maintenance personnel. Offline devices are then declared offline, and subtasks assigned to offline devices are reset to an unscheduled state.

[0077] Step 4: Task Initiation. Whenever operations are required at sea, technical personnel will pre-set the task, specifying the minimum number of devices (minNeedDevice) and the maximum number (maxNeedDevice). If maxNeedDevice is not set, the default is maxNeedDevice = 1.5 * minNeedDevice. The task start and end times are also typically set. For example, if the task requires a minimum of 4 devices (i.e., minNeedDevice = 4), then the default maxNeedDevice = 1.5 * minNeedDevice = 6.

[0078] Step 5: Refer to Figure 3 The system expands equipment capacity according to task requirements. After the task is successfully set up, upon reaching the start time, the scheduling server divides the task into several subtasks based on the task information. Then, it performs equipment expansion operations based on the active requests of the subtasks.

[0079] Step 6: Notify the expansion and allocate devices to execute sub-tasks. The expansion process will select the best-performing devices to execute sub-tasks based on real-time operational data.

[0080] Step 7: Reduce equipment capacity according to task requirements. Based on the task execution status and real-time equipment operating information, reduce equipment capacity as needed.

[0081] Step 8: Notify the device to scale down, stop the current subtask, and stop assigning new subtasks to the device. If the device confirms the scaling down, it will proactively stop assigning new subtasks and stop the currently executing subtask.

[0082] Step 9: Task Complete. The task is complete when all subtasks are finished.

[0083] Specifically, embodiments of this application perform task scheduling based on the status of the device and the task. (Refer to...) Figure 4 As shown, the device has a total of 5 states, as follows:

[0084] Offline: When the scheduling server cannot receive information from the device within 3 minutes, it will be determined to be in an offline state.

[0085] Idle: The device is in an idle state when it starts, a subtask is completed, or a scaling down is completed. At this time, the device will enter a standby state and wait for a subtask to be assigned to wake it up.

[0086] Expanding capacity: The state when requesting device expansion. In this embodiment, only idle devices can perform expansion operations, so that the same device can only execute one task at a time, ensuring efficient task execution and improving the completeness of task completion.

[0087] During operation: The state during capacity expansion. This application's equipment does not undergo capacity expansion scheduling during operation; therefore, the next state of the equipment in operation may be idle or in capacity reduction.

[0088] Downsizing: The state when a device is requested to be downsized. In this embodiment, after downsizing is completed, there are no running tasks on the device, and the device is in an idle state.

[0089] This application embodiment scales up and down based on the status of equipment and tasks. Specifically, for equipment in an idle state with tasks waiting to be scheduled, the idle equipment is selected to execute the tasks. Therefore, the scheduling server in this application embodiment evaluates the performance information of drilling equipment in real time based on CPU utilization and remaining memory. The scheduling server in this application embodiment dynamically performs scaling up and down operations based on task information to ensure that tasks are completed as expected. This application embodiment can automate and efficiently complete the process from equipment registration to equipment scaling up and down scheduling, which improves operational efficiency and reduces equipment consumption and energy consumption such as electricity.

[0090] Optionally, in one embodiment of the present invention, reference is made to... Figure 5 According to the expansion strategy, the first device is scheduled to execute the first task, including:

[0091] Step S410: Obtain the first information of each first device in the first device set;

[0092] Step S420: Determine the weight of each first device based on the first information;

[0093] Step S430: Sort the devices in descending or ascending order of weight, and determine the first device in the first order as the first device that needs to be expanded.

[0094] In some possible implementations, for idle devices, a suitable device can be selected for task scheduling by matching the device's capacity with the capacity required by the tasks waiting to be scheduled; alternatively, task scheduling can be based on the device's historical utilization rate; or it can be based on the length of time the device has been idle. Of course, the capacity expansion strategy can be modified according to actual scheduling needs to meet customer requirements.

[0095] Optionally, in one embodiment of the present invention, the first information of the first device includes a first CPU utilization rate and a first memory utilization rate; the method determines the first weight of the first device through the following steps:

[0096] The first probability is determined by using the first value as the base and the second value as the exponent; the second value is related to the first CPU utilization rate.

[0097] The second probability is determined by using the first value as the base and the third value as the exponent; the third value is related to the first memory usage rate.

[0098] The first probability and the second probability are averaged to determine the first weight.

[0099] In some possible implementations, the expansion operation of this application embodiment can be performed through the following steps, specifically:

[0100] Step 21: Check the previous capacity expansion / contraction information. Regarding the capacity expansion process, when a device is confirmed for expansion, there are two state transitions. Each transition requires the following steps:

[0101] Idle - Expanding Capacity: The scheduling server updates the device status to "Expanding Capacity" in the database, marks the subtask as scheduled, and stores information such as the parameters and data download links required for subtask execution in the database. Then, it sends an expansion command to the device to schedule the subtask. The expansion command mainly includes: device MAC address, device IP address, subtask information, etc. After receiving the expansion command, the device queries the database and actively downloads the parameters and necessary data for subtask execution. The subtask needs to complete a series of preparatory tasks from issuance to execution, so it takes time for the device to become ready after receiving the command. The "Expanding Capacity" status rejects new scheduling requests.

[0102] Expanding capacity - In operation: Once the device is ready, update the device status to "in operation" in the database, and then it will switch to the "in operation" status.

[0103] Regarding the capacity reduction process, when a device is determined to be reduced in size, there are two state transitions, and the following steps need to be taken for each transition:

[0104] During operation - scaling down: The scheduling server updates the device status to "scaling down" in the database and then sends a scaling down command to the device. The scaling down command mainly includes: device MAC address, device IP address, subtask information, etc. Upon receiving the scaling down command, the device needs to check the completion status of the current subtask, save the current subtask execution information, report to the scheduling server, and stop executing the current subtask. The "scaling down" status rejects new scheduling requests.

[0105] Scaling down - Idle: After the device finishes processing the current subtask, it updates the device status to idle in the database and then switches to the idle state.

[0106] In general, if a device is in a scaling-up state, it performs scaling-up preparation work. Once ready, it changes the database status to "working" and then begins executing subtasks. If the device is in a scaling-down state, it performs scaling-down preparation work. Once ready, it changes the database status to "idle." After a subtask is completed, the device actively reports to the scheduling server. The scheduling server marks the subtask as completed in the database and removes it from further scheduling. When the server receives subtask execution exception information (such as the device refusing to execute a subtask during scaling-down, or the device losing connection), the subtask status is changed from "scheduled" to "unscheduled." If the device remains in a scaling-up or scaling-down state for more than 10 seconds, the scheduling server will resend the scaling-up and scaling-down commands to prevent the device from missing these commands due to network packet loss.

[0107] Step 22: Obtain the current status information of all resources. This involves obtaining the status information of all devices, which is divided into two parts: device status information: this can be obtained by querying the database for the status information of each device; and real-time device operating information: this is obtained through device active reporting and scheduling server broadcasts, including the average CPU utilization over 30 seconds and the current memory utilization (this is configurable, the default is 30 seconds). Based on this information, we can obtain the number of devices currently in an idle state (curAvailDeviceNum) and the number of devices currently in a busy state (curBusyDeviceNum).

[0108] Step 23: Obtain the current task execution status information. This involves obtaining the execution information of all subtasks of the current task. This is primarily achieved by querying the database. The execution information includes the status of each subtask and its corresponding allocated device information. For example, ... Figure 6 Subtask 1 is assigned to device 3 and is in a scheduling state; subtask 2 is assigned to device 4 and is in an execution state, and so on. Of course, this embodiment can include a visual interface to show staff the task processing procedure. Furthermore, different colored boxes or font colors can be used to differentiate different task states, thereby enhancing the customer experience.

[0109] Based on the above information, we can obtain the number of currently unscheduled subtasks (curLeftSubTaskNum, i.e., unscheduled), the number of currently completed subtasks (curFinishSubTaskNum, i.e., completed), and the data of subtasks currently waiting to be executed or being executed (curDoingSubTaskNum, i.e., being scheduled or being executed).

[0110] Step 24: Are there any idle devices currently? If the number of devices in the current idle state, curAvailDeviceNum, is 0, it means the number of remaining idle devices is 0. Notify the operation and maintenance personnel that the entire device system is fully loaded, and go to Step 212. Otherwise, go to Step 25.

[0111] Step 25: Is the task completed? If the number of unscheduled subtasks currently, curLeftSubTaskNum, is 0, it means the task is completed. For the first check, notify the person in charge of the task that the task is completed, and go to Step 212. Otherwise, go to Step 26.

[0112] Step 26: Obtain the resource information required for the current task execution. That is, obtain the parameter information of the current task, which is mainly obtained by querying the database. The main information is the minimum number of devices minNeedDevice and the maximum number of devices maxNeedDevice occupied by the task. Then the current number of devices required, curNeedDeviceNum = maxNeedDevice - curDoingSubTaskNum.

[0113] Step 27: Does the current number of used devices exceed the maximum number set for the task? If the number of subtasks currently waiting to be executed or being executed, curDoingSubTaskNum, is greater than maxNeedDevice, that is, it exceeds the maximum number of devices, reject the expansion scheduling, and regularly notify the person in charge of the task and the operation and maintenance personnel, and go to Step 212. If curDoingSubTaskNum is greater than minNeedDevice, then choose to expand based on a certain probability: If random(0,1) < MinProb (this can be configured, the default is 0.5), go to Step 212, otherwise go to Step 28. Here, random(0,1) is to get a random value within [0,1]. The reason for this design consideration is: If it is less than minNeedDevice, it means that devices need to be scheduled quickly to meet the minimum requirements of the task; if it exceeds maxNeedDevice, it means that the maximum requirements of the task have been met and no new devices will be scheduled additionally; for the remaining cases, choose to schedule according to the probability MinProb. This is to avoid a single task quickly抢占 all devices.

[0114] Step 28: Traverse all unscheduled subtasks. For all unscheduled subtasks, traverse and schedule the best devices one by one. If the traversal is completed, go to Step 212. Otherwise, go to Step 29.

[0115] It should be noted that the text "单个任务快速抢占所有设备" in the original text seems to be an incorrect expression. I translated it as "a single task quickly抢占 all devices" for the purpose of maintaining the integrity of the translation according to the original text. You may need to check and correct this part if necessary.Step 29: Check if the scaling cooling time is satisfied. For the current subtask, assume the timestamp of the last scaling operation is lastOptStamp, the current time is now, and the minimum scaling interval minOptInterval is 30 (this can be configured, default is 30s). If now < lastOptStamp + minOptInterval, that is, the time since the last scaling operation has not exceeded the minimum interval minOptInterval, reject the scheduling and go to Step 28. Otherwise, go to Step 210. Additionally, for the current executing task, if the number of already scheduled devices - the number of devices to be scaled down <= minNeedDevice, directly reject the scaling down and go to Step 28, because it is necessary to ensure the minimum number of devices required for task execution.

[0116] Reasons for setting the minimum interval: To avoid frequent scaling scheduling for a single subtask and prevent system oscillations. Considering that some subtasks may be too complex, or the devices may not be able to support, or network issues may cause data not to be ready, etc., the following scenario may occur: In the first round of scheduling, devices are expanded and allocated first. Then, if the devices find that they cannot execute, they will immediately notify the scheduling server to perform a scaling down operation. Then, the second round of scheduling will repeat the first round, and so on. Therefore, to avoid an infinite loop of "expansion scheduling -> scaling down scheduling -> expansion scheduling scaling down scheduling ->......" for devices and subtasks, the minimum scaling interval is set. In summary, the minimum scaling interval minOptInterval is the minimum time interval for a single subtask to experience frequent scaling due to irresistible circumstances.

[0117] Step 210: Select appropriate devices for the task for scheduling.

[0118] Step 211: Request to schedule devices and complete the expansion scheduling operation. Assume the current subtask is SubTask and the selected device is BestDevice. The optimal device has been selected in Step 210. This step is to prepare for the expansion scheduling, including the following operations:

[0119] Modify the status information of the subtask SubTask: Modify it to "scheduling", and the allocated device is BestDevice;

[0120] Modify the status information of the scheduled device BestDevice: Modify it to "expanding", as well as other task parameter information;

[0121] Send an expansion instruction: Send an expansion instruction to the BestDevice device. Go to Step 28.

[0122] Step 212: This expansion operation is completed, waiting for the next execution. Reaching this step indicates that the current round of expansion operation is completed.

[0123] Steps 21 to 212 constitute the expansion completion process, which is executed cyclically every second. The expansion process is task-oriented, while the reduction process is device-oriented; the two are mutually constraining and efficiently drive task completion.

[0124] Optionally, in one embodiment of the present invention, the method further includes:

[0125] Get the first timestamp of the last expansion or contraction operation;

[0126] If the time between the current moment and the first timestamp is less than the first threshold, no expansion or contraction operation will be performed.

[0127] It is understood that the first threshold in this application embodiment is the minimum expansion / contraction interval (minOptInterval) in the above expansion / contraction operation embodiment. By judging the expansion cooling time, it is determined whether to perform expansion and contraction operations to protect the security and availability of the system.

[0128] Optionally, in one embodiment of the present invention, reference is made to... Figure 7 As shown, the method also includes:

[0129] Step S510: If the second task is completed, update the status of the second device to idle and update the status of the second task to completed;

[0130] Alternatively, in step S520: if the second device completes the power-on or restart operation, update the status of the second device to idle.

[0131] Optionally, in one embodiment of the present invention, the method further includes:

[0132] Receive the first information of the current moment sent by the second device and store it in the database;

[0133] Obtain the first information of the second device in the database at the previous moment. If the time between the previous moment and the current moment is greater than the second threshold, update the status of the second device to offline, update the status of the task executed by the second device to waiting for scheduling, and issue the first fault warning information.

[0134] Optionally, in one embodiment of the present invention, the method further includes:

[0135] Record the second information of the second device; the second device is used to identify the newly added device, and the second information includes network address information and device address information;

[0136] Receive the third information sent by the second device when it is powered on; if the third information is inconsistent with the second information, issue a second fault warning message.

[0137] Reference Figure 8As shown, the embodiments of this application perform a reduction-size operation on the device, specifically in the following ways:

[0138] Scenario 1: Continuously High Device Load: The device's CPU or memory continuously experiences extremely high load, meaning the device's performance cannot support the execution of the currently assigned subtasks. If a device's CPU utilization consistently reaches 120% or higher, or its memory utilization consistently reaches 100%, then the device with CPU utilization exceeding the fourth threshold in the second device set is identified as the second device; or, the device with memory utilization exceeding the fifth threshold in the second device set is identified as the second device. Alternatively, it can be configured that if the duration exceeds MaxDuration1 (this is configurable, the default is 60 seconds), the device is considered to be under high load.

[0139] Scenario 2: The device is unable to enter the working state due to abnormal environment. For example, when the device is in the expansion or reduction state, it needs to prepare the parameters and necessary resources for downloading subtasks. If there are network fluctuations or terminal issues, the device will be unable to enter the working state for a long time. This is likely to happen when a ship is sailing in bad sea conditions. If the device is found to be in the expansion or reduction state for more than 30 seconds, the maintenance personnel should be notified. If the duration exceeds MaxDuration2 (which can be configured, the default is 180 seconds), the node is considered to be abnormal and unable to execute. In this case, the device is no longer waited for and is directly determined to be unable to enter the working state.

[0140] Scenario 3: Equipment Malfunction. This refers to a physical failure caused by severe sea conditions, prolonged high-intensity use, or unknown factors. If no real-time operational information is received from the equipment for more than 90 seconds, the equipment is considered offline. Theoretically, the equipment needs to report real-time operational information to the scheduling server 90 times (once per second). If no latest information is received for 90 consecutive times, the equipment is considered offline.

[0141] Scenario 4: Device continuously operating with very low load. If the device is operating, its CPU utilization is lower than CPUMinPercent (configurable, default is 20%), and its memory utilization is lower than MemMinPercent (configurable, default is 20%). Note that only if both conditions are met simultaneously is it allowed to be considered for scaling down. That is, devices with CPU utilization lower than the fourth threshold and memory utilization lower than the fifth threshold are identified as the second set of devices. If a device's CPU utilization is found to be lower than CPUMinPercent, and its memory utilization is consistently lower than MemMinPercent, and if this continues for more than MaxDuration3 (configurable, default is 60 seconds), then the device is considered to be in a continuously low load state.

[0142] In view of the above situation, the embodiments of this application take the following measures:

[0143] Scenario 1: Device continuously under extremely high load: Change the status of the assigned subtasks to "awaiting scheduling," change the device status to "shrinking," and notify the device to immediately stop task execution. After shrinking is complete, change the device status to "idle" and do not schedule any tasks for a period of time.

[0144] Scenario 2: The device is unable to enter working mode due to abnormal environment. Change the status of the assigned subtask to unscheduled, change the device status to shrinking, and notify the device to immediately stop task execution.

[0145] Scenario 3: Device malfunction. Change the status of the assigned subtask to unscheduled (i.e., waiting for scheduling), change the device status to offline, and notify the device to immediately stop task execution.

[0146] Scenario 4: The device is continuously operating with very low load. Change the status of the assigned subtasks to unscheduled, change the device status to scaling down, and notify the device to immediately stop task execution. A potential issue here is that if the subtask allocation is unreasonable, frequent scaling up and down operations may occur everywhere. The minimum scaling interval (minOptInterval) can alleviate this problem to some extent. This scenario will notify the operations and maintenance personnel, who may need to manually adjust the task parameters upon receiving the Case 4 notification.

[0147] In this embodiment of the application, the device to be expanded is prepared for expansion. Specifically, the status of the first task is updated to be scheduled, and the status of the first device is updated to be expanded. The third information required to execute the first task is put into the database, and an expansion instruction is sent to the first device.

[0148] The scaling control method proposed in this invention includes the following steps: acquiring first information for each device; the first information includes CPU utilization and memory utilization; determining the first state of each device; the first state includes working, idle, offline, scaling up, and scaling down; determining the second state of each task; the second state includes waiting to be scheduled, scheduling, executing, and completed; if there is a first task with the second state of waiting to be scheduled, and there is a first set of devices with the first state of idle, scheduling the first device to execute the first task according to the scaling strategy, updating the state of the first task to scheduling, and updating the state of the first device to scaling up; the first device... The set includes a first device, and the expansion strategy includes a strategy of selecting the first device from the first device set based on first information; the first device is used to represent a device in an idle state; or, if there is a second device set in a working state, the second device is scaled down according to the scaling-down strategy, and the state of the second task is updated to waiting for scheduling, and the state of the second device is updated to scaling down; if the second device completes the scaling-down operation, the state of the second device is updated to idle; the second device set includes a second device, the second task is a task scheduled to the second device, and the scaling-down strategy is used to represent whether to perform a scaling-down operation on the second device based on the first information of the second device. This application embodiment improves the accuracy of evaluation by determining device information, device state, and task state, and evaluating devices and tasks based on real-time information; and by performing expansion and scaling-down scheduling based on device information, it can achieve real-time expansion and scaling-down operations and improve resource utilization.

[0149] Secondly, refer to the appendix Figure 9 A control system for scaling up or down a device according to an embodiment of the present invention is described.

[0150] Figure 9 This is a schematic diagram of a control system for expanding and shrinking equipment according to an embodiment of the present invention. The system specifically includes:

[0151] The first module 910 is used to obtain the first information of each device; the first information includes CPU utilization and memory utilization.

[0152] The second module 920 is used to determine the first state of each device; the first state includes working, idle, offline, expanding capacity and shrinking capacity.

[0153] The third module 930 is used to determine the second state of each task; the second state includes waiting for scheduling, scheduling, executing, and completed.

[0154] The fourth module 940 is used to schedule the first task to execute the first task according to the expansion strategy if there is a first task in the second state of waiting to be scheduled and there is a first set of devices in the first state of being idle, and update the status of the first task to be scheduled and update the status of the first device to be expanded; the first set of devices includes the first device, and the expansion strategy includes a strategy of selecting the first device from the first set of devices according to the first information; the first device is used to represent the device in the idle state.

[0155] The fifth module 950 is used to perform a scaling-down operation on the second device according to the scaling-down strategy if there is a second device set in the first state of being in operation, and update the status of the second task to waiting for scheduling and update the status of the second device to scaling down; if the second device completes the scaling-down operation, update the status of the second device to being idle; the second device set includes the second device, the second task is the task scheduled to the second device, and the scaling-down strategy is used to characterize whether to perform a scaling-down operation on the second device based on the first information of the second device.

[0156] It is evident that the content of the above method embodiments is applicable to this system embodiment. The specific functions implemented in this system embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0157] Reference Figure 10 This invention provides a control device for expanding or shrinking equipment capacity, comprising:

[0158] At least one processor 810;

[0159] At least one memory 820 is used to store at least one program;

[0160] When the at least one program is executed by the at least one processor 810, the at least one processor 810 implements the device scaling control method.

[0161] Similarly, the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0162] This invention also provides a computer-readable storage medium storing a processor-executable program, which, when executed by a processor, is used to perform the aforementioned device scaling control method.

[0163] Similarly, the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0164] In some alternative embodiments, the functions / operations mentioned in the block diagrams may not occur in the order shown in the operation diagrams. For example, depending on the functions / operations involved, two consecutively shown blocks may actually be executed substantially simultaneously, or the blocks may sometimes be executed in reverse order. Furthermore, the embodiments presented and described in the flowcharts of this invention are provided by way of example to provide a more comprehensive understanding of the technology. The disclosed methods are not limited to the operations and logic flows presented herein. Alternative embodiments are contemplated in which the order of various operations is altered and sub-operations described as part of a larger operation are executed independently.

[0165] Furthermore, although the invention has been described in the context of functional modules, it should be understood that, unless otherwise stated, one or more of the functions and / or features may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in a separate physical device or software module. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the invention. Rather, given the properties, functions, and internal relationships of the various functional modules in the apparatus disclosed herein, the actual implementation of the module will be understood within the scope of conventional skill of an engineer. Therefore, those skilled in the art can implement the invention as set forth in the claims using ordinary techniques without excessive experimentation. It is also understood that the specific concepts disclosed are merely illustrative and not intended to limit the scope of the invention, which is determined by the full scope of the appended claims and their equivalents.

[0166] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several programs to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0167] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequential list of executable programs for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, a program execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can retrieve and execute a program from or in conjunction with such a program execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can mean any means that can contain, store, communicate, propagate, or transmit a program for use by or in conjunction with a program execution system, apparatus, or device.

[0168] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0169] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable program execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0170] In the foregoing description of this specification, references to terms such as "one embodiment," "another embodiment," or "some embodiments" indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0171] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

[0172] The above is a detailed description of the preferred embodiments of the present invention. However, the present invention is not limited to the embodiments described. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the present invention. All such equivalent modifications or substitutions are included within the scope defined by the claims of the present invention.

Claims

1. A method for controlling the expansion and contraction of a device, characterized by, The method comprises the following steps: obtaining first information of each device; the first information comprises CPU usage and memory usage; determining a first state of each device; the first state comprises working, idle, offline, expansion and contraction; determining a second state of each task; the second state comprises waiting for scheduling, scheduling, execution and completion; if there is a first task in the second state of waiting for scheduling, and there is a first device set in the first state of idle, scheduling the first device to execute the first task according to the expansion strategy, updating the state of the first task to scheduling, and updating the state of the first device to expansion; the first device set comprises the first device, and the expansion strategy comprises a strategy of selecting the first device from the first device set according to the first information; the first device is used to represent a device in the idle state; or, if there is a second device set in the first state of working, performing a contraction operation on the second device according to the contraction strategy, updating the state of the second task to waiting for scheduling, and updating the state of the second device to contraction; if the second device completes the contraction operation, updating the state of the second device to idle; the second device set comprises the second device, the second task is a task scheduled to the second device, and the contraction strategy is used to represent whether to perform a contraction operation on the second device according to the first information of the second device.

2. The method of claim 1, wherein the method further comprises: The scheduling of the first device to execute the first task according to the expansion strategy comprises: obtaining the first information of each first device in the first device set; determining the weight of each first device according to the first information; sorting the devices in descending or ascending order according to the weight, and determining the first device in the first position as the first device to be expanded.

3. The method of claim 2, wherein the method further comprises: The first information of the first device comprises first CPU usage and first memory usage; the method determines the first weight of the first device by the following steps: determining a first probability with a first value as the base and a second value as the exponent; the second value is related to the first CPU usage; determining a second probability with the first value as the base and a third value as the exponent; the third value is related to the first memory usage; averaging the first probability and the second probability to determine the first weight.

4. The method of claim 1, wherein the method further comprises: The method further comprises the following steps: obtaining a first timestamp of the last expansion or contraction; if the time between the current time and the first timestamp is less than a first threshold, not performing the expansion or contraction operation.

5. The method of claim 1, wherein the method further comprises: The method further comprises: if the second task is executed, updating the state of the second device to idle and updating the state of the second task to completed; or, if the second device completes the power-on or restart operation, updating the state of the second device to idle.

6. The method of claim 1, wherein, The method further comprises: receiving the first information of the current time sent by the second device and storing it in the database; Acquire first information of the second device at a last time in the database, if a time between the last time and the current time is greater than a second threshold, update a state of the second device as an offline state, update a state of a task executed by the second device as a waiting scheduling state, and send a first fault warning information.

7. The method of claim 1, wherein the method further comprises: The method further comprises: Record second information of a second device; the second device is used to represent an added device, and the second information comprises network address information and device address information; Receive third information sent by the second device when starting; if the third information is inconsistent with the second information, send a second fault warning information.

8. A control system for scaling of a device, characterized in that Comprise: A first module, configured to acquire first information of each device; the first information comprises CPU usage and memory usage; A second module, configured to determine a first state of each device; the first state comprises working, idle, offline, expansion, and contraction; A third module, configured to determine a second state of each task; the second state comprises waiting scheduling, scheduling, execution, and completion; A fourth module, configured to if there is a first task with the second state as waiting scheduling, and there is a first device set with the first state as idle, schedule the first device to execute the first task according to an expansion strategy, and update the state of the first task as scheduling, and update the state of the first device as expansion; the first device set comprises the first device, and the expansion strategy comprises a strategy of selecting the first device from the first device set according to the first information; The first device is used to represent a device with the state as idle; A fifth module, configured to if there is a second device set with the first state as working, perform a contraction operation on the second device according to a contraction strategy, and update a second task as waiting scheduling, and update the state of the second device as contraction; if the second device completes the contraction operation, update the state of the second device as idle; the second device set comprises the second device, the second task is a task scheduled to the second device, and the contraction strategy is used to represent whether to perform the contraction operation on the second device according to the first information of the second device.

9. A device capacity control apparatus, characterized by comprising: Comprise: At least one processor; At least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the control method for device expansion and contraction according to any one of claims 1 to 7.

10. A computer readable storage medium having stored therein a program that is executable by a processor, characterized in that, The program executable by the processor is used to implement the control method for device expansion and contraction according to any one of claims 1 to 7 when executed by the processor.

Images