Business credential migration-oriented full-process monitoring method and system

Abstract

The invention relates to the field of data processing, in particular to a service credential migration-oriented full-process monitoring method and system, and the method comprises the steps: obtaining a resource consumption estimation coefficient of each service module according to the change relation of each throughput curve along with an available computing resource curve; screening out a plurality of migration tasks from each service module for pausing, wherein the size relationship of pause indexes of the migration tasks paused in different service modules and the size relationship of resource consumption estimation coefficients between different service modules have the maximum consistency; recording the difference between the pause index of the unpaused migration task in each business module and the pause index of the paused migration task as a first difference; and after all the migration tasks are paused, the paused migration tasks are obtained again by using the variation of the available computing resource curve along with the throughput curve of each service module and the first difference. According to the method, the risk existing in the credential migration process when computing resources are insufficient and resource allocation is improper is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, in particular to a full-process monitoring method and system for business signal creation migration. BACKGROUND

[0002] Signal creation migration refers to a full-stack replacement process of migrating an existing business system from a traditional technology system (such as an x86 architecture, a foreign database / operating system) to a self-controllable signal creation technology system (such as an ARM architecture, a domestic database / operating system). The process usually adopts a hot migration method, that is, data migration tasks (such as database migration, configuration file synchronization) are performed while the business is normally running. In the migration process, the entire process of signal creation migration is monitored through visual computing resource usage, business running status, and migration progress, so as to control the migration running status of each link.

[0003] In the hot migration scenario, the business module needs to handle user requests and migration tasks (such as database synchronization and verification) at the same time, resulting in double occupation of computing resources (such as CPU). When the computing resources are insufficient, if the resources are not reasonably allocated, it will cause business running risks and migration process risks. The former causes user request response delay or failure due to insufficient computing resources, and the latter causes unexpected interruption of migration tasks due to insufficient resources, prolonging the migration period. The conventional signal creation migration monitoring method cannot reasonably allocate computing resources in a timely manner according to the monitoring results to ensure the smooth progress of business running and migration process, resulting in risks in the entire signal creation migration process. SUMMARY

[0004] To solve the above problems, the present application provides a full-process monitoring method and system for business signal creation migration.

[0005] The full-process monitoring method and system for business signal creation migration of the present application adopt the following technical solutions: One embodiment of the present application provides a full-process monitoring method for business signal creation migration, which comprises the following steps: monitoring the available computing resource curve of a plurality of business modules and the throughput curve of each business module, each business module containing a plurality of migration businesses, calculating the resource consumption intensity of each migration task, and obtaining the resource consumption estimation coefficient of each business module according to the change relationship between each throughput curve and the available computing resource curve; when the available computing resource curve is less than a first preset threshold, a plurality of migration tasks are suspended from each business module, and the size relationship of the suspension indicators of the suspended migration tasks in different business modules has maximum consistency with the size relationship of the resource consumption estimation coefficients between different business modules; wherein the suspension indicator of the suspended migration task is obtained from the resource consumption intensity of the suspended migration task and the remaining task quantity of the suspended migration task; The difference between the suspension index of the non-suspended migration task in each service module and the suspension index of the suspended migration task is recorded as a first difference; after suspending all migration tasks, the amount of change of the available computing resource curve with the throughput curve of each service module is recorded as a first change amount, and the resource consumption estimation coefficient and the suspension index obtained under each service module are corrected by using the difference between the first change amount and the first difference, and the suspended migration task is suspended again.

[0006] Preferably, the resource consumption intensity of each migration task is calculated, including the following specific steps: For all migration tasks currently running in each service module, each migration task corresponds to a start time, the migration task M1 with the smallest start time is obtained, and the migration task M2 with the second smallest start time is obtained; for the time period after the start of M1 and before the start of M2, the change feature of the throughput curve of each service module in the time period is obtained, which is recorded as the first throughput change feature of the migration task M1; The migration task M3 with the third smallest start time is obtained; for the time period after the start of M2 and before the start of M3, the change feature of the throughput curve of each service module in the time period is obtained, which is recorded as Q; the sum of the first throughput change features of the running migration tasks other than M2 in the time period is obtained, which is recorded as Q0, and Q-Q0 is recorded as the first throughput change feature of the migration task M2; Similarly, until the migration task M with the largest start time; for the time period after the start of M and before the current time, the change feature of the throughput curve of each service module in the time period is obtained, which is still recorded as Q; the sum of the first throughput change features of the running migration tasks other than M in the time period is obtained, which is still recorded as Q0, and Q-Q0 is recorded as the first throughput change feature of the migration task M; The resource consumption intensity of each migration task is positively correlated with the size of the first throughput change feature of each migration task.

[0007] Preferably, the resource consumption estimation coefficient of each service module is obtained according to the change relationship between each throughput curve and the available computing resource curve, including the following specific steps: In a first preset time period before the current time, the Pearson correlation coefficient of each throughput curve and the available computing resource curve is obtained, all the Pearson correlation coefficients corresponding to the throughput curves are normalized, and the normalized Pearson correlation coefficient is taken as the resource consumption estimation coefficient of each service module.

[0008] Preferably, the suspension of a number of migration tasks from each business module is performed according to the following steps: A number of migration tasks are randomly selected from each business module, and a suspension index of the selected migration tasks is calculated, which is obtained from the resource consumption intensity of the selected migration tasks and the remaining task amount of the selected migration tasks; The suspension index of the selected migration tasks is taken as the suspension index of each business module; all the migration tasks randomly selected from all the business modules are taken as a group of tasks to be suspended; The consistency of the tasks to be suspended is calculated according to the size relationship between the suspension index of different business modules and the size relationship between the resource consumption estimation coefficients of different business modules; The tasks to be suspended at the maximum consistency are obtained, and the migration tasks selected from each business module and contained in the tasks to be suspended at the maximum consistency are taken as the suspended migration tasks in each business module.

[0009] Preferably, the suspension index is obtained according to the following steps: The selected migration tasks, the suspended migration tasks, or the non-suspended migration tasks constitute a migration task set; The sum of the resource consumption intensity of all the migration tasks in the migration task set is taken as a first index H1, and the sum of the remaining task amount of all the migration tasks in the migration task set is taken as a second index H2; H1+m×H2 is taken as the suspension index of each business module, and m represents a preset migration attention coefficient in each business module.

[0010] Preferably, the resource consumption estimation coefficient and the suspension index obtained in each business module are corrected by using the difference between the first change and the first difference, and the suspended migration tasks are suspended again according to the following steps: The difference between the first change and the first difference is taken as a correction coefficient of each business module; The resource consumption estimation coefficient of each business module is updated by using the correction coefficient of each business module, and the migration attention coefficient in each business module is updated; wherein the update result of the resource consumption estimation coefficient is positively correlated with the correction coefficient, and the update result of the migration attention coefficient is negatively correlated with the correction coefficient; The suspended migration tasks are selected from each business module again by using the update result of the resource consumption estimation coefficient and the update result of the migration attention coefficient.

[0011] Preferably, the specific steps for obtaining the change characteristics of the throughput curve of each service module in the time period include the following: The absolute value of the slope of the throughput curve of each service module in the time period is taken as the change characteristic, wherein the slope is obtained by fitting the throughput curve in the time period into a straight line by using the least square method, and the slope is the slope of the straight line.

[0012] Preferably, the consistency of the tasks to be suspended is calculated according to the size relationship of the suspension indicators of different service modules and the size relationship of the resource consumption estimation coefficients between different service modules, and the specific steps include the following: A plurality of service modules are randomly selected from all service modules to form a module combination, for all module combinations, the resource consumption estimation coefficients of all service modules in each module combination form a first sequence, the suspension indicators of all service modules in each module combination form a second sequence, the Pearson correlation coefficient of the first sequence and the second sequence is taken as the correlation coefficient of each module combination, and the average of the correlation coefficients of all module combinations is taken as the consistency.

[0013] Preferably, the resource consumption estimation coefficient of each service module is updated and the migration attention coefficient in each service module is updated by using the correction coefficient of each service module, and the specific steps include the following: The correction coefficients of all service modules are linearly normalized, the resource consumption estimation coefficient of each service module is denoted as x1, the normalized correction coefficient in each service module is denoted as w1, the value of the migration attention coefficient in each service module is denoted as m1, the updated result of the resource consumption estimation coefficient is x1*(1+w1-w0), and the updated result of the migration attention coefficient m is m1*(1-w1+w0); wherein w0 represents a preset reference value.

[0014] Another embodiment of the present application provides a full-process monitoring system for service signal creation migration, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes all steps of the full-process monitoring method for service signal creation migration when executing the computer program.

[0015] The technical scheme of the present application has the following advantages: The present application selects appropriate migration tasks from each service module for suspension, thereby realizing the reallocation of the computing resources of different service modules, so that when the computing resources are tight, the risk caused by the tight computing resources can be inhibited by reasonably allocating the computing resources.

[0016] Wherein, the present application has the greatest consistency between the size relationship of the suspension indicators of the suspended migration tasks in different business modules and the size relationship of the resource consumption estimation coefficients between different business modules when the migration tasks in each business module are screened out for suspension; wherein the suspension indicators of the suspended migration tasks are obtained from the resource consumption intensity of the suspended migration tasks and the remaining task amount of the suspended migration tasks. This process can select migration tasks that can significantly reduce the risk of computing resource shortage from business modules that have a greater impact on available computing resources, and the suspension of these migration tasks helps to ensure that normal business and migration processes cannot proceed smoothly; avoid selecting migration tasks that cannot reduce the risk of computing resource shortage in business modules that have a smaller impact on available computing resources, and the suspension of these migration tasks will still pose a risk to normal business and migration processes. In summary, this process avoids the problem of hardware resource shortage that is not conducive to the normal business operation and migration process execution of all business modules as much as possible without affecting the migration process.

[0017] Further, the present application records the difference between the suspension indicators of the non-suspended migration tasks in each business module and the suspension indicators of the suspended migration tasks as a first difference; after suspending all migration tasks, the difference between the change amount of the available computing resource curve and the throughput curve of each business module and the first difference is used to correct the resource consumption estimation coefficient and the suspension indicator obtained under each business module, and the suspended migration tasks are re-acquired for suspension. This process re-acquires the migration tasks for suspension again after suspending the migration tasks, avoiding the problem that the available computing resources cannot be significantly recovered in a short period of time and still pose a risk of insufficient computing resources. This process can ensure that the problem of insufficient available computing resources is inhibited and the computing resources can be quickly recovered, further reducing the risk in normal business operation and migration processes. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0019] Figure 1 The step flow chart of the full-process monitoring method for service-oriented service migration provided by an embodiment of the present application. DETAILED DESCRIPTION

[0020] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of the full-process monitoring method and system for business information technology innovation migration proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0022] The following description, in conjunction with the accompanying drawings, details the specific solution of the full-process monitoring method and system for business information technology innovation migration provided by this invention. Example 1:

[0023] Please see Figure 1 This document illustrates a flowchart of a full-process monitoring method for business information technology innovation migration provided by an embodiment of the present invention. The method includes the following steps: Step S101: Monitor the available computing resource curves of several business modules and the throughput curve of each business module. Each business module contains several migration services.

[0024] This embodiment of the information technology innovation migration uses the hot migration method, that is, the business data is migrated while the business is running normally. The business data includes databases, configuration files, applications, etc. This embodiment takes the migration of databases as an example for description.

[0025] During the migration of domestic IT applications, it is necessary to monitor the usage of hardware resources in real time to avoid insufficient hardware resources affecting the normal operation of services and the progress of the migration process. This embodiment uses CPU utilization monitoring as an example. The CPU utilization 'a' is read every second, and 1-'a' is taken as the available computing resources per second. The available computing resources at all times form an available computing resource curve (the horizontal axis is seconds, and the vertical axis is available computing resources). Multiple business modules run simultaneously in all processes on the CPU, and each business module runs its business normally, responding to user requests every second.

[0026] In this embodiment, the average number of requests per second in each business module within a preset time period before the domestic IT innovation migration is recorded as the throughput baseline D0 of each business module. The preset time period in this embodiment refers to the two days before the domestic IT innovation migration.

[0027] In the process of the signal creation migration, the number of requests per second of each business module in normal business operation is recorded as the throughput D1 per second, and D1 / D0 is recorded as the relative throughput per second; the relative throughput per second constitutes the throughput curve (the horizontal coordinate is second, and the vertical coordinate is relative throughput) of each business module, which describes the size of the throughput per second in the process of the signal creation migration relative to the throughput baseline. The smaller the relative throughput, the weaker the user data processing capacity in the process of the signal creation migration compared with that before the signal creation migration, because the database migration needs to be performed at the same time.

[0028] In the process of the signal creation migration, each business module needs to migrate all databases related to business operation, and the migration process of each database is recorded as a migration task, that is, each business module needs to perform several migration tasks while normally operating the business; when each migration task is performed, the database needs to be migrated and consistency check and integrity check are performed on the database. The migration method of the database is known, and the specific process is not described in this embodiment.

[0029] Step S102, the resource consumption intensity of each migration task is calculated.

[0030] In each business module, several migration tasks are performed at the same time at the same time, and the maximum number of migration tasks performed at the same time in this embodiment is set to 6; each migration task will affect the business operation of each business module, for example, reduce the throughput of the business module in business operation; in this embodiment, the resource consumption intensity of each migration task is used to describe the influence of the execution of each migration task on the business operation of the business module. The greater the resource consumption intensity of each migration task, the greater the influence of the execution of each migration task on the business operation of the business module.

[0031] As an example, the calculation method of the resource consumption intensity of each migration task is as follows: For all migration tasks currently running in each business module, each migration task corresponds to a start time, the migration task M1 with the smallest start time is obtained, the migration task M2 with the second smallest start time is obtained, for the time period after the start of M1 and before the start of M2, the change feature of the throughput curve of each business module in the time period is obtained, which is recorded as the first throughput change feature of the migration task M1; obtaining a third smallest migration task M3 at a starting time, obtaining a change characteristic of a throughput curve of each service module in a time period after the starting of M2 and before the starting of M3, denoted as Q, obtaining a sum of first throughput change characteristics of running migration tasks other than M2 in the time period, denoted as Q0, and taking Q-Q0 as the first throughput change characteristic of the migration task M2; obtaining a fourth smallest migration task M4 at a starting time, obtaining a change characteristic of a throughput curve of each service module in a time period after the starting of M3 and before the starting of M4, denoted as Q, obtaining a sum of first throughput change characteristics of running migration tasks other than M3 in the time period, denoted as Q0, and taking Q-Q0 as the first throughput change characteristic of the migration task M3; obtaining a fourth smallest migration task M4 at a starting time, obtaining a change characteristic of a throughput curve of each service module in a time period after the starting of M3 and before the starting of M4, denoted as Q, obtaining a sum of first throughput change characteristics of running migration tasks other than M3 in the time period, denoted as Q0, and taking Q-Q0 as the first throughput change characteristic of the migration task M3;

[0032] It should be noted that in the embodiment, every second is a time.

[0033] At this point, the first throughput change characteristic of each migration task is obtained. The greater the first throughput change characteristic, the more obvious the change in the throughput of each service module caused by the migration task. The resource consumption strength of each migration task is positively correlated with the size of the first throughput change characteristic of each migration task.

[0034] As an example, the method for obtaining the resource consumption strength of each migration task comprises: For all migration tasks in all service modules, the first throughput change characteristics of all migration tasks are normalized to obtain the resource consumption strength of each migration task. In the embodiment, the softmax formula is used for normalization.

[0035] As an example, the method for obtaining the change characteristic of the throughput curve of each service module in the time period comprises: The absolute value of the slope of the throughput curve of each service module in the time period is taken as the change characteristic. The method for obtaining the slope is as follows: for the throughput curve in the time period, the least square method is used to fit a straight line, and the slope is the slope of the straight line.

[0036] As another example, the method for obtaining the change characteristic of the throughput curve of each service module in the time period comprises: The difference between the maximum value and the minimum value of the throughput curve of each service module in the time period is taken as a change feature.

[0037] In particular, when the difference between the maximum value and the minimum value of a plurality of start times is less than 3, the plurality of start times are regarded as the same start time, and the migration tasks corresponding to the start times are regarded as the same migration task.

[0038] In step S103, a resource consumption estimation coefficient of each service module is obtained according to the change relationship between each throughput curve and the available computing resource curve.

[0039] The resource consumption estimation coefficient describes whether each service module will affect the available computing resource curve when performing a migration task in the Xinhua migration process. Different service modules have different business logic and database access when operating services, and the impact on the available computing resource curve when migrating databases is also different. The greater the resource consumption estimation coefficient of each service module, the more the service module will significantly change the computing resource curve when migrating databases. The smaller the resource consumption estimation coefficient of each service module, the less the service module will significantly change the computing resource curve when migrating databases.

[0040] As an optional example, the method for calculating the resource consumption estimation coefficient of each service module includes: In a first preset time period before the current time, a Pearson correlation coefficient between each throughput curve and the available computing resource curve is obtained. The greater the Pearson correlation coefficient, the more the service module will affect the use of computing resources when migrating. In this embodiment, the normalized Pearson correlation coefficients of all throughput curves are used as the resource consumption estimation coefficients of each service module. In this embodiment, the softmax formula is used for normalization.

[0041] The first preset time period in this embodiment refers to one hour before the current time (including the current time).

[0042] In step S104, when the available computing resource curve is less than a first preset threshold, a plurality of migration tasks are screened out from each service module according to the resource consumption estimation coefficient of each service module and the resource consumption strength of each migration task, and are suspended.

[0043] When the available computing resource of the available computing resource curve at the current time is less than the first preset threshold th1, it indicates that there is a problem of computing resource shortage, which is not conducive to the normal operation of all service modules and the execution of the migration process. For example, there may be a risk of serious user response lag or unexpected interruption of the migration process. In this embodiment, th1 is equal to 0.23.

[0044] At this time, according to the resource consumption estimation coefficient of each service module, a number of migration tasks are screened out from each service module for suspension.

[0045] The embodiment screens migration tasks for suspension based on the resource consumption estimation coefficient of each service module to release computing resources and ensure the smooth running of service modules and other migration tasks. Different service modules have different influences on the change of available computing resources when performing normal services and performing migration tasks (i.e., the resource consumption estimation coefficients of different service modules are different), so the screening methods of different service modules for screening suspended migration tasks are different (i.e., the resource consumption intensity of the screened migration tasks, the remaining task amount, etc. are different) when ensuring the smooth running of service modules and other migration tasks. The process of screening appropriate migration tasks from different service modules for suspension is equivalent to reasonable allocation of computing resources.

[0046] As an example, according to the resource consumption estimation coefficient of each service module, a number of migration tasks are screened out from each service module for suspension, including the following method: Obtain the percentage of the remaining data amount of each migration task, which is denoted as the remaining task amount of each migration task.

[0047] Randomly select a number of migration tasks from each service module, and the number of migration tasks selected from different service modules may be different. For all the selected migration tasks in all service modules, these migration tasks are denoted as a group of tasks to be suspended. For the number of migration tasks selected from each service module in the group of tasks to be suspended, the sum of the resource consumption intensity of the migration tasks is denoted as a first index H1, and the sum of the remaining task amount of the migration tasks is denoted as a second index H2. The larger the first index, the more the selected migration tasks help to avoid the current computing resources from continuing to decrease, and the larger the second index, the more it helps the subsequent migration process to continuously affect the computing resources.

[0048] H1+m×H2 is denoted as the suspension index of each service module. m represents a preset migration attention coefficient, which is used to represent whether the continuous influence of the subsequent migration process on the computing resources needs to be paid attention to. In the embodiment, the initial value of m is set to 0.8.

[0049] The larger the suspension index, the more the selected migration tasks can reduce the risk that the continuous decrease of computing resources leads to the failure of the normal services and the migration process to proceed smoothly (i.e., the risk of computing resource shortage or deficiency) after the selected migration tasks are suspended.

[0050] The pause indicators of each service module are calculated respectively, and then the consistency of the group of tasks to be paused is calculated according to the size relationship of the pause indicators of different service modules and the size relationship of the resource consumption estimation coefficients between different service modules.

[0051] The consistency is used to describe whether the suspension of the selected migration tasks meets the influence of the change of the available computing resources on different service modules.

[0052] The greater the consistency, the more the selection method of the migration tasks can select the migration tasks that significantly reduce the risk of computing resource shortage from the service modules that have a greater impact on the available computing resources. The suspension of these migration tasks helps to ensure that the normal business and the migration process cannot proceed smoothly. The smaller the consistency, the less the selection method of the migration tasks selects appropriate migration tasks based on the service modules that have a greater impact on the available computing resources. The suspension of these migration tasks cannot reduce the risk of resource shortage, which is not conducive to the normal business and the migration process to proceed smoothly.

[0053] The migration tasks are randomly selected from all tasks until all tasks to be paused are obtained. Each group of tasks to be paused corresponds to a consistency. The tasks to be paused when the consistency is the largest are obtained. The tasks to be paused contain several migration tasks selected from each service module. These migration tasks are used as the suspended migration tasks in each service module. The suspended migration tasks will not execute the migration process in the future.

[0054] In this process, by selecting appropriate migration tasks from different service modules for suspension, the computing resources are reasonably allocated. In the case of as little as possible affecting the migration process, the problem of hardware resource shortage that is not conducive to the normal business operation and the migration process execution of all service modules is avoided, and the overall hot migration efficiency of the ChinaSoft is improved.

[0055] After suspending part of the migration tasks in each service module according to the above method, the average of the available computing resources of all time points in the recent time points (for example, the last 5 time points) is obtained every certain time point (for example, after every 5 time points). When the average is greater than a second preset threshold th2, all suspended migration tasks in all service modules are restarted to continue the migration process. Wherein th2 is greater than th1, and the embodiment is described by taking th2=0.45 as an example.

[0056] As an optional example, the consistency of the group of tasks to be paused is calculated according to the size relationship of the pause indicators of different service modules and the size relationship of the resource consumption estimation coefficients between different service modules, which includes the following method: The resource consumption estimation coefficients of all the service modules form a first sequence, and the suspension indexes of all the service modules form a second sequence. The Pearson correlation coefficient between the first sequence and the second sequence is taken as the consistency. It should be noted that the same position in the first sequence and the second sequence corresponds to the same service module.

[0057] As a preferred example, the consistency of the group of tasks to be suspended is calculated according to the size relationship of the suspension indexes of different service modules and the size relationship of the resource consumption estimation coefficients between different service modules. The method comprises the following steps: n service modules are randomly selected from all the service modules. In this embodiment, n is equal to half (rounded down) of the number of all the service modules. The n service modules form a module combination. All the module combinations are obtained. The resource consumption estimation coefficients of all the service modules in each module combination form a first sequence, and the suspension indexes of all the service modules in each module combination form a second sequence. The Pearson correlation coefficient between the first sequence and the second sequence is taken as the correlation coefficient of each module combination. The average of the correlation coefficients of all the module combinations is taken as the consistency.

[0058] In some embodiments, when the number of service modules is large and the number of migration tasks allowed to run simultaneously in each service module is large, the process of constantly randomly selecting migration tasks from all the tasks until all the tasks to be suspended are obtained has a large amount of calculation. At this time, a method for reducing the amount of calculation is as follows: When a migration task is randomly selected in each service module, the selected migration task is at most 4. All the migration tasks randomly selected from all the service modules are taken as a group of tasks to be suspended. Constantly randomly selecting migration tasks from all the tasks until 20 groups of tasks to be suspended are obtained, obtaining the task to be suspended with the largest consistency. The task to be suspended contains a number of migration tasks selected from each service module. The migration tasks are taken as the suspended migration tasks in each service module, and the suspended migration tasks do not perform the migration process in the subsequent process.

[0059] Thus, this embodiment ends.

[0060] In this embodiment, suitable migration tasks are selected from each service module for suspension, and then the calculation resources of different service modules are redistributed, so that when the calculation resources are tight, the risks caused by the tight calculation resources can be inhibited by reasonably allocating the calculation resources.

[0061] The embodiment has the greatest consistency between the size relationship of the suspension indicators of the suspended migration tasks in different business modules and the size relationship of the resource consumption estimation coefficients between different business modules when the embodiment screens out some suspended migration tasks from each business module. The suspension indicator of the suspended migration task is obtained from the resource consumption intensity of the suspended migration task and the remaining task amount of the suspended migration task. The process can select migration tasks that can significantly reduce the risk of computing resource shortage from business modules that have a greater impact on available computing resources. The suspension of these migration tasks helps to ensure that normal business and migration processes cannot proceed smoothly; avoids selecting migration tasks that cannot reduce the risk of computing resource shortage in business modules that have a smaller impact on available computing resources, and the suspension of these migration tasks will still pose a risk to normal business and migration processes. In summary, the process avoids the problem of hardware resource shortage that is not conducive to the normal business operation and migration process execution of all business modules as much as possible without affecting the migration process. Embodiment

[0062] In the embodiment one, when the available computing resources are in shortage (i.e., the available computing resources at the current time are less than the first preset threshold th1), the appropriate migration tasks are screened out from each business module for suspension, thereby relieving the risk of insufficient computing resources at the current time (i.e., suppressing the trend of the available computing resources continuing to decrease at the current time). However, the available computing resources may not be significantly recovered in a short time after the current time (i.e., the available computing resources are recovered in a short time), resulting in a risk of insufficient computing resources in the future. The reason is that the resource consumption estimation coefficient and the suspension indicator of each business module are incorrectly evaluated in the embodiment one, or the resource consumption estimation coefficient and the suspension indicator in the embodiment one have obvious errors in the obtained results, resulting in that the suspended migration tasks screened out from each business module are not optimal, for example, only suitable for the current time, but not suitable for the migration process after the current time.

[0063] The embodiment reselects the suspended migration tasks by performing the following steps to ensure that the migration process after the current time can be executed smoothly.

[0064] In step S105, the difference between the suspension indicator of the non-suspended migration task in each business module and the suspension indicator of the suspended migration task is recorded as a first difference. After suspending all the migration tasks, the suspended migration tasks are reselected by using the change amount of the available computing resource curve with the throughput curve of each business module and the first difference.

[0065] For each service module, the migration tasks therein are divided into suspended migration tasks and non-suspended migration tasks; the suspension indicator of the non-suspended migration tasks is marked as R1; the suspension indicator of the suspended migration tasks is marked as R2 (i.e. the suspension indicator of each service module obtained in step S104).

[0066] The method for obtaining the suspension indicator of the non-suspended migration tasks is the same as the method for obtaining the suspension indicator of each service module, and the specific process is as follows: the sum of the resource consumption intensities of the non-suspended migration tasks is obtained and marked as a first indicator H3, and the sum of the remaining task quantities of the non-suspended migration tasks is marked as a second indicator H4. R1 = H3 + m x H4, where m represents a preset migration attention coefficient.

[0067] (R1-R2) / R2 is marked as a first difference, which is used to describe whether the migration resources are reasonably allocated in the subsequent migration process after the suspended part of the migration tasks in each service module is suspended.

[0068] It should be noted that for the suspended migration tasks, when the available computing resource curve is greater than the second preset threshold th2, the suspended migration tasks need to be re-enabled after being suspended, i.e. the suspended tasks are restarted after the computing resources are recovered (which is specifically recorded in step S104 of Embodiment 1).

[0069] Based on this, the larger the first difference is, i.e. the smaller the suspension indicator of the suspended migration tasks is and the larger the suspension indicator of the non-suspended migration tasks is. The smaller the suspension indicator of the suspended migration tasks is, the smaller the resource consumption intensity of the suspended migration tasks is and the smaller the remaining migration data quantity of the suspended migration tasks is, and at this time, after the computing resources are recovered and the suspended migration tasks are restarted, the restarted migration tasks will not cause a serious risk of insufficient computing resources to the migration process. The larger the suspension indicator of the non-suspended migration tasks is, the larger the resource consumption intensity of the non-suspended migration tasks is and the larger the remaining migration data quantity of the non-suspended migration tasks is, and at this time, before the computing resources are recovered, the non-suspended migration tasks still have a serious risk of insufficient computing resources to the migration process. Therefore, the larger the first difference is, the more it indicates that the risk of insufficient computing resources is not reasonably distributed to the following two processes: the process from the suspension of the migration tasks to the recovery of the computing resources, and the process of restarting the suspended migration tasks after the recovery of the computing resources; this leads to the fact that even if the current time avoids the continuous decrease of the available computing resources by suspending part of the migration tasks, it is still possible that the subsequent process cannot be guaranteed to have a high risk of insufficient computing resources.

[0070] On the contrary, when the first difference is larger, it means that the risk of insufficient computing resources is reasonably allocated to the process before the computing resources are recovered and the process after the computing resources are recovered. At the current moment, part of the migration tasks are suspended to avoid the available computing resources from continuing to decrease, and the subsequent process can be effectively ensured without a high risk of insufficient computing resources.

[0071] Based on this, all suspended migration tasks in each business module are suspended at the current moment, and after a number of moments (for example, after 5 moments, which is the same as the number of moments described in step S104 of Embodiment 1), the average of the available computing resources in the available computing resource curve in the number of moments (that is, in the 5 moments elapsed) is obtained. When the average is greater than or equal to the second preset threshold th2, it means that the computing resources are significantly recovered, and at this time, all suspended migration tasks in the business module are restarted.

[0072] When the average is less than the second preset threshold th2, it means that the computing resources have not been significantly recovered, that is, the method of selecting suspended migration tasks from each business module in the above-mentioned Embodiment 1 is not optimal, that is, there is an error or inaccuracy in the process of obtaining the resource consumption estimation coefficient and the suspension index in Embodiment 1.

[0073] At this time, the change amount of the available computing resource curve with the throughput curve of each business module is obtained, denoted as the first change amount of each business module. The difference between the first change amount and the first difference is denoted as the correction coefficient of each business module.

[0074] For the business module with a larger correction coefficient, the change of the throughput of the business module significantly affects the change of the available computing resource curve, which means that suspending the migration tasks from the business module can effectively solve the problem of insufficient computing resources, and the suspended migration tasks will not have a significant risk of insufficient computing resources in the subsequent process. Based on this, the present embodiment can increase the resource consumption estimation coefficient of the business module, and at the same time, reduce the migration attention coefficient m in the process of obtaining the suspension index of the business module, and then reacquire the suspended migration tasks.

[0075] Wherein, increasing the resource consumption estimation coefficient of the business module helps to suspend the migration tasks from the business module to further alleviate the problem of no significant increase in computing resources; reducing the migration attention coefficient m can make the reacquired suspended migration tasks no longer focus on the remaining data amount of the migration tasks (that is, no longer focus on the influence of the subsequent migration process on the migration tasks), but focus more on the consumption of resources by the migration tasks, so as to further alleviate the problem of no significant increase in computing resources.

[0076] For the service module with a small correction coefficient, the change of the throughput of the service module cannot obviously affect the change of the available computing resource curve, indicating that pausing the migration task from the screening of the service module cannot effectively solve the problem of insufficient computing resources, and the paused migration task has a significant risk of insufficient computing resources in the subsequent process. Based on this, the embodiment can reduce the resource consumption estimation coefficient of the service module, and increase the migration attention coefficient m in the acquisition process of the pause index of the service module, and then reacquire the paused migration task.

[0077] Wherein, reducing the resource consumption estimation coefficient of the service module means that the paused migration task from the service module is no longer relied on to further alleviate the problem that the computing resources do not have obvious growth. At this time, the service module is allowed to continue to run the resource consumption relatively large migration task, ensuring that the resource consumption relatively large migration task can be executed in time, so that it no longer participates in the subsequent migration process, and ensures the smooth execution of the subsequent migration process. Increasing the migration attention coefficient m means that the migration task with a large remaining data volume in the service module is mainly paused (that is, the migration task that mainly pauses the subsequent migration process is mainly paused), avoiding the situation that the subsequent computing resources are not continuously relieved.

[0078] In summary, by updating the resource consumption estimation coefficient of each service module and the migration attention coefficient m in the acquisition process of the pause index, the situation that the computing resources are not continuously relieved is avoided, and the risk of continuous tension of computing resources is reduced. The update result of the resource consumption estimation coefficient is positively correlated with the correction coefficient, and the update result of the migration attention coefficient m is negatively correlated with the correction coefficient.

[0079] Further, the update result of the resource consumption estimation coefficient and the update result of the migration attention coefficient m are used to re-screen the paused migration task in each service module according to the method of embodiment one. Then the paused migration tasks are paused at the current time.

[0080] It should be noted that the paused migration tasks obtained in embodiment one and the paused migration tasks obtained in embodiment two can be different. The paused migration tasks obtained in embodiment one are denoted as S1, and the paused migration tasks obtained in embodiment two are denoted as S2. All the migration tasks in S2 are paused, and if there is a migration task contained in S1 but not contained in S2, the migration task is re-enabled. In addition, if the migration task in S2 has been executed, it is deleted from S2.

[0081] It should be noted that, in the embodiment, the resource consumption strength of each migration task is also used when re-screening the suspended migration tasks in each service module, and the resource consumption strength is the settlement result in the above-mentioned embodiment one. In addition, it should be noted that the current time in embodiment one and the current time when the suspended migration tasks are re-screened in embodiment two are not the same time, and the latter is after the former by several times (for example, after 5 times).

[0082] As an example, the change amount of the available computing resource curve with the change amount of the throughput curve of each service module is obtained, denoted as the first change amount of each service module, including the following steps: The available computing resource curve and the throughput curve in the several times after the migration task is suspended are intercepted.

[0083] The intercepted available computing resource curve and the throughput curve are normalized respectively, and the softmax formula is used for normalization in the embodiment, and the purpose is to remove the dimension of the available computing resource curve and the throughput curve.

[0084] The difference between the maximum value and the minimum value of the intercepted and normalized available computing resource curve is denoted as C1, and the difference between the maximum value and the minimum value of the intercepted and normalized throughput curve is denoted as C2, and C1 / (C2+1) is denoted as the first change amount. The purpose of adding one to the denominator is to avoid the denominator being zero.

[0085] As a preferred example, the resource consumption estimation coefficient of each service module and the migration attention coefficient m in the process of obtaining the suspension index are updated, wherein the update result of the resource consumption estimation coefficient is positively correlated with the correction coefficient, and the update result of the migration attention coefficient m is negatively correlated with the correction coefficient, including the following steps: The correction coefficients of all service modules are linearly normalized, the resource consumption estimation coefficient of each service module is denoted as x1, the normalized correction coefficient in each service module is denoted as w1, the value of the migration attention coefficient m in each service module is denoted as m1, the update result of the resource consumption estimation coefficient is x1×(1+w1-w0), and the update result of the migration attention coefficient m is m1×(1-w1+w0). Wherein w0 represents a preset reference value, when w1 is greater than w0, x1 is increased and m1 is decreased; when w1 is less than w0, x1 is decreased and m1 is increased; in the embodiment, w0 is set to 0.5, so that in the service module with a larger correction coefficient, the resource consumption estimation coefficient of the service module is increased, and the migration attention coefficient m in the service module is decreased; in the service module with a smaller correction coefficient, the resource consumption estimation coefficient of the service module is decreased, and the migration attention coefficient m in the service module is increased. In other embodiments, the preferred value range of w0 is (0, 1).

[0086] As an optional example, the resource consumption estimation coefficient of each service module is updated, and the migration attention coefficient m in the acquisition process of the index is suspended; wherein the update result of the resource consumption estimation coefficient is positively correlated with the correction coefficient, the update result of the migration attention coefficient m is negatively correlated with the correction coefficient, and the method comprises the following steps: The correction coefficients of all service modules are linearly normalized, the resource consumption estimation coefficient of each service module is denoted as x1, the normalized correction coefficient in each service module is denoted as w1, and the value of the migration attention coefficient m in each service module is denoted as m1.

[0087] When w1 is greater than or equal to 0.5, the update result of the resource consumption estimation coefficient is x1*1.5, and the update result of the migration attention coefficient m is m1*1.5. When w1 is less than 0.5, the update result of the resource consumption estimation coefficient is x1*0.5, and the update result of the migration attention coefficient m is m1*0.5.

[0088] Thus, the embodiment ends.

[0089] In the embodiment, after suspending the migration task, the migration task is reacquired for suspension, which avoids the problem that the available computing resources cannot be obviously recovered in a short time and the risk of insufficient computing resources still exists. The problem of insufficient available computing resources is inhibited, and the computing resources are quickly recovered, further reducing the risk in the normal business operation and the migration process. Embodiment two:

[0090] The embodiment provides a full-process monitoring system for service-oriented service innovation migration, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor executes all steps of all embodiments when executing the computer program.

[0091] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for monitoring the whole process of service-oriented signal innovation migration, characterized in that, The method includes the following steps: Monitor the available computing resource curves of several business modules and the throughput curve of each business module. Each business module contains several migration services. Calculate the resource consumption intensity of each migration task. Obtain the resource consumption estimation coefficient of each business module based on the relationship between the throughput curve and the available computing resource curve. When the available computing resources curve is less than a first preset threshold, a number of migration tasks are screened out from each business module and paused. The relationship between the pause index of the paused migration tasks in different business modules has the greatest consistency with the relationship between the resource consumption estimation coefficients of different business modules. The pause index of the paused migration task is obtained by the resource consumption intensity of the paused migration task and the remaining task quantity of the paused migration task. The difference between the pause index of the non-paused migration task and the pause index of the paused migration task in each business module is recorded as the first difference. After all migration tasks are paused, the change in the available computing resource curve with the throughput curve of each business module is recorded as the first change. The difference between the first change and the first difference is used to correct the resource consumption estimation coefficient and the pause index obtained under each business module, and the paused migration tasks are re-acquired and paused.

2. The method of claim 1, wherein the method further comprises: The specific steps involved in calculating the resource consumption of each migration task are as follows: For all migration tasks currently running in each business module, each migration task corresponds to a start time. The migration task with the smallest start time, M1, is obtained, and the migration task with the second smallest start time, M2, is obtained. For the time period after M1 starts and before M2 starts, the throughput curve of each business module changes within the time period, and is recorded as the first throughput change characteristic of migration task M1. Get the third smallest migration task M3 at the start time; for the time period after M2 starts and before M3 starts, get the change characteristics of the throughput curve of each business module during the time period, denoted as Q; get the sum of the first throughput change characteristics of the running migration tasks other than M2 during the time period, denoted as Q0, and record Q-Q0 as the first throughput change characteristic of migration task M2. And so on, until the largest migration task M at the start time; for the time period after M starts and before the current time, obtain the change characteristics of the throughput curve of each business module within the time period, still denoted as Q; obtain the sum of the first throughput change characteristics of the running migration tasks outside M within the time period, still denoted as Q0, and record Q-Q0 as the first throughput change characteristic of migration task M. The resource consumption of each migration task is positively correlated with the magnitude of the first throughput change characteristic of each migration task.

3. The method of claim 1, wherein the method further comprises: The specific steps for obtaining the resource consumption estimation coefficient for each business module based on the relationship between each throughput curve and the available computing resource curve are as follows: The Pearson correlation coefficient of each throughput curve and the available computing resource curve is obtained within a first preset time period before the current time, the Pearson correlation coefficients corresponding to all throughput curves are normalized, and the normalized Pearson correlation coefficients are used as resource consumption estimation coefficients of each service module.

4. The method of claim 1, wherein the method further comprises: The migration tasks in each service module are suspended, and the size relationship of the suspension indicators of the suspended migration tasks in different service modules has maximum consistency with the size relationship of the resource consumption estimation coefficients between different service modules. The specific steps include the following: A plurality of migration tasks are randomly selected from each service module, the suspension indicators of the selected migration tasks are calculated, and the suspension indicators of the selected migration tasks are obtained from the resource consumption intensity of the selected migration tasks and the residual task quantity of the selected migration tasks; The suspension indicators of the selected migration tasks are used as the suspension indicators of each service module; and all the migration tasks randomly selected from all the service modules are used as a group of tasks to be suspended. The consistency of the tasks to be suspended is calculated according to the size relationship of the suspension indicators of different service modules and the size relationship of the resource consumption estimation coefficients between different service modules. The tasks to be suspended with maximum consistency are obtained, and the migration tasks selected from each service module and contained in the tasks to be suspended with maximum consistency are used as the suspended migration tasks in each service module.

5. The method of claim 4, wherein the method further comprises: The specific steps for obtaining the suspension indicators include the following: The selected migration tasks, the suspended migration tasks, or the non-suspended migration tasks constitute a migration task set. The sum of the resource consumption intensities of all the migration tasks in the migration task set is used as a first indicator H1, and the sum of the residual task quantities of all the migration tasks in the migration task set is used as a second indicator H2; H1+m×H2 is used as the suspension indicator of each service module, and m represents a preset migration attention coefficient of each service module.

6. The method of claim 5, wherein the method further comprises: The resource consumption estimation coefficients and the suspension indicators obtained under each service module are corrected by using the difference between the first change amount and the first difference, and the suspended migration tasks are reselected for suspension. The specific steps include the following: The difference between the first change amount and the first difference is used as a correction coefficient of each service module. The resource consumption estimation coefficients of each service module are updated by using the correction coefficient of each service module, and the migration attention coefficient in each service module is updated; wherein the update result of the resource consumption estimation coefficient is positively correlated with the correction coefficient, and the update result of the migration attention coefficient is negatively correlated with the correction coefficient. The suspended migration tasks are reselected from each service module by using the update result of the resource consumption estimation coefficient and the update result of the migration attention coefficient.

7. The method of claim 2, wherein the method further comprises: The change characteristics of the throughput curve of each service module within the time period are obtained. The specific steps include the following: The absolute value of the slope of the throughput curve of each service module within the time period is used as the change characteristic; wherein the slope is obtained by fitting the throughput curve within the time period into a straight line by using the least square method, and the slope is the slope of the straight line.

8. The method of claim 4, wherein the method further comprises: The consistency of the to-be-suspended task is calculated according to the size relationship between the suspension indexes of different service modules and the size relationship between the resource consumption estimation coefficients of different service modules, and the specific steps include the following: Randomly select a plurality of service modules from all service modules to form a module combination; for all module combinations; the resource consumption estimation coefficients of all service modules in each module combination form a first sequence, and the suspension indexes of all service modules in each module combination form a second sequence; the Pearson correlation coefficient of the first sequence and the second sequence is taken as the correlation coefficient of each module combination, and the average of the correlation coefficients of all module combinations is taken as the consistency.

9. The method of claim 6, wherein the method further comprises: The resource consumption estimation coefficient of each service module is updated by using the correction coefficient of each service module, and the migration attention coefficient in each service module is updated, and the specific steps include the following: The correction coefficients of all service modules are linearly normalized, the resource consumption estimation coefficient of each service module is denoted as x1, the normalized correction coefficient in each service module is denoted as w1, and the value of the migration attention coefficient in each service module is denoted as m1; the update result of the resource consumption estimation coefficient is x1*(1+w1-w0), and the update result of the migration attention coefficient m is m1*(1-w1+w0); wherein w0 represents a preset reference value.

10. A full-process monitoring system for business service innovation migration, the system comprising: The memory, the processor, and the computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to perform all steps of the full-process monitoring method for service-oriented service creation migration according to any one of claims 1-9.

Images