Cross-boundary platform data processing method and system

Abstract

The invention relates to the technical field of cross-border platform data processing, in particular to a cross-border platform data processing method and system. The invention provides a cross-boundary platform data processing method and system. The cross-boundary platform data processing method comprises the following steps: acquiring a first data set of equipment to obtain a first equipment set, a second equipment set and a third equipment set; checking an updating notification of the equipment to obtain first updating equipment, second updating equipment and a standby equipment set; obtaining a second data set of the equipment, and obtaining first standby equipment and second standby equipment based on the first updating equipment, the second updating equipment and the standby equipment set; and based on the first updating device, the second updating device, the first standby device and the second standby device, carrying out device switching. According to the method, the device sets are divided by obtaining the multi-dimensional data set of the device, the updating device and the standby device are determined, and the effects of accurately managing device resources and improving the working efficiency are achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cross-border platform data processing, and particularly relates to a cross-border platform data processing method and system. BACKGROUND

[0002] Under the impetus of digital transformation, cross-border platforms have gradually become the key carriers for enterprises to realize resource integration and business innovation. Such platforms integrate equipment and data resources from different industries and different technical architectures, aiming to mine data value and improve operational efficiency through data sharing and collaborative processing. However, in actual operation, cross-border platform data processing faces many technical bottlenecks.

[0003] Currently, at the equipment management level, traditional data processing modes are difficult to achieve fine control of equipment resources. As the platform scale continues to expand, the number of equipment increases and the types are diverse, and the real-time running state, task execution and resource allocation of the equipment are difficult to be accurately grasped. At the same time, the contradiction between equipment updating and maintenance and business continuity is increasingly prominent. When the equipment needs to be updated, due to the lack of effective standby equipment screening mechanism and task migration strategy, it often leads to business interruption, data loss or processing delay, seriously affecting the quality of platform services.

[0004] Therefore, it is urgent to develop a cross-border platform data processing method and system. SUMMARY

[0005] In order to overcome the shortcoming that the task time increases due to equipment updating, the present application provides a cross-border platform data processing method and system.

[0006] The technical scheme is as follows: a cross-border platform data processing method, comprising the following steps:

[0007] S1: obtaining a first data set of equipment, obtaining a first equipment set, a second equipment set and a third equipment set;

[0008] S2: checking the update notification of the equipment, obtaining a first updated equipment, a second updated equipment and a standby equipment set;

[0009] S3: obtaining a second data set of the equipment, based on the first updated equipment, the second updated equipment and the standby equipment set, obtaining a first standby equipment and a second standby equipment;

[0010] S4: based on the first updated equipment, the second updated equipment and the first standby equipment, the second standby equipment, performing equipment switching.

[0011] Preferably, in S1, the first data set of the equipment is obtained, and the first equipment set, the second equipment set and the third equipment set are obtained, comprising:

[0012] The first data set of the acquisition device includes real-time operation parameters, task execution status, state representation signals, task scheduling information and resource allocation signals.

[0013] The first device set is a set of devices that are working.

[0014] The second device set is a set of devices that are about to work.

[0015] The third device set is a set of devices that have no work arrangement.

[0016] Preferably, the update notification of the checking device in S2 obtains the first update device, the second update device and the standby device set, including:

[0017] The update notification of the checking device in real time obtains the devices that need to be updated, and based on the first device set, the second device set and the third device set, the first update device, the second update device and the standby device set are obtained, and the standby device set is a device that does not need to be updated and normally executes work.

[0018] Preferably, the second data set of the acquisition device in S3 obtains the first standby device and the second standby device based on the first update device, the second update device and the standby device set, including:

[0019] S31: Obtain the second data set of the device based on the first update device, the second update device and the standby device set, and obtain the first standby device set and the second standby device set.

[0020] S32: Obtain the third data set of the device based on the first standby device set and the second standby device set, and obtain the first standby device and the second standby device.

[0021] The second data set of the device includes hardware configuration, software environment and load state.

[0022] The third data set of the device includes historical interruption times and average recovery time.

[0023] Preferably, the second data set of the acquisition device obtains the first standby device set and the second standby device set based on the first update device, the second update device and the standby device set, including:

[0024] Obtain the second data set of the device based on the first update device, the second update device and the standby device set by the formula:

[0025] Sim(A, B i ) = α × F hardware (A, B i ) + β × F software (A, B i)+γ×F load (A, B i );

[0026] get the first backup device set and the second backup device set, wherein, Sim(A, B i ) is a similarity function of the first update device, the second update device A and the backup device B i , α+β+γ=1, and α, β, γ ∈ [0, 1] are weight coefficients of the hardware configuration, the software environment and the load state respectively, F hardware (A, B i ) represents a hardware configuration similarity function of the first update device, the second update device A and the backup device B i , F software (A, B i ) represents a software environment similarity function, F load (A, B i ) represents a load state similarity function.

[0027] Preferably, the third data set of the device is obtained based on the first backup device set and the second backup device set, and the first backup device and the second backup device are obtained, comprising:

[0028] The third data set of the device is obtained based on the first backup device set and the second backup device set, and the first backup device and the second backup device are obtained by the formula:

[0029]

[0030] get the working continuity of each backup device in the backup device set, wherein, Cont(B i ) is a working continuity coefficient of the backup device B i , w N and w T are weight coefficients, is the historical interruption times of the backup device B i , is the average recovery time of the backup device B i ;

[0031] Score(B i ) = w × Sim(A, B i ) + (1-w) × Conv(B i );

[0032] get the first backup device and the second backup device, wherein Score(B i ) is a comprehensive score of the backup device B i , w is a weight coefficient of similarity, 0 ≤ w ≤ 1, and the backup device with the highest comprehensive score is selected as the first backup device and the second device.

[0033] Preferably, the device switching based on the first update device, the second update device and the first standby device and the second standby device in S4 comprises:

[0034] During the device switching process, a breakpoint continuation and data checking mechanism is adopted to monitor the integrity of data transmission and the continuity of task execution in real time.

[0035] After the device switching is completed, the performance of the first standby device and the second standby device is monitored, and if performance abnormalities occur, an emergency rollback mechanism is started to switch the task to the first update device or to reselect a standby device.

[0036] A cross-platform data processing system comprises:

[0037] A data acquisition module is configured to acquire a first data set, a second data set and a third data set of a device.

[0038] A device state analysis module is configured to divide a first device set, a second device set, a third device set, a first update device, a second update device and a standby device set, and to determine a first standby device set, a second standby device set, a first standby device and a second standby device.

[0039] A device switching control module is configured to perform a device switching operation, and to monitor and manage the switching process and the device after switching.

[0040] The present application has the following advantages: the present application first acquires a multi-dimensional first data set of a device and divides a device set, thereby achieving clear understanding of the distribution and use of device resources, achieving the effect of accurately managing device resources and improving resource utilization, and then selecting standby devices based on a multi-dimensional data set and a formula, cooperating with a data transmission and switching guarantee mechanism, thereby achieving selection of high-quality devices for data processing and ensuring reliable transmission, and achieving the effect of improving data transmission efficiency and processing reliability. BRIEF DESCRIPTION OF DRAWINGS

[0041] Figure 1 A flowchart of a cross-platform data processing method of the present application is shown.

[0042] Figure 2 A structural diagram of a cross-platform data processing system of the present application is shown. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present application will be clearly and completely described below. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0044] Embodiment 1: A cross-platform data processing method, as shown in Figure 1 comprises the following steps:

[0045] S1: Obtain a first data set of the device, obtain a first device set, a second device set and a third device set;

[0046] The first data set of the device comprises real-time running parameters, task execution states, state representation signals, task scheduling information and resource allocation signals;

[0047] The first device set is a device set that is working;

[0048] The second device set is a device set that is about to work;

[0049] The third device set is a device set that has no work arrangement.

[0050] It should be noted that in the process of device management and scheduling, obtaining the first data set of the device is the basis for efficient management and accurate scheduling. This set covers five core data types, namely real-time running parameters, task execution states, state representation signals, task scheduling information and resource allocation signals, which each assume different responsibilities and together provide the basis for accurate determination of the device state.

[0051] S2: Check the update notification of the device, obtain a first updated device, a second updated device and a standby device set;

[0052] Real-time check the update notification of the device to obtain the device that needs to be updated, based on the first device set, the second device set and the third device set, obtain the first updated device, the second updated device and the standby device set, and the standby device set is a device that does not need to be updated and normally executes work.

[0053] It should be noted that the real-time check device update notification is to ensure that the device is always in the best running state, timely repair vulnerabilities, and improve performance. For the first update device, if the update operation is directly performed, it may cause the task being executed to be interrupted, so it is necessary to select a suitable replacement device from the standby device set to ensure the continuity of the task. The second update device can complete the update before starting the task, reducing the impact on the overall workflow. The existence of the standby device set aims to quickly replace the first update device and / or the second update device when the device needs to be updated, and maintain the stable operation of the system.

[0054] S3: Obtain a second data set of the device, and obtain a first standby device and a second standby device based on the first update device, the second update device, and the standby device set;

[0055] S31: Obtain a second data set of the device, and obtain a first standby device set and a second standby device set based on the first update device, the second update device, and the standby device set;

[0056] S32: Obtain a third data set of the device, and obtain a first standby device and a second standby device based on the first standby device set and the second standby device set;

[0057] The second data set of the device includes hardware configuration, software environment, and load state;

[0058] The third data set of the device includes historical interruption times and average recovery time.

[0059] The second data set of the device is obtained based on the first update device, the second update device, and the standby device set, and the first standby device set and the second standby device set are obtained, including:

[0060] The second data set of the device is obtained based on the first update device, the second update device, and the standby device set, and the first standby device set and the second standby device set are obtained, including:

[0061] Sim(A, B i ) = α × F hardware (A, B i ) + β × F software (A, B i ) + γ × F load (A, B i );

[0062] The first standby device set and the second standby device set are obtained, wherein Sim(A, B i ) is a similarity function of the first update device, the second update device A, and the standby device B i , α + β + γ = 1, and weight coefficients of hardware configuration, software environment and load state, respectively, F hardware (A, B i ) represents the hardware configuration similarity function of the first update device, the second update device A and the standby device B i software (A, B i ) represents the software environment similarity function, F load (A, B i ) represents the load state similarity function.

[0063] It should be noted that when the device similarity is calculated by the formula, the first standby device set and the second standby device set are determined, each parameter has a clear meaning and function. The weight coefficients α, β and γ represent the importance of hardware configuration, software environment and load state in the evaluation of device similarity. For example, when selecting standby devices for high-performance computing devices, since the hardware configuration has a great impact on computing power, a higher weight is given to highlight the importance of hardware configuration. The hardware configuration similarity function F hardware (A, B i ) quantifies the similarity of different devices at the hardware level by comparing the processor model, memory capacity, storage type and other hardware parameters of the devices; The software environment similarity function F software (A, B i ) focuses on the operating system and application software version installed on the device to ensure that the standby device and the device to be updated have high software compatibility; The load state similarity function F load (A, B i ) is calculated according to the current workload of the device (such as the number of tasks, resource occupancy), so that the standby device selected has similar load capacity to the original device, avoiding resource waste. Through such a calculation method, a device set highly matched with the update device in multiple dimensions can be selected from the standby device set, laying a foundation for more accurate selection in the future.

[0064] The third data set of the device is obtained based on the first standby device set and the second standby device set, and the first standby device and the second standby device are obtained.

[0065] The third data set of the device is obtained based on the first standby device set and the second standby device set, and the first standby device and the second standby device are obtained.

[0066]

[0067] The working continuity of each standby device in the standby device set is obtained, wherein Cont(B i ) is the working continuity coefficient of the standby device B i , w N and w​T is a weight coefficient, is the historical interruption times of the backup device B i , is the average recovery time of the backup device B i .

[0068] It is to be noted that the historical interruption times intuitively reflect the stability of the device in the past operation process, and the fewer the interruption times, the more reliable the device; the average recovery time reflects the efficiency of the device in returning to normal work after failure, and the shorter the recovery time, the smaller the impact on the overall work flow. By weighted sum of the reciprocals of the two parameters, the historical interruption times and the average recovery time are converted into positive indicators, that is, the larger the value, the better the work continuity. For example, when evaluating the backup device of the key production device, if w N is large, it means that more attention is paid to the stability of the device, and the device with fewer historical interruption times is preferred; if w T is large, it is more inclined to the device with short average recovery time to reduce the impact of failure on production.

[0069] Score(B i ) = w x Sim(A, B i ) + (1-w) x Cont(B i ).

[0070] The first backup device and the second backup device are obtained, wherein Score(B i ) is the comprehensive score of the backup device B i , w is the weight coefficient of similarity, 0≤w≤1, and the backup device with the highest comprehensive score is selected as the first backup device and the second device.

[0071] It is to be noted that in the formula for comprehensively determining the first backup device and the second backup device, the weight coefficient is used to balance the importance of device similarity and work continuity. The device similarity ensures that the backup device is similar to the original device in function and configuration and can seamlessly take over the work; the work continuity ensures that the backup device has high reliability in future operation. By adjusting the value of w, the selection focus can be flexibly adjusted according to actual needs. For example, when switching devices in a financial transaction system with extremely high real-time requirements, the value of w is increased, and the backup device with high similarity to the original device is preferred to ensure that the transaction process is not affected; in an industrial automation control system with higher stability requirements, it is appropriately reduced, and more attention is paid to the work continuity of the device. Finally, by calculating the comprehensive score, the device with the highest score is selected as the first backup device and the second backup device, realizing the optimal selection of the backup device.

[0072] Specifically, in the first standby device and the second standby device, the first update device is selected preferentially, and after the selection of the first standby device is completed, the selection of the second standby device is performed again with the device data of the second update device.

[0073] S4: performing device switching based on the first update device, the second update device, and the first standby device and the second standby device.

[0074] In the device switching process, a breakpoint resuming mechanism and a data verification mechanism are used to monitor the integrity of data transmission and the continuity of task execution in real time.

[0075] After the device switching is completed, the performance of the first standby device and the second standby device is monitored, and if a performance anomaly occurs, an emergency rollback mechanism is started to switch the task to the first update device or to reselect a standby device.

[0076] It should be noted that, in the device switching process, the breakpoint resuming mechanism records the current execution progress, data state and other key information of the task at the moment of device switching, and the task is continued from the breakpoint after the new standby device is started, avoiding the time waste and resource consumption caused by starting the task from the beginning. The data verification mechanism verifies the data in the transmission process in real time by using hash verification, CRC verification and other technical means, and immediately triggers the retransmission mechanism once data errors or losses are found, to ensure the integrity and accuracy of the data. After the device switching is completed, the performance of the first standby device and the second standby device is monitored to discover potential problems in time. The emergency rollback mechanism is the last line of defense, and when a device performance anomaly is monitored and affects the normal execution of the task, the task is switched back to the original first update device or a suitable device is reselected from the standby device set, to minimize the negative impact of device switching on business and ensure the stable operation of the entire system.

[0077] Embodiment 2: A cross-platform data processing system, as shown in Figure 2 , comprising:

[0078] A data collection module for obtaining a first data set, a second data set and a third data set of a device;

[0079] A device state analysis module for dividing a first device set, a second device set, a third device set, a first update device, a second update device and a standby device set, and determining a first standby device set, a second standby device set, a first standby device and a second standby device;

[0080] A device switching control module for performing a device switching operation and monitoring and managing the switching process and the switched device.

[0081] Although the present application has been described in detail with reference to the above embodiments, it is apparent to those skilled in the art that various changes or modifications can be made to the present application without departing from the principles and spirit of the application as defined in the claims. Therefore, the detailed description of the disclosed embodiments is merely intended to explain the present application, and is not intended to limit the scope of protection of the present application, which is defined by the content of the claims.

Claims

1. A cross-platform data processing method, characterized in that: The method comprises the following steps: S1: obtaining a first data set of the device, obtaining a first device set, a second device set and a third device set; S2: checking the update notification of the device, obtaining a first update device, a second update device and a standby device set; S3: obtaining a second data set of the device, based on the first update device, the second update device and the standby device set, obtaining a first standby device and a second standby device; S4: based on the first update device, the second update device and the first standby device, the second standby device, performing device switching.

2. The cross-boundary platform data processing method according to claim 1, characterized in that: In S1, the first data set of the device is obtained, and the first device set, the second device set and the third device set are obtained, which comprises: The first data set of the device comprises real-time running parameters, task execution status, state indication signals, task scheduling information and resource allocation signals; The first device set is a device set that is working; The second device set is a device set that is about to work; The third device set is a device set that has no work arrangement.

3. The cross-boundary platform data processing method according to claim 1, characterized in that: In S2, the update notification of the device is checked, and the first update device, the second update device and the standby device set are obtained, which comprises: Real-time check the update notification of the device, obtain the device that needs to be updated, based on the first device set, the second device set and the third device set, obtain the first update device, the second update device and the standby device set, the standby device set is a device that does not need to be updated and normally executes work.

4. The cross-boundary platform data processing method according to claim 1, characterized in that: In S3, the second data set of the device is obtained, based on the first update device, the second update device and the standby device set, the first standby device and the second standby device are obtained, which comprises: S31: obtaining a second data set of the device, based on the first update device, the second update device and the standby device set, obtaining a first standby device set and a second standby device set; S32: obtaining a third data set of the device, based on the first standby device set and the second standby device set, obtaining a first standby device and a second standby device; The second data set of the device comprises hardware configuration, software environment and load state; The third data set of the device comprises historical interruption times and average recovery time.

5. The cross-platform data processing method according to claim 4, wherein: The second data set of the device is obtained based on the first update device, the second update device and the standby device set, and the first standby device set and the second standby device set are obtained, which comprises: The second data set of the device is obtained based on the first update device, the second update device and the standby device set, and the first standby device set and the second standby device set are obtained by the formula: Yes(A,B i )=α×F hardware (A,B i )+β×F software (A,B i )+γ×F load (A, Bi); a first set of backup devices and a second set of backup devices, wherein Sim(A, B i ) is a similarity function of the first update device, the second update device A and the backup device B i , α+β+γ=1, and α, β, γ∈[0, 1] are weight coefficients of the hardware configuration, the software environment and the load state respectively, F hardware (A, B i ) represents a hardware configuration similarity function of the first update device, the second update device A and the backup device B i , F software (A, B i ) represents a software environment similarity function, and F load (A, B i ) represents a load state similarity function.

6. The cross-platform data method of claim 4, wherein: The third data set of the device is obtained based on the first standby device set and the second standby device set, and the first standby device and the second standby device are obtained by the formula: In S4, the device switching is performed based on the first update device, the second update device and the first standby device, the second standby device, which comprises: the working continuity of each spare device in the set of spare devices, wherein Cont(B i ) is the working continuity coefficient of the spare device B i , w N and w T are weight coefficients, is the number of historical interruptions of the spare device B i , and is the average recovery time of the spare device B i . Score(B i ) = w x Sim(A, B i ) + (1 - w) x Cont(B i ); a first backup device and a second backup device are obtained, wherein Score(B i ) is a comprehensive score of the backup device B i , w is a weight coefficient of the similarity, 0≤w≤1, and the backup device with the highest comprehensive score is selected as the first backup device and the second device.

7. The cross-boundary platform data processing method according to claim 1, characterized in that: In the device switching process, breakpoint continuation and data checking mechanism are adopted to monitor the integrity of data transmission and the continuity of task execution in real time; ​ After the device switching is completed, performance monitoring is performed on the first backup device and the second backup device, and if performance abnormalities occur, an emergency rollback mechanism is started to switch the task to the first updated device or to reselect a backup device.

8. A cross-platform data processing system according to any of claims 1 to 7, characterized in that: A cross-platform data processing system comprises: A data acquisition module is configured to acquire a first data set, a second data set, and a third data set of a device; A device state analysis module is configured to divide a first device set, a second device set, a third device set, a first updated device, a second updated device, and a backup device set, and to determine a first backup device set, a second backup device set, a first backup device, and a second backup device; A device switching control module is configured to perform a device switching operation, and to monitor and manage the switching process and the device after switching.

Images