Operation system based on big data
By designing a big data-based operation system, real-time monitoring and analysis of network conditions, and automatically and manually adjusting link allocation, the problem of unreasonable network allocation in the existing technology is solved, and efficient utilization of network resources and overall performance is improved.
Patent Information
- Application Number
- CN202411475494.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2022-07-15
- Publication Date
- 2025-03-25
AI Technical Summary
The existing network allocation system lacks data statistics and analysis models, resulting in unreasonable network allocation, especially during peak network usage.
Design an operation system based on big data, including link pool module, database module, network model establishment module and network dynamic adjustment module. Through the collaborative work of these modules, they can monitor and analyze network conditions in real time, and automatically and manually adjust link allocation to ensure the rational use of network resources.
Through big data analysis and dynamic adjustment, efficient utilization of network resources is achieved, congestion of network channels is avoided, and the overall performance and reliability of the network are improved.
Smart Images
Figure CN119697000A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of network link aggregation, and in particular to an operation system based on big data. Background Art
[0002] In our daily lives, we are gradually inseparable from the Internet. The Internet establishes connections between our client devices and servers. However, the network conditions are different in many time periods. Generally, the network channels are more crowded during peak hours. However, due to the incompleteness of the network scheduling system, the network distribution is not uniform. In the current distribution system, due to the lack of data statistics, there is no data analysis model, which ultimately makes the network distribution unreasonable. Therefore, it is necessary to design an operation system based on big data with reasonable network distribution. Summary of the invention
[0003] The purpose of the present invention is to provide an operation system based on big data to solve the problems raised in the above background technology.
[0004] In order to solve the above technical problems, the present invention provides the following technical solutions: an operation system based on big data, comprising a link pool module, a database module, a network model building module and a network dynamic adjustment module, wherein the link pool module is used to store link information, the database module is used to store the status of network channels, the network model building module is used to build and analyze models based on database data, the network dynamic adjustment module is used to operate the network allocation mode, the database module is data connected to the network model building module, and the network model building module is data connected to the network dynamic adjustment module.
[0005] According to the above technical solution, the link pool module includes a main link module and a backup link module, the main link module is used for network traffic, the backup link module stores links for aggregation with the main link at any time, and the main link module and the backup link module are electrically connected; The database module includes a time unit and a network unit. The time unit is used to store time attribute data. The network unit module is used to store network attribute data. The time unit and the network unit are data connected.
[0006] According to the above technical solution, the network model building module includes a model fitting module and a demand estimation module, the model fitting module is used to draw a time network relationship chart, the demand estimation module is used to analyze the network demand of each area, and the model fitting module and the demand estimation module are data connected; The network dynamic adjustment module includes an instruction self-allocation module and an instruction input module. The instruction self-allocation module is used to automatically make adjustments according to the network model analysis results, and the instruction input module is used by the administrator to manually make network operation adjustments.
[0007] According to the above technical solution, the operation method of the operation system based on big data mainly includes the following steps: Step S1: Check the number of primary and backup links in the link pool, and evenly distribute the primary link between the server and each switching and routing device; Step S2: After allocation, link aggregation is performed on each point-to-point link, and the network status of each link is recorded; Step S3: At this time, a corresponding model is established according to the number of each link, and data is taken from the database and filled in; Step S4: Analyze the network model and estimate its network demand; Step S5: Link aggregation is automatically adjusted according to network requirements, and users can manually perform link access control according to requirements.
[0008] According to the above technical solution, step S2 further includes the following steps: Step S21: monitor the network status of the corresponding aggregated link in real time; Step S22: Record the network usage at intervals Q. When the amount of data used exceeds a threshold value W, reduce the value of Q. Step S23: converting the link number, time information and network data from decimal numbers into four-bit binary numbers; Step S24: According to the above sequence, it is assembled into a 12-bit binary number code and stored in the database.
[0009] According to the above technical solution, step S3 further includes the following steps: Step S31: When the amount of data in the database exceeds the threshold E, start building the model; Step S32: Read the data from the bottom of the database upwards, first identifying the first four bits of the data code; Step S33: If the data model corresponding to the first four bits of the code exists, continue the reading operation; if the corresponding model does not exist, create a data model; Step S34: Next, read the last eight bits, determine the time horizontal axis point in the model according to the middle four bits, and mark the network data value in the model according to the last eight bits.
[0010] According to the above technical solution, S4 further includes the following steps: Step S41: Connect the corresponding points into a straight line to form a line graph; Step S42: If the line segment slope is a positive number and the initial network flow is 0, then the midpoint of the line is selected as the marking point; if the initial flow is not 0, then the initial point of the line segment is selected as the marking point; if the line segment slope is a negative number and the terminal network flow is 0, then the midpoint of the line is selected as the marking point; if the terminal flow is not 0, then the terminal point of the line segment is selected as the marking point, and these points are connected; Step S43: connect all the time recording points except the time point 0, and form a region T approximating a rectangular parallelepiped with the above connection line; Step S44: starting from the end point, calculate the area T for the previous Y time nodes, and express its area as the network demand for the time period.
[0011] According to the above technical solution, step S5 further includes the following steps: Step S51: dynamically adjusting the links according to the demand ratio of each link of the server; Step S52: firstly, only the links in the main link pool are considered for redistribution. If there are insufficient links, links are obtained from the backup address pool. Step S53: the original link automatically runs the de-aggregation command operation first, and then performs the aggregation command operation; Step S54: The administrator can manually allocate according to actual needs.
[0012] According to the above technical solution, the step S54 further includes the following steps: Step S541: the system will allocate links according to the automatic operation of the system; Step S542: The administrator can manually modify the link aggregation state to change the state of a sub-link in the aggregated link to down; Step S543: if the down link fails to wake up after I time, it is directly put into the standby link pool; Step S544: At the same time, the sub-links in the normal state but belonging to the backup link will be automatically added to the main link pool.
[0013] According to the above technical solution, step S21 further includes the following steps: Step S211: There is a backup link in the aggregated link; Step S212: The backup link is only responsible for a small amount of data transmission. When the primary link fails, it automatically switches from the backup role to the primary role.
[0014] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: the present invention is mainly used for link operation, links are allocated according to demand, the link function is maximized as much as possible, the equipment has a main and standby link pool, the links in the pool can be converted to each other, and the system flexibility is improved. At the same time, each aggregated link is set with a main sub-link and a backup sub-link. When the main sub-link fails, the gap can be filled in time. In terms of data storage, a 12-bit binary number is used, and the segment matching principle can be used quickly to obtain query data. Storage only needs to establish one table entry, and there is no need to continuously establish three table entries of number, time and data. BRIEF DESCRIPTION OF THE DRAWINGS
[0015] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1 It is a schematic diagram of the system module composition of the present invention. DETAILED DESCRIPTION
[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0017] See also Figure 1 The present invention provides a technical solution: an operation system based on big data, including a link pool module, a database module, a network model building module and a network dynamic adjustment module, wherein the link pool module is used to store link information, the database module is used to store the status of network channels, the network model building module is used to build models and analyze according to database data, the network dynamic adjustment module is used to operate the network allocation mode, the database module and the network model building module are data connected, and the network model building module and the network dynamic adjustment module are data connected. The system is mainly used for link operation, allocating links according to demand, maximizing the role of links as much as possible, and the equipment main and standby link pool. The links in the pool can be converted to each other to improve the flexibility of the system. At the same time, each aggregated link is set with a main sub-link and a backup sub-link. When the main sub-link fails, the gap can be filled in time. In terms of data storage, a 12-bit binary number is used, and the segment matching principle can be quickly used to obtain query data. Storage only needs to establish one table item, and there is no need to continuously establish three table items of number, time and data.
[0018] The link pool module includes a main link module and a backup link module, wherein the main link module is used for network traffic, and the backup link module stores links for aggregation with the main link at any time, and the main link module and the backup link module are electrically connected; The database module includes a time unit and a network unit. The time unit is used to store time attribute data. The network unit module is used to store network attribute data. The time unit and the network unit are data connected.
[0019] The network model building module includes a model fitting module and a demand estimation module, wherein the model fitting module is used to draw a time network relationship chart, and the demand estimation module is used to analyze the network demand of each region, and the model fitting module and the demand estimation module are data connected; The network dynamic adjustment module includes an instruction self-allocation module and an instruction input module. The instruction self-allocation module is used to automatically make adjustments according to the network model analysis results, and the instruction input module is used by the administrator to manually make network operation adjustments.
[0020] The operation method of the operation system based on big data mainly includes the following steps: Step S1: Check the number of primary and backup links in the link pool, and evenly distribute the primary links between the server and each switching and routing device. First, the links are equally divided from the primary link pool. This is the initial state. Later, various data influences are considered and link adjustment is performed; Step S2: After the allocation, each point-to-point link is aggregated, and the network status of each link is recorded to prepare for the establishment of a network model. The aggregated links are numbered for easy recording in the database; Step S3: At this time, a corresponding model is established according to the number of each link, and data is taken from the database and filled in; Step S4: Analyze the network model and estimate its network demand. Demand is an important reason for changing the link operation strategy, and make reasonable allocation; Step S5: Link aggregation is automatically adjusted according to network requirements. At the same time, users can manually perform link access control according to requirements. In real life, cost issues need to be considered, so manual operations are required to expand and shrink.
[0021] Step S2 further comprises the following steps: Step S21: monitor the network status of the corresponding aggregated link in real time; Step S22: Record the network usage at intervals Q. When the data usage exceeds the threshold W, reduce the value of Q. When the network operation is relatively stable, the recording time maintains a certain regularity. However, when the data usage exceeds a certain value, shorten the recording interval to improve the recording accuracy of the data segment. Step S23: converting the link number, time information and network data from decimal numbers into four-bit binary numbers. This conversion is to combine the three tables into one when storing data to reduce database occupancy; Step S24: According to the above sequence, it is assembled into a 12-bit binary number code and stored in the database. The data segment can use the sequential matching principle to facilitate data search.
[0022] Step S3 further comprises the following steps: Step S31: When the amount of data in the database exceeds the threshold E, start building the model; Step S32: Read data from the bottom of the database upwards in sequence, first identify the first four bits of the data code, in the database, the first encapsulated data is stored at the bottom of the database because of its earlier time, and the first four bits indicate which aggregation link it comes from; Step S33: If the data model corresponding to the first four bits of the code exists, the reading operation continues; if the corresponding model does not exist, a data model is created. The existence of the model indicates that the link has been running for a period of time. Next, the time code can be matched; Step S34: Next, read the last eight bits, determine the time horizontal axis point in the model based on the middle four bits, mark the network data value in the model based on the last eight bits, and the middle four bits represent the time. The data at the bottom of the database is at an earlier time, so there is no disorder in the order. Reading the network code can obtain the network data volume corresponding to the time node.
[0023] S4 further comprises the following steps: Step S41: Connect the corresponding points into a straight line to form a line graph, first connect the time nodes with the numbers corresponding to the network data volume to form a continuous graph; Step S42: If the line segment slope is a positive number and the initial network flow is 0, then the midpoint of the line is selected as the marking point; if the initial flow is not 0, then the initial point of the line segment is selected as the marking point; if the line segment slope is a negative number and the terminal network flow is 0, then the midpoint of the line is selected as the marking point; if the terminal flow is not 0, then the terminal point of the line segment is selected as the marking point. Connect these points. The above principle can estimate the approximate amount of data in the line graph; Step S43: connect all time record points except 0 to form a rectangular area T with the above connection line, use the area per unit time to show the network status, and adjust the link according to the area ratio by comparing the area sizes in the same time period; Step S44: Taking the end point as the starting point, calculate the area T for the previous Y time nodes, and express its area as the network demand for the time period. When link adjustment is required, select a regional end, otherwise the data volume is too large to form a more obvious comparison.
[0024] Step S5 further comprises the following steps: Step S51: dynamically adjusting the links according to the demand ratio of each link of the server; Step S52: First, only the link reallocation in the main link pool is considered. If there are insufficient links, they are obtained from the backup address pool. The backup link pool is mainly used to solve the problem of insufficient main link pool, so its link reserve is set according to 80% of the network saturation. If a fully saturated network state is required later, more links can be added; Step S53: the original link automatically runs the de-aggregation command operation first, and when the aggregation command operation is performed and re-aggregation is performed, the original aggregation needs to be de-aggregated and the link added therein; Step S54: The administrator can manually allocate according to actual needs. However, in real life, since each user selects a different network package, their network channels will be subject to fundamental restrictions. After automatic allocation by the system, manual command restrictions are performed.
[0025] Step S54 further includes the following steps: Step S541: the system will allocate links according to the automatic operation of the system; Step S542: The administrator can manually modify the link aggregation status, turning the status of a sub-link in the aggregated link to down, and adjust the number of sub-links according to the network package selected by the user, regardless of whether the network requirements are met or missing, which is determined by the hard conditions; Step S543: if the down link fails to wake up after I time, it is directly placed in the standby link pool. If it is still not turned on after I time, it means that the user does not choose to widen the network bandwidth, so the link is directly placed in the standby pool, eliminating the tedious step of directly reallocating it after entering the main link pool. Step S544: At the same time, sub-links that are in normal state but belong to the backup link will be automatically added to the main link pool. Since some links are transferred from the main link pool to the backup link pool at this time, they need to be supplemented. The primary target of supplementation is the backup address pool links that are already in use, so as to avoid the result that the main pool shrinks and the backup pool expands after all links are released.
[0026] Step S21 further includes the following steps: Step S211: There is a backup link in the aggregated link. The backup link is mainly used to deal with link failures. Since network communication is unstable and extremely important, we need to do our best to ensure smooth network operation. Step S212: The backup link is only responsible for a small amount of data transmission. When the main link fails, it automatically switches from the backup role to the main role, mainly serving as a backup when the main sub-link in the aggregated link fails. Therefore, when the main link is normal, the link has less communication volume and is enabled at critical moments.
[0027] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.
[0028] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. An operation system based on big data, comprising a link pool module, a database module, a network model building module and a network dynamic adjustment module, characterized in that: The link pool module is used to store link information, the database module is used to store network channel conditions, the network model building module is used to build and analyze models based on database data, the network dynamic adjustment module is used to operate the network allocation mode, the database module is data connected to the network model building module, and the network model building module is data connected to the network dynamic adjustment module; The operation method of the operation system based on big data mainly includes the following steps: Step S1: Check the number of primary and backup links in the link pool, and evenly distribute the primary link between the server and each switching and routing device; Step S2: After allocation, link aggregation is performed on each point-to-point link, and the network status of each link is recorded; Step S3: At this time, a corresponding model is established according to the number of each link, and data is taken from the database and filled in; Step S4: Analyze the network model and estimate its network demand; Step S5: automatically adjust link aggregation according to network requirements, and users can manually perform link access control according to requirements; The step S2 further comprises the following steps: Step S21: monitor the network status of the corresponding aggregated link in real time; Step S22: Record the network usage at intervals Q. When the amount of data used exceeds a threshold value W, reduce the value of Q. Step S23: converting the link number, time information and network data from decimal numbers into four-bit binary numbers; Step S24: according to the above sequence, it is assembled into a 12-bit binary number code and stored in the database; The step S3 further comprises the following steps: Step S31: When the amount of data in the database exceeds the threshold E, start building the model; Step S32: Read the data from the bottom of the database upwards, first identifying the first four bits of the data code; Step S33: If the data model corresponding to the first four bits of the code exists, continue the reading operation; if the corresponding model does not exist, create a data model; Step S34: Next, read the last eight bits, determine the time horizontal axis point in the model according to the middle four bits, and mark the network data value in the model according to the last eight bits; The step S4 further comprises the following steps: Step S41: Connect the corresponding points into a straight line to form a line graph; Step S42: If the line segment slope is a positive number and the initial network flow is 0, then the midpoint of the line is selected as the marking point; if the initial flow is not 0, then the initial point of the line segment is selected as the marking point; if the line segment slope is a negative number and the terminal network flow is 0, then the midpoint of the line is selected as the marking point; if the terminal flow is not 0, then the terminal point of the line segment is selected as the marking point, and these points are connected; Step S43: connect all the time recording points except the time point 0, and form a region T approximating a rectangular parallelepiped with the above connection line; Step S44: starting from the end point, calculate the area T for the previous Y time nodes, and express its area as the network demand for the time period.
2. The big data-based operation system according to claim 1, characterized in that: The step S5 further comprises the following steps: Step S51: dynamically adjusting the links according to the demand ratio of each link of the server; Step S52: firstly, only the links in the main link pool are considered for redistribution. If there are insufficient links, links are obtained from the backup address pool. Step S53: the original link automatically runs the de-aggregation command operation first, and then performs the aggregation command operation; Step S54: The administrator can manually allocate according to actual needs.
3. The big data-based operation system according to claim 2, characterized in that: The step S54 further comprises the following steps: Step S541: the system will allocate links according to the automatic operation of the system; Step S542: The administrator can manually modify the link aggregation state to change the state of a sub-link in the aggregated link to down; Step S543: if the down link fails to wake up after I time, it is directly put into the standby link pool; Step S544: At the same time, the sub-links in the normal state but belonging to the backup link will be automatically added to the main link pool.
4. The big data-based operation system according to claim 3, characterized in that: The step S21 further comprises the following steps: Step S211: There is a backup link in the aggregated link; Step S212: The backup link is only responsible for a small amount of data transmission. When the primary link fails, it automatically switches from the backup role to the primary role.