Data processing method and device, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention provides a data processing method and device, electronic equipment, a storage medium and a program product, and is at least applied to the field of cloud technology and the field of node planning, the method comprises the following steps: determining a plurality of data dimensions corresponding to a node planning task and a data acquisition channel corresponding to each data dimension; acquiring to-be-analyzed data under the data dimension through the data acquisition channel corresponding to each data dimension; performing data scoring processing on the to-be-analyzed data under each data dimension to obtain a data scoring result under the corresponding data dimension; performing score fusion processing on the data score results under the plurality of data dimensions to obtain a fused score result; and in response to the score display operation, displaying at least one of the data score result and the fusion score result. Through the method and the device, an algorithm can be designed for data under different data dimensions, and overall calculation and scoring are automatically performed, so that planning and layout of nodes are guided through a quantifiable means.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of the Internet, and relate to but are not limited to a data processing method, device, electronic device, storage medium and program product. Background Art

[0002] With the continuous development of cloud vendors / OTT (providing various application services to users through the Internet), it has become an inevitable trend to deploy data centers (DC) or network service provision points (POP) in various regions. Especially for cloud services, overseas nodes have the characteristics of large investment and long construction period, which requires advance planning and layout.

[0003] However, in addition to single businesses with clear destination requirements, how should cloud vendors / OTT vendors plan overseas nodes in advance? What are the specific factors and bases that need to be considered? In related technologies, there is no systematic scoring algorithm and rules for node planning and layout, nor is there a corresponding automated system. In related technologies, cloud vendors / OTT vendors rely more on manual experience and fragmented information for planning and layout of overseas nodes, which requires a lot of time for manual judgment, and there is no detailed data support, nor is there an automated system for periodic calculation and presentation.

[0004] Therefore, in response to the above-mentioned problems existing in related technologies, a node planning and scoring strategy is urgently needed to assist in solving the information consideration problem during node planning and layout. Summary of the invention

[0005] The embodiments of the present application provide a data processing method, device, electronic device, storage medium and program product, which can be applied at least in the fields of cloud technology and node planning, design algorithms for data under different data dimensions, and automatically perform overall calculation and scoring, thereby guiding the planning and layout of nodes through quantifiable means.

[0006] The technical solution of the embodiment of the present application is implemented as follows:

[0007] An embodiment of the present application provides a data processing method, including: determining multiple data dimensions corresponding to a node planning task and data acquisition channels corresponding to each of the data dimensions; each of the data dimensions corresponds to at least one data acquisition channel; acquiring data to be analyzed under the data dimension through the data acquisition channel corresponding to each of the data dimensions; performing data scoring processing on the data to be analyzed under each of the data dimensions to obtain data scoring results under the corresponding data dimension; performing scoring fusion processing on the data scoring results under the multiple data dimensions to obtain fused scoring results; and displaying at least one of the data scoring results and the fused scoring results in response to a scoring display operation.

[0008] An embodiment of the present application provides a data processing device, including: a determination module, used to determine multiple data dimensions corresponding to a node planning task and a data acquisition channel corresponding to each of the data dimensions; each of the data dimensions corresponds to at least one data acquisition channel; an acquisition module, used to acquire the data to be analyzed under the data dimension through the data acquisition channel corresponding to each of the data dimensions; a data scoring module, used to perform data scoring processing on the data to be analyzed under each of the data dimensions to obtain a data scoring result under the corresponding data dimension; a scoring fusion module, used to perform scoring fusion processing on the data scoring results under the multiple data dimensions to obtain a fused scoring result; and a display module, used to display at least one of the data scoring result and the fused scoring result in response to a scoring display operation.

[0009] In some embodiments, the data acquisition channel includes a network traffic data channel; the data dimension includes a network traffic dimension; the acquisition module is also used to: obtain the traffic characteristic information of the interaction between the cloud platform and the network corresponding to the node planning task through the network traffic data channel on the network traffic analysis platform deployed at the network exit; aggregate the traffic characteristic information to obtain the data to be analyzed under the network traffic dimension.

[0010] In some embodiments, the acquisition module is also used to: configure a first scheduled task on the network traffic analysis platform, the first scheduled task corresponding to a preset first time period; call the first scheduled task regularly according to the first time period; when the first scheduled task is called for execution, trigger the following steps: obtain the traffic characteristic information of the interaction between the cloud platform and the network corresponding to the node planning task within the current first time period through the network traffic data channel.

[0011] In some embodiments, the device also includes: a data writing module, which is used to write the traffic characteristic information within the current first time period into a preset relational database after acquiring the traffic characteristic information within the current first time period; wherein the relational database includes multiple original data tables, and the traffic characteristic information within the current first time period is written into any one of the multiple original data tables.

[0012] In some embodiments, the relational database also includes a statistical data table; the data writing module is also used to: after obtaining the data to be analyzed under the network traffic dimension, construct a data identifier of the data to be analyzed based on the current first time period; determine a blank partition in the statistical data table where no information is written; and write the data identifier and the data to be analyzed into the blank partition.

[0013] In some embodiments, the acquisition module is also used to: determine the attribute parameters of each sub-information in the traffic characteristic information within the current first time period, the attribute parameters are used to characterize the geographical area corresponding to the sub-information and the autonomous system located in the geographical area; take the geographical area as the first aggregation dimension, aggregate the sub-information belonging to the same geographical area, and obtain a first data volume of sub-information corresponding to each geographical area; and take the geographical area and the autonomous system in the geographical area as the second aggregation dimension, aggregate the sub-information belonging to the same geographical area and the same autonomous system, and obtain a second data volume of sub-information under each autonomous system in each geographical area; determine the first data volume and the second data volume as the data to be analyzed under the network traffic dimension.

[0014] In some embodiments, the relational database also includes: a statistical data table corresponding to the first aggregation dimension and a statistical data table corresponding to the second aggregation dimension; the data writing module is also used to: after obtaining the first data amount of the sub-information corresponding to each geographical area, write the first data amount of the sub-information corresponding to each geographical area into the statistical data table corresponding to the first aggregation dimension; after obtaining the second data amount of the sub-information under each autonomous system in each geographical area, write the second data amount of the sub-information under each autonomous system in each geographical area into the statistical data table corresponding to the second aggregation dimension.

[0015] In some embodiments, the data acquisition channel includes a quality data detection channel; the data dimension includes a quality data dimension; the acquisition module is also used to: configure a second scheduled task on an external network quality detection platform, the second scheduled task corresponds to a preset second time period; according to the second time period, the second scheduled task is called regularly; when the second scheduled task is called for execution, the following steps are triggered: through the quality data detection channel deployed on the external network quality detection platform, real-time quality data under the quality data dimension is acquired; the real-time quality data is aggregated to obtain the data to be analyzed under the quality data dimension.

[0016] In some embodiments, the detection source area of ​​the real-time quality data includes: all geographical areas corresponding to the cloud platform corresponding to the node planning task; the detection target area of ​​the real-time quality data includes: sample Internet Protocol addresses in a specific autonomous system in the current existing geographical area; the acquisition module is also used for: when the real-time quality data under the quality data dimension is acquired based on the detection source area and the detection target area, aggregating multiple sub-data in the real-time quality data according to different quality data dimensions to obtain data to be analyzed under different quality data dimensions; the quality data dimensions include: delay dimension, packet loss rate dimension and unreachability rate dimension.

[0017] In some embodiments, the data acquisition channel includes a public publishing channel; the data dimension includes a public data dimension; the acquisition module is also used to: acquire public data under the public data dimension through the public publishing channel according to the publishing frequency of the public publishing channel; the public data constitutes the data to be analyzed.

[0018] In some embodiments, the data dimensions include a network traffic dimension, a quality data dimension, and a public data dimension; the data scoring module is also used to: for the data to be analyzed under the network traffic data dimension, determine the number of geographical areas corresponding to the data to be analyzed and the traffic ranking corresponding to each geographical area; based on the number of geographical areas and the traffic ranking, determine the traffic scoring result under the network traffic data dimension; for the data to be analyzed under the quality data dimension, determine the quality scoring result of the cloud platform corresponding to the node planning task and the quality scoring result between the cloud platform and other network platforms in turn based on the data to be analyzed; for the data to be analyzed under the public data dimension, determine the public data scoring result under the public data dimension based on the data to be analyzed and the number of geographical areas.

[0019] In some embodiments, the scoring fusion module is further used to: determine the weight of the data scoring result under each data dimension; and based on the weight, perform weighted summation of the data scoring results under the multiple data dimensions to obtain the fused scoring result.

[0020] An embodiment of the present application provides an electronic device, comprising: a memory for storing executable instructions; and a processor for implementing the above-mentioned data processing method when executing the executable instructions stored in the memory.

[0021] An embodiment of the present application provides a computer program product, which includes executable instructions, and the executable instructions are stored in a computer-readable storage medium; wherein a processor of an electronic device reads the executable instructions from the computer-readable storage medium, and when executing the executable instructions, the above-mentioned data processing method is implemented.

[0022] An embodiment of the present application provides a computer-readable storage medium storing executable instructions for causing a processor to execute the executable instructions to implement the above-mentioned data processing method.

[0023] The embodiments of the present application have the following beneficial effects:

[0024] Under multiple data dimensions, the data to be analyzed under each data dimension is obtained through the data acquisition channel corresponding to each data dimension; and the data to be analyzed under each data dimension is scored to obtain the data scoring results under the corresponding data dimension; at the same time, the data scoring results under multiple data dimensions are scored and fused to obtain the fused scoring results. In this way, corresponding scoring algorithms are designed for the data to be analyzed under different data dimensions, so as to automate the entire calculation and scoring process, and guide the planning and layout of nodes through quantifiable data scoring results and fused scoring results. The entire node planning task can be automatically carried out in the same system without manual experience and manual intervention, which can greatly improve the efficiency and flexibility of the node planning task; and, the data to be analyzed under multiple data dimensions is scored, so the final data scoring results and fused scoring results have a wide coverage, which can more accurately assist the execution of the node planning task. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is an optional architecture diagram of a data processing system provided in an embodiment of the present application;

[0026] Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0027] Figure 3 It is an optional flow chart of the data processing method provided in the embodiment of the present application;

[0028] Figure 4 is another optional flow chart of the data processing method provided in the embodiment of the present application;

[0029] Figure 5 This is a schematic diagram of the implementation process of obtaining traffic characteristic information provided by an embodiment of the present application;

[0030] Figure 6 It is a schematic diagram of the implementation process of information aggregation provided by the embodiment of the present application;

[0031] Figure 7 It is a structural diagram of the overseas node planning and scoring system provided in an embodiment of the present application;

[0032] Figure 8 This is a web interface diagram provided by an embodiment of the present application;

[0033] Fig. 9 It is a schematic diagram of data table partitioning provided in an embodiment of the present application;

[0034] Fig.10 This is a diagram showing the fields of the original data obtained according to an embodiment of the present application;

[0035] Fig.11 It is a field display diagram aggregated to the geographical area dimension provided in an embodiment of the present application;

[0036] Fig.12 It is a field display diagram aggregated to the geographical area + AS dimension provided in an embodiment of the present application;

[0037] Fig.13 is a data sample display diagram of quality data provided in the embodiments of the present application;

[0038] Fig.14 This is a data table field display diagram of the delay, packet loss rate and unreachable rate provided in the embodiment of the present application;

[0039] Fig.15 This is a data table field display diagram of global region data provided by an embodiment of the present application;

[0040] Fig.16 It is a calculation mapping relationship diagram between various data provided in the embodiment of the present application;

[0041] Fig.17 It is a diagram showing fields of a processed data table provided in an embodiment of the present application;

[0042] Fig.18 This is a diagram showing the fields of the final summarized data table provided in the embodiment of the present application;

[0043] Fig.19It is a system architecture diagram provided in an embodiment of the present application. DETAILED DESCRIPTION

[0044] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present application. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0045] In the following description, reference is made to "some embodiments", which describe a subset of all possible embodiments, but it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments, and may be combined with each other without conflict. Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meaning as those commonly understood by those skilled in the art of the technical field of the embodiments of the present application. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0046] In the related technologies, there are no systematic scoring algorithms and rules, nor corresponding automated systems for node planning and layout overseas or in different regions.

[0047] Based on the above-mentioned problems existing in the related technology, an embodiment of the present application provides a data processing method, which defines data dimensions for measuring node planning in various regions, and designs corresponding scoring algorithms for different data dimensions, so as to calculate the scoring results of each region based on the scoring algorithm, thereby assisting node planners to perform accurate and efficient node planning and layout based on the scoring results.

[0048] Among them, in the data processing method provided by the embodiment of the present application, first, multiple data dimensions corresponding to the node planning task and the data acquisition channels corresponding to each data dimension are determined; the data acquisition channel is used to obtain the data to be analyzed under the corresponding data dimension, and each data dimension corresponds to at least one data acquisition channel; then, through the data acquisition channel corresponding to each data dimension, the data to be analyzed under the data dimension is obtained; then, the data to be analyzed under each data dimension is scored and processed to obtain the data scoring result under the corresponding data dimension; and the data scoring results under multiple data dimensions are scored and fused to obtain the fused scoring result; finally, in response to the scoring display operation, at least one of the data scoring result and the fused scoring result is displayed. In this way, the entire calculation and scoring process is automated, and the planning and layout of the node is guided by quantifiable data scoring results and fused scoring results. The entire node planning task can be automatically carried out in the same system without manual experience and manual intervention, thereby greatly improving the efficiency and flexibility of the node planning task; and, the data to be analyzed under multiple data dimensions is scored and processed, so the final data scoring results and fused scoring results have a wide coverage, which can more accurately assist the execution of the node planning task.

[0049] Here, firstly, an exemplary application of the data processing device of the embodiment of the present application is described, and the data processing device is an electronic device for implementing the data processing method. In one implementation, the data processing device (i.e., electronic device) provided in the embodiment of the present application can be implemented as a terminal or a server. In one implementation, the data processing device provided in the embodiment of the present application can be implemented as a laptop, a tablet computer, a desktop computer, a mobile phone, a portable music player, a personal digital assistant, a dedicated messaging device, a portable game device, an intelligent robot, an intelligent home appliance, and an intelligent vehicle-mounted device, etc., any terminal with data processing, data display, and node planning functions; in another implementation, the data processing device provided in the embodiment of the present application can also be implemented as a server, wherein the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks (CDN, Content Delivery Network), and cloud servers for basic cloud computing services such as big data and artificial intelligence platforms. The terminal and the server can be directly or indirectly connected by wired or wireless communication, which is not limited in the embodiment of the present application. In the following, an exemplary application when the data processing device is implemented as a server will be described.

[0050] See also Figure 1 , Figure 1It is an optional architectural diagram of the data processing system provided in an embodiment of the present application. In order to support node planning tasks for any region, a node planning platform can be provided. The node planning platform can be implemented as a node planning application. The node planning application collects data to be analyzed under multiple data dimensions, and performs data scoring processing based on the collected data to be analyzed to obtain data scoring results and fusion scoring results for the region, so as to assist the execution of node planning tasks in the region.

[0051] In the embodiment of the present application, at least the node planning application is installed on the terminal. The data processing system 10 includes at least a terminal 100, a network 200 and a server 300, wherein the server 300 is a server of the node planning application. The server 300 can constitute the node planning device of the embodiment of the present application, that is, the node planning method of the embodiment of the present application is implemented by the server 300. The terminal 100 is connected to the server 300 via the network 200, and the network 200 can be a wide area network or a local area network, or a combination of the two.

[0052] When executing a node planning task, the terminal receives a node planning operation input by a node planner in a client of a node planning application, the node planning operation corresponds to the node planning task, and the terminal generates a node planning request based on the node planning task.

[0053] Then, the terminal 100 sends the node planning request to the server 300 through the network 200. After receiving the node planning request, the server 300 responds to the node planning request and determines multiple data dimensions corresponding to the node planning task and data acquisition channels corresponding to each data dimension; the data acquisition channel is used to obtain the data to be analyzed under the corresponding data dimension, and each data dimension corresponds to at least one data acquisition channel; then, through the data acquisition channel corresponding to each data dimension, the data to be analyzed under the data dimension is obtained; then, the data to be analyzed under each data dimension is scored and processed to obtain the data scoring result under the corresponding data dimension; and the data scoring results under multiple data dimensions are scored and fused to obtain the fused scoring result. After obtaining the data scoring result and the fused scoring result, the terminal 100 can receive the scoring display operation input by the user, and the terminal 100 generates a scoring display request based on the scoring display operation, and sends the scoring display request to the server 300. In response to the scoring display request, the server 300 sends the data scoring result and the fused scoring result to the terminal 100, so as to realize the display of at least one of the data scoring result and the fused scoring result on the terminal 100.

[0054] In some embodiments, the terminal 100 may also execute the data processing method of the embodiment of the present application, that is, after the node planning application running on the terminal 100 receives the node planning operation input by the node planner through the client, it responds to the node planning operation and determines multiple data dimensions corresponding to the node planning task and the data acquisition channel corresponding to each data dimension; the data acquisition channel is used to obtain the data to be analyzed under the corresponding data dimension, and each data dimension corresponds to at least one data acquisition channel; then, the terminal 100 obtains the data to be analyzed under the data dimension through the data acquisition channel corresponding to each data dimension; then, the terminal 100 performs data scoring processing on the data to be analyzed under each data dimension to obtain the data scoring result under the corresponding data dimension; and, the data scoring results under multiple data dimensions are scored and fused to obtain a fused scoring result; finally, the terminal 100 responds to the scoring display operation and displays at least one of the data scoring result and the fused scoring result on the current interface.

[0055] The data processing method provided in the embodiment of the present application can also be implemented based on a cloud platform and through cloud technology. For example, the server 300 can be a cloud server. The cloud server determines the multiple data dimensions corresponding to the node planning task and the data acquisition channel corresponding to each data dimension, or obtains the data to be analyzed under each data dimension through the cloud server, or performs data scoring processing on the data to be analyzed under each data dimension through the cloud server to obtain the data scoring result under the corresponding data dimension, or performs scoring fusion processing on the data scoring results under multiple data dimensions through the cloud server to obtain a fusion scoring result, etc.

[0056] In some embodiments, a cloud storage may be provided, and the data to be analyzed under each data dimension may be stored in the cloud storage, and the data scoring results and the fusion scoring results may also be stored in the cloud storage. In this way, when a scoring display operation is received, the data scoring results and the fusion scoring results may be directly obtained from the cloud storage for rapid scoring result display.

[0057] It should be noted here that cloud technology refers to a hosting technology that unifies hardware, software, network and other resources in a wide area network or local area network to achieve data computing, storage, processing and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool that is used on demand and is flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, in the future, each item may have its own identification mark, and all need to be transmitted to the background system for logical processing. Data of different levels will be processed separately. All kinds of industry data require strong system backing support, which can be achieved through cloud computing.

[0058] Figure 2 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, Figure 2 The electronic device shown may be a data processing device, which includes: at least one processor 310, a memory 350, at least one network interface 320 and a user interface 330. The various components in the data processing device are coupled together via a bus system 340. It is understood that the bus system 340 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 340 also includes a power bus, a control bus and a status signal bus. However, for the sake of clarity, the bus system 340 is not shown in FIG. Figure 2 Various buses are labeled as bus system 340 .

[0059] The processor 310 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., where the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0060] The user interface 330 includes one or more output devices 331 that enable presentation of media content, and one or more input devices 332 .

[0061] The memory 350 may be removable, non-removable or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drive, optical disk drive, etc. The memory 350 may optionally include one or more storage devices physically away from the processor 310. The memory 350 includes a volatile memory or a non-volatile memory, and may also include both volatile and non-volatile memory. The non-volatile memory may be a read-only memory (ROM), and the volatile memory may be a random access memory (RAM). The memory 350 described in the embodiment of the present application is intended to include any suitable type of memory. In some embodiments, the memory 350 can store data to support various operations, and examples of these data include programs, modules, and data structures or subsets or supersets thereof, as exemplarily described below.

[0062] An operating system 351 includes system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, etc., for implementing various basic businesses and processing hardware-based tasks; a network communication module 352 is used to reach other computing devices via one or more (wired or wireless) network interfaces 320. Exemplary network interfaces 320 include: Bluetooth, wireless compatibility certification (WiFi), and Universal Serial Bus (USB), etc.; an input processing module 353 is used to detect one or more user inputs or interactions from one of the one or more input devices 332 and translate the detected inputs or interactions.

[0063] In some embodiments, the device provided in the embodiments of the present application can be implemented in software. Figure 2 A data processing device 354 stored in the memory 350 is shown. The data processing device 354 may be a data processing device in an electronic device, which may be software in the form of a program or a plug-in, and includes the following software modules: a determination module 3541, an acquisition module 3542, a data scoring module 3543, a scoring fusion module 3544, and a display module 3545. These modules are logical, and therefore may be arbitrarily combined or further split according to the functions implemented. The functions of each module will be described below.

[0064] In some embodiments, the device provided in the embodiments of the present application can be implemented in hardware. As an example, the device provided in the embodiments of the present application can be a processor in the form of a hardware decoding processor, which is programmed to execute the data processing method provided in the embodiments of the present application. For example, the processor in the form of a hardware decoding processor can adopt one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), or other electronic components.

[0065] The data processing methods provided in each embodiment of the present application can be executed by an electronic device, wherein the electronic device can be a server or a terminal, that is, the data processing methods in each embodiment of the present application can be executed by a server, can be executed by a terminal, or can be executed through interaction between a server and a terminal.

[0066] Figure 3 This is an optional flow chart of the data processing method provided in the embodiment of the present application. Figure 3 The steps shown are explained as Figure 3 As shown, the execution subject of the data processing method is a server as an example for description, and the method includes the following steps S101 to S105:

[0067] Step S101, determining multiple data dimensions corresponding to the node planning task and a data acquisition channel corresponding to each data dimension.

[0068] Here, the node planning task may be a task of performing node planning for one or more geographical areas, where the node may be a network node, and the network node may be a network node for implementing cloud services, for example, it may include any network node such as a data center (DC), a network service provision point (POP), etc. The node planning task may include at least one of the following tasks: determining whether the geographical area can deploy network nodes, the specific location, quantity, attribute parameters and type of the deployed network nodes, etc.

[0069] The embodiments of the present application can be applied to the following scenarios: With the continuous development of the business of cloud vendors or OTT vendors, it is necessary to deploy network nodes in different geographical areas to achieve effective deployment of cloud services. At this time, it is necessary to plan network nodes for this new geographical area (which can be a country or a city). When planning, it is necessary to evaluate the geographical area in combination with some actual data of this geographical area, so as to obtain a quantifiable score that can accurately evaluate the geographical area, and then based on the score, the node planner can be guided to accurately plan the nodes for the geographical area, so as to achieve actual node construction for the geographical area. In the process of implementation, the data processing method provided by the embodiment of the present application can be used to calculate the data scoring results and fusion scoring results of the geographical area under multiple data dimensions, so as to assist the node planner to accurately plan the nodes for the geographical area based on these data scoring results and fusion scoring results.

[0070] In some embodiments, a node planning task may correspond to multiple data dimensions. Different data dimensions may acquire different data. The data in multiple data dimensions may fully reflect the actual conditions of each geographical area.

[0071] The data acquisition channel is used to acquire the data to be analyzed under the corresponding data dimension. Each data dimension corresponds to at least one data acquisition channel. The data acquisition channel can be a data acquisition channel related to cloud business or an official channel unrelated to cloud business.

[0072] In some embodiments, the data acquisition channel includes: a network flow data channel, a quality data detection channel and a public release channel. Accordingly, the data dimension includes a network flow dimension, a quality data dimension and a public data dimension.

[0073] A network traffic data channel refers to a channel used to obtain data on network process dimensions. For example, a network traffic data channel may be a network channel between a cloud platform and the Internet within a specific geographical area. All traffic data between the cloud platform and the Internet within the specific geographical area may be obtained through the network channel, thereby obtaining corresponding data to be analyzed.

[0074] The quality data detection channel refers to a channel used to obtain data of the quality data dimension. For example, the quality data detection channel can be a network channel between the cloud platform and the networks of other cloud vendors. The quality data between the cloud platform and the networks of other cloud vendors can be obtained through this network channel to obtain the corresponding data to be analyzed.

[0075] The public release channel refers to a channel used to obtain public dimension data. For example, the public release channel may be a data release platform for official data, and official data may be obtained from the data release platform to obtain corresponding data to be analyzed.

[0076] Here, we need to explain the following concepts and the differences between them:

[0077] Cloud platform: refers to services based on hardware resources and software resources, providing computing, network and storage capabilities. It can be a business platform built by the current business party based on cloud technology. For example, it can be a cloud platform of a cloud vendor that deploys node planning tasks.

[0078] Cloud vendors: refers to vendors that can build cloud platforms and provide cloud services on the cloud platforms. The method of the embodiment of the present application can provide node planning assistance to each cloud vendor, that is, it can assist the cloud vendor in node planning from multiple dimensions based on the data obtained by the cloud vendor in the target geographical area.

[0079] Cloud business: refers to the business platform provided on demand by the cloud, including operation platform, communication platform and architecture platform.

[0080] Step S102: acquiring the data to be analyzed under each data dimension through the data acquisition channel corresponding to each data dimension.

[0081] In the embodiment of the present application, after determining the data acquisition channels corresponding to the multiple data dimensions, the data within a certain period of time can be acquired through these data acquisition channels to obtain the data to be analyzed under different data dimensions. The data to be analyzed can be the original data acquired through the data acquisition channel, or it can be the data obtained after performing certain data processing on the acquired original data.

[0082] For example, for the network traffic dimension, the data to be analyzed may be the data obtained by processing the original traffic characteristic information between the cloud platform of the cloud vendor (i.e., the cloud platform corresponding to the node planning task) and the network of the target geographical area, such as aggregating the original traffic characteristic information to obtain traffic statistics within a certain time period, and using the traffic statistics as the data to be analyzed. For the quality data dimension, the data to be analyzed may be the data obtained by processing the real-time quality data detected by the external network quality detection platform, such as aggregating the real-time quality data to obtain quality data under the quality data dimension within a certain time period, and the quality data is the data to be analyzed. For the public data dimension, the data to be analyzed may be the original data obtained through a public release channel.

[0083] In the embodiment of the present application, different processing methods can be used to process original data of different data dimensions.

[0084] Step S103, performing data scoring processing on the data to be analyzed under each data dimension to obtain the data scoring result under the corresponding data dimension.

[0085] Here, data scoring processing refers to using a preset scoring algorithm to perform scoring calculation on the obtained data to be analyzed, so as to obtain a data scoring result under the data dimension corresponding to the data to be analyzed, and the data scoring result can be a specific scoring value. For different data dimensions, the calculated data scoring results can be scoring values ​​with the same order of magnitude, for example, they can all be a scoring value between 0 and 100; or, for different data dimensions, the calculated data scoring results can be scoring values ​​with different orders of magnitude. For such scoring values, the scoring values ​​under multiple data dimensions can be normalized so that the scoring values ​​under multiple data dimensions can be at the same order of magnitude, so as to facilitate subsequent calculations.

[0086] Different scoring algorithms are used to calculate the data to be analyzed in different data dimensions. The scoring algorithms for each data dimension will be illustrated below with examples.

[0087] Step S104, performing scoring fusion processing on the data scoring results under multiple data dimensions to obtain a fusion scoring result.

[0088] Here, the scoring fusion processing can be to perform a weighted summation on the data scoring results under multiple data dimensions to obtain a total scoring result, that is, a fused scoring result, and use the total scoring result to guide node layout. It can comprehensively consider the data of multiple data dimensions and conduct a comprehensive analysis and consideration of the node layout task.

[0089] Step S105 , in response to the scoring display operation, display at least one of the data scoring result and the fusion scoring result.

[0090] In the embodiment of the present application, the node planner can input a scoring display operation through the terminal, and the scoring display operation is used to request that at least one of the data scoring result and the fusion scoring result of the node planning task be displayed on the client of the node planning application. In other words, according to the scoring display operation of the node planner, it is possible to display only the data scoring result of one or more data dimensions, or only the fusion scoring result, or all the data scoring results and the fusion scoring results.

[0091] It should be noted that in the embodiments of the present application, the data to be analyzed under different data dimensions are all obtained in a public network environment. For example, the characteristic information of the traffic is extracted for statistics, including source / destination IP, protocol, traffic size, etc. The relevant data of the upper-layer business will not be detected and analyzed. It can be through the traffic analysis platform deployed at the cloud platform export of the cloud vendor to extract features of the data of the data center and the network interaction, and the upper-layer protocol will not be parsed. At the same time, no data related to user information or enterprise information will be saved, and only traffic statistics will be made. Therefore, the acquisition of these data to be analyzed is within the scope of authorization of laws and regulations and the subject of personal information, and the subsequent data use and processing behavior is also within the scope of authorization of laws and regulations and the subject of personal information.

[0092] The data processing method provided in the embodiment of the present application, in multiple data dimensions, obtains the data to be analyzed under the data dimension through the data acquisition channel corresponding to each data dimension; and performs data scoring processing on the data to be analyzed under each data dimension to obtain the data scoring result under the corresponding data dimension; at the same time, the data scoring results under multiple data dimensions are scored and fused to obtain the fused scoring result, so that the corresponding scoring algorithm is designed for the data to be analyzed under different data dimensions, so as to automate the entire calculation and scoring process, and guide the planning and layout of the nodes through quantifiable data scoring results and fused scoring results. The entire node planning task can be automatically carried out in the same system without manual experience and manual intervention, thereby greatly improving the efficiency and flexibility of the node planning task; and, the data to be analyzed under multiple data dimensions is scored, so the final data scoring results and fused scoring results have a wide coverage, which can assist the execution of the node planning task more accurately.

[0093] In some embodiments, the data processing system includes at least a terminal and a server, and the data processing method of the embodiment of the present application can be implemented through interaction between the terminal and the server.

[0094] The data processing method of the embodiment of the present application can be applied to the node planning scenario of any cloud vendor or OTT vendor. For example, the node planning scenario can be the node planning of any city in China, or the node planning of any geographical area overseas. The data processing method of the embodiment of the present application will be described below in conjunction with the data processing system.

[0095] Figure 4 is another optional flow chart of the data processing method provided in the embodiment of the present application, such as Figure 4 As shown, the method includes the following steps S201 to S222:

[0096] Step S201: The terminal receives the node planning operation input by the node planner in the client of the node planning application.

[0097] Here, the node planning operation corresponds to the node planning task. For example, the node planning task can be to perform node planning for a certain geographical area overseas. The geographical area here is the target geographical area. When generating the node planning task, the target geographical area can be input into the node planning application. After inputting the target geographical area, a confirmation button can also be provided on the client of the node planning application. The node planner can click the confirmation button to execute the subsequent data processing process of the embodiment of the present application. Among them, the input operation of the target geographical area and the click operation of the confirmation button constitute the node planning operation.

[0098] Step S202: The terminal generates a node planning request based on the node planning task.

[0099] Here, the terminal may obtain the area identifier of the target geographical area in response to the node planning operation of the node planner, and then encapsulate the area identifier into the node planning request.

[0100] Step S203: The terminal sends a node planning request to the server.

[0101] Step S204: In response to the node planning request, the server determines a plurality of data dimensions corresponding to the node planning task and a data acquisition channel corresponding to each data dimension.

[0102] Here, the data acquisition channel is used to acquire the data to be analyzed in the corresponding data dimension, and each data dimension corresponds to at least one data acquisition channel.

[0103] In the embodiment of the present application, the data acquisition channel includes: a network flow data channel, a quality data detection channel and a public release channel. Correspondingly, the data dimension includes a network flow dimension, a quality data dimension and a public data dimension.

[0104] The three data dimensions are explained below.

[0105] For the data in the network traffic dimension, the following steps S205 to S206 may be performed to obtain the data scoring result in the network traffic dimension:

[0106] In step S205, the server obtains traffic characteristic information of the interaction between the cloud platform and the network corresponding to the node planning task through the network traffic data channel on the network traffic analysis platform deployed at the network exit.

[0107] Here, the network exit refers to the network exit of the cloud vendor of the node planning task. A network traffic analysis platform can be deployed at the network exit to analyze the traffic characteristic information at the network exit. The network traffic data channel can be understood as the network channel between the network traffic analysis platform and the external network, through which the traffic characteristic information of the interaction between the cloud platform and the network can be captured.

[0108] In some embodiments, see Figure 5 , Figure 5 It is shown that in step S205, the traffic characteristic information of the interaction between the cloud platform and the network corresponding to the node planning task is obtained, which can be achieved by the following steps S2051 to S2053:

[0109] Step S2051, configuring a first scheduled task on the network traffic analysis platform, where the first scheduled task corresponds to a preset first time period.

[0110] Step S2052: Call the first scheduled task regularly according to the first time period.

[0111] In the embodiment of the present application, the first scheduled task can be called periodically according to the first time period. For example, the first time period can be one day, so the first scheduled task can be called at a fixed time every day.

[0112] Step S2053, when the first scheduled task is called for execution, the following steps are triggered: obtaining the traffic characteristic information of the interaction between the cloud platform corresponding to the node planning task and the network in the current first time period through the network traffic data channel.

[0113] When the first scheduled task is called for execution, the traffic characteristic information of the interaction between the cloud platform and the external network in the current first time period can be captured through the network traffic data channel, wherein the traffic characteristic information can be the original data of the interaction between the cloud platform and the network, for example, it can include: the total number of traffic bytes per minute, the total number of traffic packets per minute, the cloud platform side IP, the cloud platform side port, the gold, silver and bronze labels, the export computer room network module ID, the traffic direction, the protocol number, the export ID, the user side AS number, the operator ID, the province ID, the country ID, the IPV4 or IPV6 traffic and other original data.

[0114] Step S206: The server aggregates the traffic characteristic information to obtain data to be analyzed under the network traffic dimension.

[0115] Here, information aggregation refers to summarizing the traffic feature information obtained in the form of original data, so as to obtain data to be analyzed that can participate in the scoring calculation, that is, can be input into the scoring algorithm.

[0116] In some embodiments, see Figure 6 , Figure 6 It is shown that in step S206, the information aggregation of the traffic characteristic information can be implemented by the following steps S2061 to S2064:

[0117] Step S2061, determining the attribute parameters of each sub-information in the traffic characteristic information in the current first time period, where the attribute parameters are used to characterize the geographical area corresponding to the sub-information and the autonomous system located in the geographical area.

[0118] In the embodiment of the present application, the traffic characteristic information is used as the captured original data, which includes multiple sub-information, and because the traffic characteristic information is the information that interacts between the cloud platform and the external network in the current first time period, the sub-information in the traffic characteristic information can be information that interacts with multiple geographical areas or multiple autonomous systems in the same geographical area, and each sub-information can carry the regional identifier of the geographical area corresponding to the sub-information and the system identifier of the autonomous system corresponding to the sub-information, and the attribute parameters include the regional identifier and the system identifier. In this way, based on the regional identifier and the system identifier, it can be determined that each sub-information is information about the network of which geographical area the cloud platform interacts with, and it can also be determined that each sub-information is information about the network of which geographical area the cloud platform interacts with.

[0119] Step S2062, taking the geographical area as the first aggregation dimension, aggregating the sub-information belonging to the same geographical area, and obtaining a first data volume of the sub-information corresponding to each geographical area.

[0120] Here, the sub-information belonging to the same geographical area is aggregated, and the number of sub-information belonging to the same geographical area is counted to obtain a first data volume, wherein the first data volume refers to the number of information items of sub-information belonging to the same geographical area. For example, for the captured original data, each sub-information in the original data can be displayed in the form of a data table, and then the rows of sub-information belonging to the same geographical area in the data table can be counted, and each row represents a piece of data, and then the first data volume can be obtained by counting.

[0121] Step S2063, taking the geographical area and the autonomous system in the geographical area as the second aggregation dimension, aggregate the sub-information belonging to the same geographical area and the same autonomous system to obtain a second data volume of the sub-information under each autonomous system in each geographical area.

[0122] Here, the sub-information belonging to the same geographical area and the same autonomous system is aggregated, and the sub-information belonging to the same geographical area and the same autonomous system within the geographical area is summarized, and the number of sub-information belonging to the same geographical area and the same autonomous system is counted to obtain a second data volume, wherein the second data volume refers to the number of pieces of information of sub-information belonging to the same geographical area and the same autonomous system.

[0123] Step S2064: determine the first data volume and the second data volume as data to be analyzed under the network traffic dimension.

[0124] In the embodiment of the present application, if the node planning task is to perform node planning for a target geographical area, a first amount of data for the target geographical area can be obtained, and the first amount of data can be determined as the data to be analyzed under the network flow dimension. If the node planning task is to perform node planning for a target autonomous system in the target geographical area, a second amount of data in the target autonomous system of the target geographical area can be obtained, and the second amount of data can be determined as the data to be analyzed under the network flow dimension.

[0125] Step S207, after acquiring the traffic characteristic information in the current first time period, the server writes the traffic characteristic information in the current first time period into a preset relational database.

[0126] Here, the relational database includes multiple raw data tables, and the traffic characteristic information in the current first time period is written into any one of the multiple raw data tables. That is to say, when the raw data is captured, each captured sub-information can be written into the raw data table. When writing the table, the sub-information belonging to the same time period can be written into the same raw data table, the sub-information belonging to the same geographical area can be written into the same raw data table, and the sub-information belonging to the same geographical area and the same autonomous system can be written into the same raw data table. In this way, when the sub-information is subsequently aggregated, the data statistics can be completed based on the same table, thereby greatly improving the efficiency of information aggregation.

[0127] In some embodiments, when aggregating sub-information, the corresponding original data table can be obtained from the relational database, and the sub-information that meets the requirements in the original data table can be aggregated to obtain a first data volume and a second data volume, and then the first data volume and the second data volume are determined as the data to be analyzed under the network traffic dimension.

[0128] In some embodiments, the relational database may also include a statistical data table. After obtaining the data to be analyzed under the network traffic dimension, the server may also construct a data identifier of the data to be analyzed based on the current first time period; and determine a blank partition in the statistical data table where no information is written; and then write the data identifier and the data to be analyzed into the blank partition.

[0129] In some embodiments, the relational database also includes a statistical data table corresponding to the first aggregation dimension and a statistical data table corresponding to the second aggregation dimension. After obtaining the first data volume of the sub-information corresponding to each geographic area, the first data volume of the sub-information corresponding to each geographic area can also be written into the statistical data table corresponding to the first aggregation dimension; after obtaining the second data volume of the sub-information under each autonomous system in each geographic area, the second data volume of the sub-information under each autonomous system in each geographic area can also be written into the statistical data table corresponding to the second aggregation dimension.

[0130] For the data in the quality data dimension, the data scoring result under the quality data dimension can be obtained through the following steps S208 to S211:

[0131] Step S208: The server configures a second scheduled task on the external network quality detection platform. The second scheduled task corresponds to a preset second time period.

[0132] Step S209: The server calls the second scheduled task regularly according to the second time period.

[0133] In the embodiment of the present application, the second scheduled task can be called periodically according to the second time period. For example, the second time period can be one day, so the second scheduled task can be called at a fixed time every day.

[0134] Step S210, when the second scheduled task is called for execution, the following steps are triggered to execute: real-time quality data under the quality data dimension is obtained through the quality data detection channel deployed on the external network quality detection platform.

[0135] When the second scheduled task is called for execution, real-time quality data under the quality data dimension can be obtained through the quality data detection channel deployed on the external network quality detection platform, wherein the real-time quality data can be the network quality data between the detection source area and the detection target area, wherein the detection source area of ​​the real-time quality data includes: all geographical areas corresponding to the cloud platform corresponding to the node planning task; the detection target area of ​​the real-time quality data includes: sample Internet Protocol addresses in specific autonomous systems in the current existing geographical areas. For example, the detection source area of ​​the real-time quality data can be all geographical areas (Regions) of the cloud vendor of the node planning task, as well as the Regions in the same region as the mainstream cloud vendors and cloud platforms. The detection target area is the sample IP in the mainstream AS in all geographical areas in the world.

[0136] Step S211: the server aggregates the real-time quality data to obtain data to be analyzed under the quality data dimension.

[0137] In some embodiments, when real-time quality data under the quality data dimension is obtained based on the detection source area and the detection target area, multiple sub-data in the real-time quality data can be summarized according to different quality data dimensions to obtain data to be analyzed under different quality data dimensions. The quality data dimensions include: delay dimension, packet loss rate dimension, and unreachable rate dimension.

[0138] In some embodiments, the relational database may also include an original data table corresponding to the quality data dimension, and the data to be analyzed under the quality data dimension may also be written into the original data table corresponding to the quality data dimension in the relational database. For example, the data table fields in the original data table may include: the city where the DC is located, the target AS, the target country, the target province, the cloud vendor a1 delay (unit ms), the cloud vendor a1 unreachability rate (unit %), the cloud vendor a1 packet loss rate (unit %), the cloud vendor a2 delay (unit ms), the cloud vendor a2 unreachability rate (unit %), the cloud vendor a2 packet loss rate (unit %), the cloud vendor a3 delay (unit ms), the cloud vendor a3 unreachability rate (unit %), the cloud vendor a3 packet loss rate (unit %), the cloud vendor a1 delay ranking, the cloud vendor a1 unreachability rate ranking, the cloud vendor a1 packet loss rate ranking, the total number of cloud vendors participating in the ranking, the creation date, and other information.

[0139] For the data in the public data dimension, the data scoring result under the public data dimension can be obtained by following the following step S212: In step S212, the server obtains the public data under the public data dimension through the public publishing channel according to the publishing frequency of the public publishing channel; the public data constitutes the data to be analyzed.

[0140] Here, different public release channels have different release frequencies. For example, the release frequency may be one week, one month, one quarter, or one year.

[0141] Step S213, the server determines the number of geographical areas corresponding to the data to be analyzed under the network traffic data dimension and the traffic ranking corresponding to each geographical area; based on the number of geographical areas and the traffic ranking, determines the traffic scoring result under the network traffic data dimension.

[0142] Here, a preset scoring algorithm under the network traffic data dimension can be used to determine the traffic scoring result under the network traffic data dimension. That is to say, the data to be analyzed can be analyzed to determine the number of geographical areas and the traffic ranking corresponding to each geographical area, and then the number of geographical areas and the traffic ranking can be input into the scoring formula corresponding to the scoring algorithm to obtain the traffic scoring result under the network traffic data dimension.

[0143] Step S214, the server determines the quality score results of the cloud platform corresponding to the node planning task and the quality score results between the cloud platform and other network platforms based on the data to be analyzed under the quality data dimension.

[0144] Here, the scoring algorithm under the preset quality data dimension can be used to determine the quality scoring results of the cloud platform corresponding to the node planning task and the quality scoring results between the cloud platform and other network platforms. In other words, the actual data in the data to be analyzed (for example, data such as latency / unreachability rate / packet loss rate) can be input into the scoring formula corresponding to the scoring algorithm to obtain the quality scoring results of the cloud platform corresponding to the node planning task and the quality scoring results between the cloud platform and other network platforms.

[0145] Step S215 , the server determines a public data scoring result under the public data dimension for the data to be analyzed under the public data dimension based on the number of the data to be analyzed and the geographical areas.

[0146] Here, a preset scoring algorithm under the public data dimension can be used to determine the public data scoring result under the public data dimension. In other words, the acquired data to be analyzed under the public data dimension can be directly input into the scoring formula corresponding to the scoring algorithm to calculate the public data scoring result under the public data dimension.

[0147] Step S216: The server determines the weight of the data scoring result under each data dimension.

[0148] In the embodiment of the present application, different node planning tasks have different emphases on different data dimensions. Therefore, the weights of data scoring results under different data dimensions can be determined based on the emphases of the node planning tasks.

[0149] Step S217: The server performs weighted summation on the data scoring results under multiple data dimensions based on the weights to obtain a fusion scoring result.

[0150] Step S218: The terminal receives a rating display operation.

[0151] Step S219: The terminal generates a rating display request in response to the rating display operation.

[0152] Step S220: The terminal sends a rating display request to the server.

[0153] Step S221: The server sends the data scoring result and the fusion scoring result to the terminal in response to the scoring display request.

[0154] Step S222: The terminal displays at least one of the data scoring result and the fusion scoring result on the current interface.

[0155] In the embodiment of the present application, a corresponding scoring algorithm is designed for the data to be analyzed under different data dimensions, so as to automate the entire calculation and scoring process, and guide the planning and layout of the nodes through quantifiable data scoring results and fusion scoring results. The entire node planning task can be automatically carried out in the same system without manual experience and manual intervention, thereby greatly improving the efficiency and flexibility of the node planning task; and, the data to be analyzed under multiple data dimensions is scored, so the final data scoring results and fusion scoring results have a wide coverage, which can more accurately assist the execution of the node planning task.

[0156] The following is an explanation of an exemplary application of the embodiments of the present application in a practical application scenario.

[0157] In order to solve the problem of overseas node planning of the cloud platform, the embodiment of this application has created an overseas node planning scoring system. Relying on the network detection and traffic analysis capabilities of the cloud platform itself, it gathers multi-dimensional data from multiple regions overseas. And combined with business characteristics, algorithms are designed for data of different dimensions to perform overall calculation and scoring. The above data aggregation and calculation and scoring are fully automated. Use quantifiable means to guide overseas node planning and layout.

[0158] The overseas node planning scoring system (i.e., the above-mentioned data processing system) provided in the embodiment of the present application mainly defines the data dimensions used to measure node planning in overseas regions, and designs corresponding scoring algorithms for different data dimensions. The system can at least solve the following problems: all data are obtained from official or actual measurements and statistics, covering multiple dimensions; it can design different algorithms and scoring scores for different dimensional data in combination with business characteristics, and can be flexibly customized; it is fully automated from data aggregation to calculation and scoring, and can be presented through a Web interface, and the data is updated periodically.

[0159] Figure 7 is a structural diagram of the overseas node planning and scoring system provided in the embodiment of the present application, such as Figure 7 As shown, the entire system includes three modules: data module 701, calculation module 702 and Web module 703.

[0160] The data module uses MySQL as the system database, and aggregates data of various dimensions into the database through TDW scheduled tasks, SCF scheduled calls, and public network data import. The calculation module designs algorithms for data of different dimensions, and calculates scores for the aggregated data every day through SCF scheduled calls. The score results are stored in the database summary table for the Web module to call. The Web module is the Web GUI, where users use the browser to query the score results of each region and each detailed data, such as Figure 8 The figure shows a web interface diagram provided in an embodiment of the present application.

[0161] The following is a description of the data production process of the overseas node planning and scoring system of an embodiment of the present application.

[0162] The relevant data dimensions and data sources of the overseas node planning scoring system are as follows:

[0163] (1) For traffic data in the network traffic dimension: All traffic feature information (including source and destination IP, inbound and outbound traffic rate) between the cloud platform and the Internet can be captured through the network traffic analysis platform deployed at the Internet exit of the network. After preliminary aggregation and processing, these data are stored in the big data platform in real time. The amount of this part of data is huge, and about 30 billion data can be generated every day. Fig. 9 As shown in the figure, the first column represents the data table partition, one partition per day, in the format of p_yyyymmdd. The second column represents the data volume. Fig.10 It is a display diagram of the original data fields obtained. Fig.10 By counting the original fields in Fig. 9 Data sheet shown.

[0164] Here, the unit of data volume is a row in the data table (or a piece of data). The data here is the traffic between the cloud platform data center and the Internet. The characteristic information of the traffic can be extracted for statistics, such as source / destination IP, protocol, traffic size, etc. For the upper-layer business, it will not perceive and parse, so the corresponding data can be obtained without authorization. Through the traffic analysis platform deployed at the cloud platform DC egress, the characteristics of the traffic between the DC and the Internet will be extracted. The upper-layer protocol will not be parsed, and no data will be saved. Only traffic statistics will be made.

[0165] In the embodiment of the present application, for the overseas node planning and scoring system, unlike network operations and services, there are not such high requirements for the real-time nature of the data, and it is sufficient to only count the data in a daily period. Therefore, by configuring the scheduled task, the detailed data of the previous day can be aggregated every morning and written to MysqlDB. The amount of data will be greatly reduced after aggregation according to system requirements, thereby improving the efficiency of subsequent data calculations. A total of two scheduled tasks are set here, which are aggregated into two data tables. Data Table 1 aggregates the traffic to the geographic area dimension, which is used to record the total traffic from the cloud platform to the destination geographical area. The daily data volume is 239 (number of geographical areas), and the fields aggregated to the geographic area dimension are as follows: Fig.11 shown.

[0166] Data Table 2 aggregates traffic to the geographic region + autonomous system (AS) dimension, which is used to record the traffic from the cloud platform to the destination geographic region + AS. The daily data volume is about 80,000. The fields aggregated to the geographic region + AS dimension are as follows: Fig.12 shown.

[0167] (2) For quality data in the quality data dimension: Continuously perform detection through the external network quality detection platform to obtain real-time quality data. The detection source includes all geographical regions of the cloud platform, as well as the regions of the mainstream cloud vendors and the cloud platform. The detection purpose is to sample IP addresses in the mainstream AS in all geographical regions around the world. These data are stored in the database and provide APIs for custom queries. Data samples are as follows: Fig.13 shown.

[0168] In the embodiment of the present application, through the scheduled task of Serverless Cloud Function (SCF), the summary quality detection data of the previous day is queried every morning (taking the average value of one day) and written into MySQL DB. The quality data is subdivided into the following three dimensions: latency, packet loss rate and unreachable rate (packet loss of more than 30% per minute is considered unreachable in this minute). The data table fields are as follows Fig.14 shown.

[0169] (3) For global regional data in the public data dimension: the data comes from official data releases. It mainly includes the following three dimensions: GDP, population, and Internet population. The update frequency of this part of data mainly depends on the update of official channel data, which is generally updated once a year. After downloading this part of data from the website, import it into MySQL DB. The data table fields are as follows: Fig.15 shown.

[0170] After obtaining the data of the above three dimensions, the data calculation process in the embodiment of the present application is described below.

[0171] Fig.16 : is a calculation mapping relationship diagram between various data provided in the embodiment of the present application, wherein the data processing logic is as follows:

[0172] (1) The flow data of the geographical area latitude does not need to be processed and is directly used as input to the scoring algorithm;

[0173] (2) According to the detection destination geographical area + AS, the geographical area + AS latitude traffic peak data is matched to each piece of quality detection data. The traffic peak is used as the weight to calculate the weighted average quality data from each source DC to each target geographical area, and the DC with the shortest delay covering the target geographical area is selected as the final quality data to this target geographical area as the input of the scoring algorithm. Among them, the processed data table fields are as follows Fig.17 shown.

[0174] (3) The GDP and population data of each geographic region in the world do not need to be processed and can be directly used as input into the scoring algorithm.

[0175] The scoring algorithm provided in the embodiment of the present application is described below.

[0176] (1) For the traffic score (i.e., the data scoring result of the network traffic dimension), the following formula (1) can be used for calculation:

[0177]

[0178] Where s1 represents the traffic score of a specific geographic area; r1 represents the traffic ranking of a specific geographic area; and n represents the total number of geographic areas.

[0179] (2) The latency score (i.e., a data scoring result in the quality data dimension) can be calculated using the following formula (2):

[0180]

[0181] Among them, s2 represents the latency score of a specific geographical area; z represents the optimal latency (in ms) of the cloud platform covering a specific geographical area.

[0182] (3) The quality scores (i.e., another data scoring result of the quality data dimension) such as latency, unreachability rate, and packet rate compared with other cloud vendors can be calculated using the following formula (3):

[0183]

[0184] Among them, s3 represents the quality score of a specific geographical area; lr i Indicates the number of autonomous systems (AS) covering a specific geographic area compared to other cloud vendors. i )’s latency, unreachability rate, and packet loss rate rankings; b i Indicates a specific geographic area AS i The traffic peak value; n represents the total number of ASs in a specific geographical area; p represents the number of cloud vendors participating in the ranking.

[0185] (4) Population / GDP score (i.e., the data scoring result of the public data dimension) can be calculated using the following formula (4):

[0186]

[0187] Among them, s4 represents the population / GDP score of a specific geographic area; r4 represents the population / GDP ranking of a specific geographic area; and n represents the total number of geographic areas.

[0188] After calculation, the score of each of the above items is a value between 0 and 1. The final summary data table fields are as follows Fig.18 shown.

[0189] After obtaining the data scoring results of each data dimension, the data scoring results can be systematically displayed. Fig.19 This is a system architecture diagram provided by the embodiment of the present application. Fig.19 As shown, the summary scores in the database can be presented using a standard Web GUI solution. To improve the usability and flexibility of the system, the following technical settings are made:

[0190] (1) The scores of each dimension are configurable and take effect immediately. This is applicable to different business parties who have different emphasis and preferences on each scoring item. The default score is 40 points for traffic, 20 points for population, 10 points for GDP, 10 points for latency, 10 points for latency comparison with competitors, 5 points for packet loss comparison with competitors, and 5 points for unreachability comparison with competitors, with a total score of 100 points.

[0191] (2) The web interface displays the current day's rating data by default. If the current day's data has not yet been generated, the most recent day's data will be displayed. You can select a specific date to view historical rating data.

[0192] (3) The system provides an application program interface (API), which can trigger score recalculation through API calls, and the date of the data source can be specified. This setting is mainly to deal with abnormalities in the upstream data source.

[0193] The overseas node planning and scoring system provided in the embodiment of the present application can bring at least the following benefits:

[0194] On the basis of ensuring that the data source is accurate and reliable, it provides multi-dimensional and measurable data support for overseas node planning and layout, avoiding the deviation caused by subjective judgment. It saves a lot of manual analysis and planning time and improves work efficiency. It provides a complete set of solutions and algorithms that different manufacturers / teams can flexibly customize according to their own business characteristics, which has universal guiding significance.

[0195] It should be noted that the network traffic analysis platform in the embodiment of the present application can also use Netflow sampling data instead, which can effectively save deployment costs; the data dimensions and algorithms can be flexibly adjusted according to the differences in business characteristics and focus. For example, the number of business users, traffic growth rate, number of Internet users, number of IP addresses, etc. in the target geographical area can be introduced.

[0196] It can be understood that in the embodiments of the present application, the content involving user information, such as information to be analyzed under various data dimensions, if it involves data related to user information or corporate information, when the embodiments of the present application are applied to specific products or technologies, it is necessary to obtain user permission or consent, or to blur this information to eliminate the correspondence between this information and the user; and the relevant data collection and processing should be strictly in accordance with the requirements of relevant national laws and regulations when applied in examples, obtain the informed consent or separate consent of the subject of personal information, and carry out subsequent data use and processing within the scope of authorization of laws and regulations and the subject of personal information.

[0197] The following is a description of an exemplary structure of the data processing device 354 provided in the embodiment of the present application implemented as a software module. In some embodiments, for example, Figure 2As shown, the data processing device 354 includes: a determination module 3541, which is used to determine multiple data dimensions corresponding to the node planning task and a data acquisition channel corresponding to each of the data dimensions; each of the data dimensions corresponds to at least one data acquisition channel; an acquisition module 3542, which is used to acquire the data to be analyzed under the data dimension through the data acquisition channel corresponding to each of the data dimensions; a data scoring module 3543, which is used to perform data scoring processing on the data to be analyzed under each of the data dimensions to obtain the data scoring results under the corresponding data dimension; a scoring fusion module 3544, which is used to perform scoring fusion processing on the data scoring results under the multiple data dimensions to obtain a fused scoring result; a display module 3545, which is used to display at least one of the data scoring result and the fused scoring result in response to a scoring display operation.

[0198] In some embodiments, the data acquisition channel includes a network traffic data channel; the data dimension includes a network traffic dimension; the acquisition module is also used to: obtain the traffic characteristic information of the interaction between the cloud platform and the network corresponding to the node planning task through the network traffic data channel on the network traffic analysis platform deployed at the network exit; aggregate the traffic characteristic information to obtain the data to be analyzed under the network traffic dimension.

[0199] In some embodiments, the acquisition module is also used to: configure a first scheduled task on the network traffic analysis platform, the first scheduled task corresponding to a preset first time period; call the first scheduled task regularly according to the first time period; when the first scheduled task is called for execution, trigger the following steps: obtain the traffic characteristic information of the interaction between the cloud platform and the network corresponding to the node planning task within the current first time period through the network traffic data channel.

[0200] In some embodiments, the device also includes: a data writing module, which is used to write the traffic characteristic information within the current first time period into a preset relational database after acquiring the traffic characteristic information within the current first time period; wherein the relational database includes multiple original data tables, and the traffic characteristic information within the current first time period is written into any one of the multiple original data tables.

[0201] In some embodiments, the relational database also includes a statistical data table; the data writing module is also used to: after obtaining the data to be analyzed under the network traffic dimension, construct a data identifier of the data to be analyzed based on the current first time period; determine a blank partition in the statistical data table where no information is written; and write the data identifier and the data to be analyzed into the blank partition.

[0202] In some embodiments, the acquisition module is also used to: determine the attribute parameters of each sub-information in the traffic characteristic information within the current first time period, the attribute parameters are used to characterize the geographical area corresponding to the sub-information and the autonomous system located in the geographical area; using the geographical area as the first aggregation dimension, aggregating the sub-information belonging to the same geographical area, and obtaining a first data volume of sub-information corresponding to each geographical area; and using the geographical area and the autonomous system as the second aggregation dimension, aggregating the sub-information belonging to the same geographical area and the same autonomous system, and obtaining a second data volume of sub-information under each autonomous system in each geographical area; determining the first data volume and the second data volume as the data to be analyzed under the network traffic dimension.

[0203] In some embodiments, the relational database also includes: a statistical data table corresponding to the first aggregation dimension and a statistical data table corresponding to the second aggregation dimension; the data writing module is also used to: after obtaining the first data amount of the sub-information corresponding to each geographical area, write the first data amount of the sub-information corresponding to each geographical area into the statistical data table corresponding to the first aggregation dimension; after obtaining the second data amount of the sub-information under each autonomous system in each geographical area, write the second data amount of the sub-information under each autonomous system in each geographical area into the statistical data table corresponding to the second aggregation dimension.

[0204] In some embodiments, the data acquisition channel includes a quality data detection channel; the data dimension includes a quality data dimension; the acquisition module is also used to: configure a second scheduled task on an external network quality detection platform, the second scheduled task corresponds to a preset second time period; according to the second time period, the second scheduled task is called regularly; when the second scheduled task is called for execution, the following steps are triggered: through the quality data detection channel deployed on the external network quality detection platform, real-time quality data under the quality data dimension is acquired; the real-time quality data is aggregated to obtain the data to be analyzed under the quality data dimension.

[0205] In some embodiments, the detection source area of ​​the real-time quality data includes: all geographical areas corresponding to the cloud platform corresponding to the node planning task; the detection target area of ​​the real-time quality data includes: sample Internet Protocol addresses in a specific autonomous system in the current existing geographical area; the acquisition module is also used for: when the real-time quality data under the quality data dimension is acquired based on the detection source area and the detection target area, aggregating multiple sub-data in the real-time quality data according to different quality data dimensions to obtain data to be analyzed under different quality data dimensions; the quality data dimensions include: delay dimension, packet loss rate dimension and unreachability rate dimension.

[0206] In some embodiments, the data acquisition channel includes a public publishing channel; the data dimension includes a public data dimension; the acquisition module is also used to: acquire public data under the public data dimension through the public publishing channel according to the publishing frequency of the public publishing channel; the public data constitutes the data to be analyzed.

[0207] In some embodiments, the data dimensions include a network traffic dimension, a quality data dimension, and a public data dimension; the data scoring module is also used to: for the data to be analyzed under the network traffic data dimension, determine the number of geographical areas corresponding to the data to be analyzed and the traffic ranking corresponding to each geographical area; based on the number of geographical areas and the traffic ranking, determine the traffic scoring result under the network traffic data dimension; for the data to be analyzed under the quality data dimension, determine the quality scoring result of the cloud platform corresponding to the node planning task and the quality scoring result between the cloud platform and other network platforms in turn based on the data to be analyzed; for the data to be analyzed under the public data dimension, determine the public data scoring result under the public data dimension based on the data to be analyzed and the number of geographical areas.

[0208] In some embodiments, the scoring fusion module is further used to: determine the weight of the data scoring result under each data dimension; and based on the weight, perform weighted summation of the data scoring results under the multiple data dimensions to obtain the fused scoring result.

[0209] It should be noted that the description of the device of the embodiment of the present application is similar to the description of the above method embodiment, and has similar beneficial effects as the method embodiment, so it is not repeated. For technical details not disclosed in the embodiment of the device, please refer to the description of the method embodiment of the present application for understanding.

[0210] The embodiment of the present application provides a computer program product, which includes an executable instruction, which is a computer instruction; the executable instruction is stored in a computer-readable storage medium. When a processor of an electronic device reads the executable instruction from the computer-readable storage medium and the processor executes the executable instruction, the electronic device executes the method described in the embodiment of the present application.

[0211] The present application embodiment provides a storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will execute the method provided by the present application embodiment, for example, Figure 3 The method shown.

[0212] In some embodiments, the storage medium can be a computer-readable storage medium, for example, a ferroelectric random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPR OM), a flash memory, a magnetic surface memory, an optical disk, or a compact disk read-only memory (CD-ROM), etc.; it can also be various devices including one or any combination of the above memories.

[0213] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0214] As an example, executable instructions may, but need not correspond to, a file in a file system, may be stored as part of a file storing other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files storing one or more modules, subroutines, or code portions). As an example, executable instructions may be deployed to be executed on one electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed at multiple locations and interconnected by a communication network.

[0215] The above is only an embodiment of the present application and is not intended to limit the protection scope of the present application. Any modifications, equivalent substitutions and improvements made within the spirit and scope of the present application are included in the protection scope of the present application.

Claims

1. A data processing method, characterized in that: The method comprises: Determine multiple data dimensions corresponding to the node planning task and a data acquisition channel corresponding to each of the data dimensions; each of the data dimensions corresponds to at least one data acquisition channel; Acquire the data to be analyzed under each data dimension through the data acquisition channel corresponding to each data dimension; Performing data scoring processing on the data to be analyzed under each of the data dimensions to obtain data scoring results under the corresponding data dimensions; Performing scoring fusion processing on the data scoring results under the multiple data dimensions to obtain a fusion scoring result; In response to a score display operation, at least one of the data score result and the fusion score result is displayed.

2. The method according to claim 1, characterized in that: The data acquisition channel includes a network traffic data channel; the data dimension includes a network traffic dimension; The acquiring the data to be analyzed under each data dimension through the data acquisition channel corresponding to each data dimension includes: Obtain traffic characteristic information of the interaction between the cloud platform and the network corresponding to the node planning task through the network traffic data channel on the network traffic analysis platform deployed at the network egress; The traffic characteristic information is aggregated to obtain the data to be analyzed under the network traffic dimension.

3. The method according to claim 2, characterized in that The method of obtaining traffic characteristic information of the interaction between the cloud platform and the network corresponding to the node planning task through the network traffic data channel on the network traffic analysis platform deployed at the network egress includes: Configuring a first scheduled task on the network traffic analysis platform, wherein the first scheduled task corresponds to a preset first time period; According to the first time period, regularly calling the first scheduled task; When the first scheduled task is called for execution, the following steps are triggered: obtaining, through the network traffic data channel, traffic characteristic information of the interaction between the cloud platform corresponding to the node planning task and the network during the current first time period.

4. The method according to claim 3, characterized in that The method further comprises: After acquiring the flow characteristic information in the current first time period, writing the flow characteristic information in the current first time period into a preset relational database; The relational database includes a plurality of original data tables, and the traffic characteristic information in the current first time period is written into any one of the plurality of original data tables.

5. The method according to claim 4, characterized in that The relational database also includes a statistical data table; the method also includes: After obtaining the data to be analyzed under the network traffic dimension, constructing a data identifier of the data to be analyzed based on the current first time period; Determining a blank partition in the statistical data table where no information is written; The data identifier and the data to be analyzed are written into the blank partition.

6. The method according to claim 5, characterized in that The aggregating the traffic characteristic information to obtain the data to be analyzed under the network traffic dimension includes: Determine the attribute parameters of each sub-information in the traffic characteristic information in the current first time period, wherein the attribute parameters are used to characterize the geographical area corresponding to the sub-information and the autonomous system located in the geographical area; Taking the geographical area as a first aggregation dimension, aggregating the sub-information belonging to the same geographical area to obtain a first data volume of the sub-information corresponding to each geographical area; and Taking the geographical area and the autonomous system in the geographical area as the second aggregation dimension, aggregating the sub-information belonging to the same geographical area and the same autonomous system to obtain a second data volume of the sub-information under each autonomous system in each geographical area; The first data volume and the second data volume are determined as the data to be analyzed under the network traffic dimension.

7. The method according to claim 6, characterized in that The relational database also includes: a statistical data table corresponding to the first aggregation dimension and a statistical data table corresponding to the second aggregation dimension; the method also includes: After obtaining the first data amount of the sub-information corresponding to each geographical area, writing the first data amount of the sub-information corresponding to each geographical area into a statistical data table corresponding to the first aggregation dimension; After obtaining the second data volume of the sub-information under each autonomous system in each geographical area, the second data volume of the sub-information under each autonomous system in each geographical area is written into a statistical data table corresponding to the second aggregation dimension.

8. The method according to claim 1, characterized in that The data acquisition channel includes a quality data detection channel; the data dimension includes a quality data dimension; The acquiring the data to be analyzed under each data dimension through the data acquisition channel corresponding to each data dimension includes: Configuring a second scheduled task on the external network quality detection platform, wherein the second scheduled task corresponds to a preset second time period; According to the second time period, regularly calling the second scheduled task; When the second scheduled task is called for execution, the following steps are triggered: obtaining real-time quality data under the quality data dimension through the quality data detection channel deployed on the external network quality detection platform; The real-time quality data is aggregated to obtain the data to be analyzed under the quality data dimension.

9. The method according to claim 8, characterized in that The detection source area of ​​the real-time quality data includes: all geographical areas corresponding to the cloud platform corresponding to the node planning task; the detection target area of ​​the real-time quality data includes: sample Internet Protocol addresses in a specific autonomous system in the current existing geographical area; The data aggregation of the real-time quality data to obtain the data to be analyzed under the quality data dimension includes: When real-time quality data under the quality data dimension is obtained based on the detection source area and the detection target area, According to different quality data dimensions, multiple sub-data in the real-time quality data are summarized to obtain data to be analyzed under different quality data dimensions; the quality data dimensions include: delay dimension, packet loss rate dimension and unreachable rate dimension.

10. The method according to claim 1, characterized in that The data acquisition channel includes a public release channel; the data dimension includes a public data dimension; The acquiring the data to be analyzed under each data dimension through the data acquisition channel corresponding to each data dimension includes: According to the release frequency of the public release channel, obtain the public data under the public data dimension through the public release channel; The public data constitute the data to be analyzed.

11. The method according to any one of claims 1 to 10, characterized in that: The data dimensions include network traffic dimension, quality data dimension and public data dimension; The data scoring process is performed on the data to be analyzed under each of the data dimensions to obtain the data scoring results under the corresponding data dimensions, including: For the data to be analyzed under the network traffic data dimension, determine the number of geographical areas corresponding to the data to be analyzed and the traffic ranking corresponding to each geographical area; based on the number of geographical areas and the traffic ranking, determine the traffic scoring result under the network traffic data dimension; For the data to be analyzed under the quality data dimension, determine the quality score results of the cloud platform corresponding to the node planning task and the quality score results between the cloud platform and other network platforms in sequence based on the data to be analyzed; For the data to be analyzed under the public data dimension, a public data scoring result under the public data dimension is determined based on the data to be analyzed and the number of the geographical areas.

12. The method according to claim 11, characterized in that The step of performing scoring fusion processing on the data scoring results under the multiple data dimensions to obtain a fusion scoring result includes: Determine the weight of the data scoring results under each data dimension; Based on the weights, the data scoring results under the multiple data dimensions are weighted and summed to obtain the fusion scoring result.

13. A data processing device, characterized in that: The device comprises: A determination module, used to determine multiple data dimensions corresponding to the node planning task and a data acquisition channel corresponding to each of the data dimensions; each of the data dimensions corresponds to at least one data acquisition channel; An acquisition module, used for acquiring the data to be analyzed under each data dimension through a data acquisition channel corresponding to each data dimension; A data scoring module is used to perform data scoring processing on the data to be analyzed under each of the data dimensions to obtain data scoring results under the corresponding data dimensions; A scoring fusion module is used to perform scoring fusion processing on the data scoring results under the multiple data dimensions to obtain a fusion scoring result; A display module is used to display at least one of the data scoring result and the fusion scoring result in response to the scoring display operation.

14. An electronic device, characterized in that: include: A memory for storing executable instructions; A processor, configured to implement the data processing method according to any one of claims 1 to 12 when executing the executable instructions stored in the memory.

15. A computer-readable storage medium, characterized in that: Executable instructions are stored, which are used to cause a processor to execute the executable instructions to implement the data processing method according to any one of claims 1 to 12.

16. A computer program product or a computer program, characterized in that The computer program product or computer program comprises executable instructions stored in a computer-readable storage medium; When the processor of the electronic device reads the executable instructions from the computer-readable storage medium and executes the executable instructions, the data processing method according to any one of claims 1 to 12 is implemented.

Images