Network data intelligent distribution system and method based on Internet

Abstract

The invention discloses a network data intelligent distribution system and method based on the Internet, relates to the technical field of network data intelligent distribution, and solves the problems that in the prior art, the transmission rate of network data distribution is limited, so that instructions with low priorities need to wait for a long time, and the transmission efficiency is low. The waiting time can increase the delay of overall transmission, and the overall response efficiency is low. The method comprises the following steps: collecting network data, constructing a network database server, and dividing the priority of the network data; sending a request instruction of the mobile terminal to a network database server, obtaining corresponding network data, and evaluating the priority of the request instruction; extracting the data size of the network data corresponding to the request instruction, and transmitting the network data corresponding to the request instruction to the mobile terminal in parallel based on the priority of the request instruction; ordered and rapid response to the request instructions at the same time is realized.

Description

Technical Field

[0001] The present invention belongs to the field of network data intelligent distribution, and in particular to an Internet-based network data intelligent distribution system and method. Background Art

[0002] With the development of national economy and technology and the improvement of people's living standards, data sharing and distribution technologies have been applied to all aspects of the information field. Due to the development of information dissemination technology, the amount of massive data to be distributed is becoming more and more heavy, which requires a stable and fast distribution service system.

[0003] The invention patent with publication number CN112653636B discloses a network data intelligent distribution service system, including a distributed message queue server, an instruction input module, an instruction analysis module, a data distribution module, an evaluation module and a message management module; the instruction analysis module is used to analyze the data distribution instruction and obtain the advantage value of the data distribution instruction; it can judge the priority level of the data distribution instructions requested at the same time according to the advantage value, and then process them in batches; the message management module is used to manage the messages cached in the distributed message queue server, reduce the cache pressure of the distributed message queue server, and thus improve the data distribution efficiency; when the data distribution module receives the data distribution instruction, it generates a distribution signal and transmits the distribution signal to the evaluation module; the evaluation module is used to receive the distribution signal and analyze it to obtain the distribution valuation; it is convenient for managers to have an intuitive understanding of the distribution situation of the system.

[0004] The prior art realizes the orderly transmission of data distribution instructions by analyzing the priority of distribution instructions and then responding in batches. However, the system processes instructions in a serial transmission manner, and low-priority instructions need to wait for a long time before being processed. This waiting time not only increases the transmission delay of low-priority instructions, but also affects the overall transmission efficiency, resulting in the problem of low overall response efficiency. Therefore, the present invention provides a network data intelligent distribution system and method based on the Internet. Summary of the invention

[0005] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes an Internet-based network data intelligent distribution system and method, which is used to solve the technical problem that the prior art has limitations in the transmission rate of network data distribution, resulting in low-priority instructions having to wait for a long time, and this waiting time will increase the overall transmission delay, resulting in low overall response efficiency.

[0006] To achieve the above-mentioned object, the first aspect of the present invention provides an Internet-based network data intelligent distribution system, comprising a request module, and a network data classification module and a distribution module connected thereto;

[0007] Network data classification module: used to build a network database server and prioritize network data;

[0008] Request module: used to send the request instruction of the mobile terminal to the network database server, obtain the corresponding network data, and evaluate the priority of the request instruction;

[0009] Distribution module: used to extract the data volume of the network data corresponding to the request instruction, and based on the priority of the request instruction, transmit the network data corresponding to the request instruction to the mobile terminal in parallel.

[0010] Preferably, the construction of the network database server includes:

[0011] The network data is classified according to the data type to obtain several different types of network data groups, and the collection of several network data groups is constructed as a network database server.

[0012] Preferably, the prioritization of network data includes:

[0013] Evaluate the importance of each type of network data and prioritize them; then evaluate the importance of each type of network data and prioritize them; the importance is evaluated by experts, and the larger the value, the higher the priority.

[0014] The present invention classifies network data according to data types, making data management more orderly and clear. Each type of data has its own specific storage and processing method, and after classification, storage space and computing resources can be used more effectively. By building a network database server, these classified data can be centrally managed to facilitate data query, analysis and application. This greatly improves the efficiency of data processing, especially in a big data environment. And by prioritizing network data, it can ensure that more important data is processed and analyzed first when resources are limited, which helps to quickly respond to key business needs and improve the efficiency and quality of decision-making.

[0015] Prioritization can also help identify which data is core data and which data is auxiliary data, thereby being more focused and efficient in data analysis and mining.

[0016] Preferably, the priority of the evaluation request instruction includes:

[0017] The priority of the mobile terminal is obtained, the priority of the mobile terminal is multiplied by the corresponding weight coefficient, and the priority of the network data corresponding to the request instruction is multiplied by the corresponding weight coefficient, and then the sum of the two is calculated as the priority of the request instruction.

[0018] The present invention comprehensively considers the priorities of network data corresponding to mobile terminals and request instructions, analyzes the priority of request instructions, and enables the system to manage network resources more finely. Traditional network resource management methods often focus on overall load balancing and flow control, but lack consideration of the priority of specific request instructions. The present invention optimizes the allocation of transmission resources by introducing a priority evaluation mechanism for request instructions, which not only improves the utilization efficiency of network resources, but also avoids waste and idleness of resources.

[0019] Preferably, obtaining the priority of the mobile terminal includes:

[0020] Extracting the performance parameters of the mobile terminal, evaluating the priority of the performance of the mobile terminal, obtaining the transmission distance between the mobile terminal and the network database server, and evaluating the priority of the transmission distance; wherein the priority of the mobile terminal performance and the transmission distance is evaluated by experts; the better the performance of the mobile terminal or the shorter the transmission distance, the higher the priority;

[0021] The product of the priority of the mobile terminal performance and the corresponding weight coefficient is calculated, the product of the priority of the transmission distance and the corresponding weight coefficient is calculated, and the two are summed to obtain the priority of the mobile terminal.

[0022] The present invention comprehensively considers the performance of mobile terminals and the transmission distance between them and the network database server, analyzes the priority of mobile terminals, and defines that mobile terminals with superior performance and mobile terminals with shorter transmission distances can obtain higher priorities, thereby ensuring that they can obtain sufficient bandwidth and response speed even when the network is busy, avoiding excessive occupation of resources by terminals with poor performance or long distances, and improving the transmission efficiency and resource utilization efficiency of the overall network.

[0023] Preferably, the obtaining of the transmission distance between the mobile terminal and the network database server includes:

[0024] The network database server sends a location acquisition instruction to the mobile terminal, obtains the real-time location of the mobile terminal, calculates the distance between the two, and obtains the transmission distance between the two.

[0025] Preferably, the transmitting of the network data corresponding to the request instruction to the mobile terminal in parallel includes:

[0026] When the network database server receives several request instructions at the same time, it transmits them according to the priority of the request instructions; the higher the priority, the earlier it is transmitted; if the network data have the same priority, the network data with the largest data volume will be transmitted first, from the smallest to the largest;

[0027] The network data corresponding to the request instruction is divided into a number of network data segments in turn according to the bandwidth of the transmission channel; each network data segment is numbered in sequence, and the network data segment is transmitted through the data transmission channel of the corresponding bandwidth;

[0028] The transmission channel transmits the network data fragments to the message buffer area, and the message buffer area reassembles the received network data fragments according to the sequence of numbers, and transmits them to the mobile terminal after the reassembly is completed.

[0029] The present invention divides network data into several fragments, and divides them cyclically in sequence according to the bandwidth of the transmission channel, which can make full use of transmission channels with different bandwidths and avoid bandwidth waste; and the message buffer area reorganizes the received network data fragments according to the numbering order to ensure the integrity and correctness of the data. This method ensures that when the network database server receives request instructions from several mobile terminals at the same time, it responds to the mobile terminals in an orderly and rapid manner.

[0030] Preferably, when the transmission channel transmits a plurality of network fragment data, the network fragment data are sequentially transmitted to the message buffer area through the transmission channel according to the numbers of the network fragment data.

[0031] Preferably, the request module also includes sending a prohibition instruction to the mobile terminal and setting a prohibition duration when the number of requests from the mobile terminal exceeds a threshold within a preset time; wherein the prohibition duration can be determined based on the number of requests from the mobile terminal within the preset time, and the more requests, the longer the prohibition duration.

[0032] In order to prevent mobile terminals from abusing server resources or conducting malicious attacks (such as DDoS attacks) while protecting the stability and security of the server, the present invention limits the number of requests and implements temporary prohibition measures when the threshold is exceeded. The system can effectively manage and control access traffic, ensure the rational allocation and use of resources, and build an efficient, secure and stable network database server environment.

[0033] Preferably, the present invention provides a method for intelligently distributing network data based on the Internet, comprising the following steps:

[0034] Step 1: Collect network data, build a network database server, and prioritize network data;

[0035] Step 2: Send the request instruction of the mobile terminal to the network database server, obtain the corresponding network data, and evaluate the priority of the request instruction;

[0036] Step three: extract the data volume of the network data corresponding to the request instruction, and based on the priority of the request instruction, transmit the network data corresponding to the request instruction to the mobile terminal in parallel.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] In order to process several request instructions at the same time, the present invention needs to evaluate the priority of the request instructions from multiple aspects, so that the server can respond to each request instruction in an orderly manner and avoid resource congestion and conflict on the server side. That is, the priority of the request instruction is jointly evaluated by the priority of the network data corresponding to the request instruction and the priority of the mobile terminal, and the evaluation result is accurate; and based on the priority of the request instruction, the network data corresponding to the request instruction is transmitted to the mobile terminal in parallel, ensuring a rapid response to high-priority requests; in addition, in order to maximize the use of network resources, the present invention further divides the network data into multiple segments according to the bandwidth of the transmission channel, and multiple transmission channels are transmitted in parallel, which significantly speeds up the data transmission speed; the present invention analyzes the priority of the request instruction and the multi-channel parallel transmission method to ensure that when the server receives multiple request instructions at the same time, it can efficiently and orderly process these requests, and help maintain the load balance of the server to avoid excessive concentration or idle resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0040] Figure 1 It is a system structure block diagram of the present invention;

[0041] Figure 2 This is a flow chart of the network data transmission method of the present invention;

[0042] Figure 3 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0043] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] See also Figure 1 , the first aspect of the present invention provides an Internet-based network data intelligent distribution system, including a request module, and a network data classification module and a distribution module connected thereto;

[0045] The network data classification module constructs a network database server and assigns priorities to network data;

[0046] Specifically, the collected network data is classified according to data types to obtain several different types of network data groups, and the set of several network data groups is constructed as the network database server.

[0047] Evaluate the importance of each type of network data and assign a priority order; then evaluate the importance of each network data within each type and assign a priority; among them, the importance is evaluated by experts, and the larger the value, the higher the priority.

[0048] Network data such as web content: web files composed of HTML, CSS, JavaScript, etc.; multimedia files: media files such as pictures, audio, and video; documents: file formats such as PDF, Word documents, Excel spreadsheets, etc.; application programs: executable files, software update packages, etc. Application layer data: API responses: data in formats such as JSON and XML returned by Web APIs; database query results: data obtained from remote databases, such as user information, order records, etc.; log data: log files generated by application programs and services for debugging and monitoring.

[0049] Suppose the experts evaluate the importance of network data in a web page in ascending order as HTML < CSS < JavaScript. Then HTML has the lowest priority, JavaScript has the highest priority, and the priority of CSS is between HTML and JavaScript.

[0050] The request module sends a request instruction from the mobile terminal to the network database server, obtains the corresponding network data, and evaluates the priority of the request instruction;

[0051] Specifically, obtain the priority of the mobile terminal, multiply the priority of the mobile terminal by the corresponding weight coefficient, and multiply the priority of the network data corresponding to the request instruction by the corresponding weight coefficient, and then sum the two to obtain the priority of the request instruction.

[0052] For example: the priority of the mobile terminal is marked as YB, and its corresponding weight coefficient is A; the priority of the corresponding network data is YW, and its corresponding weight coefficient is C; then through the formula YQ = YB × A + YW × C; A + B = 1.

[0053] The priority of the request instruction in the present invention is used to evaluate the speed of response of the request instruction; the speed of response of the request instruction is mainly closely related to the priority of the mobile terminal, the priority of the corresponding network data, and their respective corresponding weight coefficients. When any one of the priority levels is higher, the priority of the request instruction is higher, and then the network database server needs to respond to this request instruction as soon as possible.

[0054] Extract performance parameters of mobile terminals, such as smart phones and tablet computers, including processors, memory, etc.; evaluate the performance of mobile terminals in priority, and have the network database server send a location acquisition command to the mobile terminal to obtain the real-time location of the mobile terminal, calculate the distance between the two, obtain the transmission distance between the two, and evaluate the priority of the transmission distance; the priority of mobile terminal performance and transmission distance is evaluated by experts; the better the performance of the mobile terminal or the shorter the transmission distance, the higher the priority;

[0055] The priority of the mobile terminal performance is multiplied by the corresponding weight coefficient, and the priority of the transmission distance is multiplied by the corresponding weight coefficient, and then the sum of the two is calculated to obtain the priority of the mobile terminal.

[0056] For example: based on the performance parameters of the mobile terminal, the expert comprehensively evaluates its priority as YZ, and its corresponding weight coefficient is D; and based on the distance between the mobile terminal and the network database server, that is, the transmission distance, the expert evaluates its priority as YL, and its corresponding weight coefficient is E; then the formula YQ=YZ×D+YL×E; D+E=1.

[0057] The evaluation of the priority of the mobile terminal of the present invention is mainly closely related to the priority of the mobile terminal performance, the priority of the transmission distance, and the corresponding weight coefficients. When any one of the priorities is higher, the priority of the mobile terminal is higher; that is, when the network data priority is the same, the higher the priority of the mobile terminal, the higher the priority of the request instruction sent by the mobile terminal, and the network database server will give priority to responding to the request instruction sent by the mobile terminal.

[0058] The request module also includes sending a prohibition instruction to the mobile terminal and setting a prohibition duration when the number of requests from the mobile terminal exceeds a threshold within a preset time; wherein the prohibition duration can be determined based on the number of requests from the mobile terminal within the preset time, and the more requests, the longer the prohibition duration.

[0059] For example, some malicious users may use automated tools to launch a large number of requests to the server in an attempt to exhaust server resources or cause service unavailability (DDoS attack). By setting a request count threshold, the system can take action when an abnormally high frequency of requests is detected to prevent potential attacks.

[0060] Even if it is not a malicious attack, some users may frequently request the same resource due to misoperation or other reasons, causing increased server load. For example, an application may continuously retry requests without proper error handling, resulting in unnecessary resource consumption.

[0061] The distribution module extracts the data volume of the network data corresponding to the request instruction, and based on the priority of the request instruction, transmits the network data corresponding to the request instruction to the mobile terminal in parallel.

[0062] Specifically, when the network database server receives several request instructions at the same time, it transmits them according to the priority of the request instructions; the higher the priority, the earlier it is transmitted; if the network data have the same priority, the network data will be transmitted first, in order of size, from small to large;

[0063] See also Figure 2 , dividing the network data corresponding to the request instruction into a number of network data segments in turn according to the bandwidth of the transmission channel; numbering each network data segment in sequence, and transmitting the network data segment through the data transmission channel of the corresponding bandwidth; wherein, the network data segments of the same transmission channel are transmitted in the order of numbering;

[0064] The transmission channel transmits the network data fragments to the message buffer area, and the message buffer area reassembles the received network data fragments according to the sequence of numbers, and transmits them to the mobile terminal after the reassembly is completed.

[0065] It should be noted that the network data segments corresponding to the request instructions are transmitted in sequence in the corresponding transmission channels.

[0066] For example, at a certain moment, several mobile terminals simultaneously initiate request instructions to the network database server, which can be divided into the following two situations:

[0067] The first case: when the request instructions of several mobile terminals are all different; the network data corresponding to the request instructions are obtained, and the network data is divided according to the transmission bandwidth of several transmission channels;

[0068] For example, suppose there are three mobile terminals A, B and C, which respectively send the following requests to the server: Terminal A requests a high-definition video stream, 500M; Terminal B requests to download a medium-sized software package, 200MB; Terminal C requests a series of small files, such as pictures and texts, 10MB; There are three available transmission channels in the network, and their bandwidths are: 20Mbps, 10Mbps, 15Mbps.

[0069] According to the priority evaluation, the priorities of the three mobile terminals are ranked as A>B>C; A has the highest priority and is transmitted first;

[0070] For example, the network data corresponding to the A request instruction is obtained and divided according to the bandwidth of the transmission channel. The network data corresponding to the A request is divided into 20MB, 10MB, and 15MB in sequence until the division is completed. The obtained network data segments are 11 20MB, 11 10MB, 11 15MB, and 1 5MB respectively; among which, 5MB is transmitted in a 10bps transmission channel, and the other network data segments are transmitted in a transmission channel of the corresponding bandwidth; and the network data are numbered in the connection order, such as the first network data segment (20MB) divided is marked as 1, the second network segment data segment (10MB) is marked as 2, the third network segment data segment (15MB) is marked as 3, the fourth network segment data segment (20MB) is marked as 4, the fifth network segment data segment (10MB) is marked as 5, the sixth network segment data segment (15MB) is marked as 6, and so on are marked in sequence;

[0071] A 20bps transmission channel needs to transmit 11 20MB network data segments, so the 20bps transmission channel transmits in sequence according to the numbering order of the network data segments (20MB); the transmission order of transmission channels of other bandwidths is the same; and all transmission channels transmit at the same time, transmitting the network data segments to the message buffer area, which is reorganized in the numbering order of the network data segments and transmitted to the mobile terminal.

[0072] According to the bandwidth, the 20bps and 15bps transmission channels simultaneously complete the network data fragments corresponding to mobile terminal A; therefore, the two transmission channels will first transmit the next priority B network data fragment (20MB, 15MB), that is, each transmission channel is continuously transmitting data to complete the requests of A, B, and C.

[0073] It should be noted that, when the data size of the remaining network data fragment is smaller than the bandwidth of any transmission channel, the network data fragment is allocated to the transmission channel with the closest bandwidth for transmission.

[0074] The second situation: when there are identical request instructions from mobile terminals, the data volume of the network data corresponding to each request instruction is extracted, and the network data with a smaller data volume is transmitted first.

[0075] See also Figure 3 The present invention provides a method for intelligently distributing network data based on the Internet, comprising the following steps:

[0076] Step 1: Collect network data, build a network database server, and prioritize network data;

[0077] Step 2: Send the request instruction of the mobile terminal to the network database server, obtain the corresponding network data, and evaluate the priority of the request instruction;

[0078] Step three: extract the data volume of the network data corresponding to the request instruction, and based on the priority of the request instruction, transmit the network data corresponding to the request instruction to the mobile terminal in parallel.

[0079] Part of the data in the above formula is calculated by removing the dimension and taking its numerical value. The formula is a formula closest to the actual situation obtained by software simulation of a large amount of collected data; the preset parameters and preset thresholds in the formula are set by technical personnel in this field according to actual conditions or obtained through simulation of a large amount of data.

[0080] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. An Internet-based network data intelligent distribution system, characterized in that: It includes a request module, and a network data classification module and a distribution module connected thereto; Network data classification module: used to build a network database server and prioritize network data; Request module: used to send the request instruction of the mobile terminal to the network database server, obtain the corresponding network data, and evaluate the priority of the request instruction; Distribution module: used to extract the data volume of the network data corresponding to the request instruction, and based on the priority of the request instruction, transmit the network data corresponding to the request instruction to the mobile terminal in parallel.

2. The Internet-based network data intelligent distribution system according to claim 1, characterized in that: The construction of the network database server includes: The network data is classified according to the data type to obtain several different types of network data groups, and the collection of several network data groups is constructed as a network database server.

3. The Internet-based network data intelligent distribution system according to claim 2, characterized in that: The prioritization of network data includes: Evaluate the importance of each type of network data and prioritize them; then evaluate the importance of each type of network data and prioritize them.

4. The Internet-based network data intelligent distribution system according to claim 3, characterized in that: The priority of the evaluation request instruction includes: Obtain the priority of the mobile terminal, calculate the product of the priority of the mobile terminal and the corresponding weight coefficient, and the product of the priority of the network data corresponding to the request instruction and the corresponding weight coefficient, and sum them to obtain the priority of the request instruction.

5. The Internet-based network data intelligent distribution system according to claim 4, characterized in that: The obtaining the priority of the mobile terminal includes: Extracting performance parameters of the mobile terminal, performing priority evaluation on the performance of the mobile terminal, and obtaining the transmission distance between the mobile terminal and the network database server, and evaluating the priority of the transmission distance; The product of the priority of the mobile terminal performance and the corresponding weight coefficient is calculated, the product of the priority of the transmission distance and the corresponding weight coefficient is calculated, and the two are summed to obtain the priority of the mobile terminal.

6. The Internet-based network data intelligent distribution system according to claim 5, characterized in that: The obtaining of the transmission distance between the mobile terminal and the network database server includes: The network database server sends a location acquisition instruction to the mobile terminal, obtains the real-time location of the mobile terminal, calculates the distance between the two, and obtains the transmission distance between the two.

7. The Internet-based network data intelligent distribution system according to claim 1, characterized in that: The transmitting of network data corresponding to the request instruction to the mobile terminal in parallel includes: When the network database server receives several request instructions at the same time, it transmits them according to the priority of the request instructions; When the network data have the same priority, the network data with the largest data size will be transmitted first, from the smallest to the largest. The network data corresponding to the request instruction is divided into a number of network data segments in turn according to the bandwidth of the transmission channel; each network data segment is numbered in sequence, and the network data segment is transmitted through the data transmission channel of the corresponding bandwidth; The transmission channel transmits the network data fragments to the message buffer area, and the message buffer area reassembles the received network data fragments according to the sequence of numbers, and transmits them to the mobile terminal after the reassembly is completed.

8. The Internet-based network data intelligent distribution system according to claim 7, characterized in that: When the transmission channel transmits a plurality of network segment data, the network segment data are sequentially transmitted to the message buffer area through the transmission channel according to the numbers of the network segment data.

9. The Internet-based network data intelligent distribution system according to claim 1, characterized in that: The request module also includes sending a prohibition instruction to the mobile terminal and setting a prohibition time when the number of requests from the mobile terminal exceeds a threshold within a preset time.

10. A method for intelligently distributing network data based on the Internet, based on the operation of an intelligently distributing network data based on the Internet as claimed in any one of claims 1 to 9, characterized in that: The following steps are involved: Step 1: Collect network data, build a network database server, and prioritize network data; Step 2: Send the request instruction of the mobile terminal to the network database server, obtain the corresponding network data, and evaluate the priority of the request instruction; Step three: extract the data volume of the network data corresponding to the request instruction, and based on the priority of the request instruction, transmit the network data corresponding to the request instruction to the mobile terminal in parallel.

Images