Efficient data transmission method and transmission system

Abstract

The present invention belongs to the field of electrical communication technology, and specifically relates to key distribution, and more particularly to an efficient data transmission method and transmission system, which comprises: encrypting data, and judging whether the data is normal after obtaining the encrypted data; distributing the encrypted data to a corresponding server; after the data is distributed to the corresponding server, transferring the data stored in the corresponding server through one of the servers; realizing encryption of data, avoiding data leakage or theft, and improving data transmission efficiency.

Description

Technical Field

[0001] The present invention belongs to the field of electrical communication technology, and specifically relates to key distribution, and more particularly to an efficient data transmission method and transmission system. Background Art

[0002] Data needs to be encrypted during transmission, and keys are used to prevent the data from being stolen. However, due to the large amount of data during transmission, the data transmission efficiency will be reduced. In addition, due to the differences in performance of each server, a data stream will be transmitted and stored through multiple servers, which will reduce the data transmission and storage efficiency.

[0003] Therefore, due to the low data transmission efficiency and technical issues of encryption requirements, it is necessary to design a new and efficient data transmission method and transmission system.

[0004] It should be noted that the above information disclosed in this background technology section is only used to understand the background technology of the present application concept, and therefore, the above description is not considered to constitute information of the prior art. Summary of the invention

[0005] The embodiments of the present disclosure at least provide an efficient data transmission method and a transmission system.

[0006] In a first aspect, an embodiment of the present disclosure provides an efficient data transmission method, comprising the following steps:

[0007] Encrypt the data and determine whether the data is normal after obtaining the encrypted data;

[0008] Distribute the encrypted data to the corresponding server;

[0009] After the data is distributed to the corresponding server, the data already stored in the corresponding server is transferred through one of the servers.

[0010] In an optional implementation, the method for encrypting data includes:

[0011] Encrypt the data, that is

[0012] Acquire an original data check value according to the original data, wherein the original data check value includes: a summary value of the original data, and a start and end position of the original data;

[0013] Construct encrypted data based on original data, original data check value and hashed data.

[0014] In an optional implementation, the hash data is some random hash data appended to the original data, and the hash data is used to make the summary value of the encrypted data the same as the summary value of the original data.

[0015] In an optional implementation, the method for determining whether the encrypted data is normal after obtaining the encrypted data includes:

[0016] After obtaining the encrypted data, first obtain the summary value of the entire encrypted data, then obtain the summary value of the original data recorded in the original data check value, and then obtain the summary value of the original data in the encrypted data. If the summary values ​​of the three are the same, it is determined that the original data is normal.

[0017] In an optional implementation, the method of distributing the encrypted data to the corresponding server includes:

[0018] Group the servers according to their network port transmission speeds, group the servers with the same network port transmission speeds into one group, and sort all the servers in the group from small to large according to the network port transmission speeds;

[0019] In a server group whose network port transmission speed is lower than a preset speed, determine the health status of the servers therein.

[0020] In an optional implementation, the method of distributing the encrypted data to the corresponding server further includes:

[0021] Identify the original data from the encrypted data, divide the original data according to the length of the data, record the server group corresponding to each segment of data after the division, and the position sequence of each segment of data in the original data, and add the position sequence of each segment of data to the segment of data;

[0022] Each piece of data is sent to the corresponding group of servers for storage. Each group of servers determines the time required to store the corresponding data. If the server group with the highest network port transmission speed among the servers used to store the data requires the longest time, the corresponding data will be further divided.

[0023] In an optional implementation, each piece of data is sent to a server of a corresponding group for storage.

[0024] The original data is sent to the server group with the fastest network port transmission speed, and then each piece of data is sent to the corresponding server group through the server group with the fastest network port transmission speed. The fragmented files in the original data are sent to the server group with the slowest network port transmission speed, and are first stored through the server with the best health status in this group of servers;

[0025] After the server group with the highest network port transmission speed sends each segment of data to the server of the corresponding group, the server with the highest network port transmission speed transfers the data in other groups to the server group with the same network port transmission speed at every preset time interval.

[0026] In an optional implementation, when data stored in the server group needs to be retrieved, the data is retrieved by verifying the first key corresponding to the position sequence of each piece of data in the original data and the preset second key at the same time.

[0027] In a second aspect, an embodiment of the present disclosure further provides a data transmission system using the above-mentioned efficient data transmission method, including:

[0028] The host computer is configured to encrypt the data, determine whether the data is normal after obtaining the encrypted data, and then distribute the encrypted data to the corresponding server.

[0029] In a third aspect, an embodiment of the present disclosure further provides a computer device including a memory, a processor, and a computer program stored in the memory, characterized in that the processor executes the computer program to implement the steps of the above-mentioned efficient data transmission method.

[0030] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium having a computer program / instruction stored thereon, wherein the computer program / instruction, when executed by a processor, implements the steps of the above-mentioned efficient data transmission method.

[0031] In a fifth aspect, an embodiment of the present disclosure further provides a computer program product, including a computer program / instructions, characterized in that when the computer program / instructions are executed by a processor, the steps of the above-mentioned efficient data transmission method are implemented.

[0032] The beneficial effects of the present invention are that the present efficient data transmission method encrypts data and determines whether the data is normal after obtaining the encrypted data; distributes the encrypted data to the corresponding server; after the data is distributed to the corresponding server, the data already stored in the corresponding server is transferred through one of the servers; the encryption of the data is achieved, data leakage or theft is avoided, and the data transmission efficiency is improved.

[0033] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0034] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0036] Figure 1 A flowchart of an efficient data transmission method provided by an embodiment of the present disclosure;

[0037] Figure 2 A schematic diagram of a server provided in an embodiment of the present disclosure;

[0038] Figure 3 A schematic diagram of data partitioning provided for an embodiment of the present disclosure. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0040] As used herein, the phrases "in one embodiment," "according to one embodiment," "in some embodiments," and the like generally refer to the fact that the particular feature, structure, or characteristic following the phrase may be included in at least one embodiment of the present disclosure. Therefore, a particular feature, structure, or characteristic may be included in more than one embodiment of the present disclosure, so that these phrases do not necessarily refer to the same embodiment. As used herein, the terms "example," "exemplary," and the like are used to "serve as an example, instance, or illustration." Any implementation, aspect, or design described herein as "example" or "exemplary" is not necessarily to be construed as preferred or superior to other implementations, aspects, or designs. On the contrary, the use of the terms "example," "exemplary," and the like is intended to present concepts in a concrete manner.

[0041] Research has found that in related technologies, since the servers used to store data will be replaced, the replaced servers can still be used, resulting in performance differences between servers. For example, there are differences in the transmission speed of the servers, and there are also differences in the health of the hard disks used to build the servers, resulting in differences in the efficiency of data transmission, resulting in reduced data transmission efficiency; and data is stored in the server for a long time to form cold data, which can easily lead to data loss and data corruption.

[0042] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0043] like Figure 1 As shown, in the first aspect, the embodiment of the present disclosure provides an efficient data transmission method, comprising the following steps: encrypting data, and determining whether the data is normal after obtaining the encrypted data; allocating the encrypted data to the corresponding server; after the data is allocated to the corresponding server, transferring the data already stored in the corresponding server through one of the servers; encrypting the data to prevent data theft and improve data security; storing data of different segments in an entire data stream through different servers to improve data storage efficiency; transferring the stored data through the server to avoid the emergence of cold data and avoid data damage caused by long-term storage of data.

[0044] In an optional embodiment, the method for encrypting data includes: encrypting data, that is, obtaining an original data check value according to the original data, the original data check value includes: a summary value of the original data, and the start and end positions of the original data; constructing encrypted data according to the original data, the original data check value and the hash data; the hash data is some random messy data attached to the original data, and the summary value of the encrypted data is made the same as the summary value of the original data by the hash data. After obtaining the encrypted data, first obtain the summary value of the entire encrypted data, then obtain the summary value of the original data recorded in the original data check value, and then obtain the summary value of the original data in the encrypted data. If the summary values ​​of the three are the same, it is judged that the original data is normal; the original data is processed by various means to avoid tampering with the original data, thereby improving the security performance of the original data; the encryption process of the data can be processed in the upper computer, and the upper computer performs encryption processing after receiving the original data, and then sends the encrypted data to the server, etc.

[0045] like Figure 2As shown, in an optional implementation, the method of allocating encrypted data to corresponding servers includes: grouping and dividing servers according to the network port transmission speed of the servers, dividing servers with the same network port transmission speed into one group, and sorting all groups of servers from small to large according to the network port transmission speed; in the server group whose network port transmission speed is lower than the preset speed, judging the health status of the servers therein; there are multiple servers, including newly purchased servers and replaced servers, the replaced servers can still be used, the new servers will have better performance, that is, the new servers have faster network port transmission speed, faster read and write speed, and better health, and the old servers may have lower network port transmission speed, slower read and write speed, and lower health; the servers are composed of hard disk arrays; according to the network port transmission speed of each server, the servers with the same network port transmission speed are divided into one group; the same group of servers may include multiple servers, each of which has a different health level, and the servers in the same group of servers are sorted from high to low according to the health level; after sorting the server groups according to the network port transmission speed, the server group with the largest (fastest) network port transmission speed is selected, and this group of servers can be used for data transfer and distribution.

[0046] like Figure 3As shown, in an optional implementation, the method of allocating encrypted data to corresponding servers also includes: identifying original data from encrypted data, dividing the original data according to the length of the data, recording the server group corresponding to each segment of data after the division, and the position sequence of each segment of data in the original data, and adding the position sequence of each segment of data to the segment of data; sending each segment of data to the corresponding group of servers for storage, each group of servers determines the time required to store the corresponding data, if the server group with the largest network port transmission speed among the servers used to store the data requires the longest time, then the corresponding data is further divided; for example, there are now four groups of servers, and the corresponding network port transmission speeds are 100M, 1G, 10G and 20G respectively, each group of servers can have multiple servers and / or multiple hard disk arrays, etc., or each network port transmission speed corresponds to multiple groups of servers, then the server group corresponding to 20G is used for data transfer and distribution, the fragmented files in the complete original data are sent to the server group corresponding to 100M, and are first stored from the servers with high health in the group of servers. Since the fragmented files are very small, the storage of the fragmented files is only related to the service The read and write speed of the device is determined by the server group corresponding to the 100M network port transmission speed, and the complete original data is stored. The complete original data is divided into many segments of data with different data lengths. The longest segment of data corresponds to the server group with a 10G network port transmission speed, the middle segment of data corresponds to the server group with a 1G network port transmission speed, and the shortest segment of data corresponds to the server group with a 100M network port transmission speed. The data transmission time corresponding to the three groups of servers is basically the same, so as to improve the data transmission time. If the data transmission time of the longest segment is longer than that of the other two groups of servers, the longest segment of data can be divided again, such as dividing the longest segment of data into two segments, so as to simultaneously send them to two groups of servers with a network port transmission speed of 10G, so as to speed up the data transmission speed and improve the efficiency of data transmission. When the data is divided and divided in the original data, the position of the divided data remains unchanged, so that the complete original data can be formed when the data is retrieved. After the data is divided, a position order can be given to the divided data according to the position of the divided data in the original data. The position order of each segment of data is added to the segment of data, and the position order is used as a key to facilitate the encryption of each segment of data.

[0047] In an optional implementation, each segment of data is sent to a server of a corresponding group for storage, that is, the original data is sent to a server group with the highest network port transmission speed, and then each segment of data is sent to a corresponding server group via the server group with the highest network port transmission speed, and the fragmented files in the original data are sent to a server group with the lowest network port transmission speed, and are first stored through a server in the group with the best health status; after the server group with the highest network port transmission speed sends each segment of data to a server of a corresponding group, the data in other groups is transferred to a server group with the same network port transmission speed via the server with the highest network port transmission speed at preset intervals. group; first send the divided original data to the server group corresponding to 20G, and then send each segment of data to the corresponding server group through the server. If the data is stored in the server for too long, it will become cold data, resulting in data missing or loss. Therefore, at intervals, each group of servers sends the data to the server group corresponding to 20G, and then sends it to the new server group through the server group corresponding to 20G. The network port transmission speed of the new server group is the same as the network port transmission speed of the original storage of the corresponding segment data. After the data is transferred, the data segment that was about to become cold data forms new data in the new server group, avoiding data loss and missing.

[0048] In an optional embodiment, when it is necessary to retrieve data stored in a server group, the data is retrieved by verifying the first key corresponding to the position order of each segment of data in the original data and the preset second key at the same time; the second key can be generated in advance by software and sent to the retrieval end when the data needs to be retrieved; the first key is generated when the data is divided, and the data division is caused by differences in server performance, which can be regarded as generated according to the hardware level. Keys are generated from both the hardware and software levels to ensure security when data is retrieved.

[0049] Based on the same technical concept, at least one embodiment also provides a data transmission system that adopts the above-mentioned efficient data transmission method, including: a host computer, configured to encrypt the data, and after obtaining the encrypted data, determine whether the data is normal, and then distribute the encrypted data to the corresponding server.

[0050] Based on the same technical concept, at least one embodiment further provides a computer device including a memory, a processor and a computer program stored in the memory, characterized in that the processor executes the computer program to implement the steps of the above-mentioned efficient data transmission method.

[0051] Based on the same technical concept, at least one embodiment further provides a computer-readable storage medium having a computer program / instruction stored thereon, characterized in that the computer program / instruction implements the steps of the above-mentioned efficient data transmission method when executed by a processor.

[0052] Based on the same technical concept, at least one embodiment further provides a computer program product, including a computer program / instruction, characterized in that the computer program / instruction implements the steps of the above-mentioned efficient data transmission method when executed by a processor.

[0053] To sum up, this efficient data transmission method encrypts the data and determines whether the data is normal after obtaining the encrypted data; distributes the encrypted data to the corresponding server; after the data is distributed to the corresponding server, the data already stored in the corresponding server is transferred through one of the servers; the encryption of the data is realized, data leakage or theft is avoided, and the data transmission efficiency is improved.

[0054] A computer program (also referred to as a program, software, software application, script, or code) may be written in any form of programming language (including compiled or interpreted languages) and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program, or in multiple coordinated files (e.g., files that store one or more modules, subroutines, or portions of code). A computer program may be deployed for execution on one or more computers that are located at one site or distributed across multiple sites and interconnected by a communications network.

[0055] The processes and logic flows described in this document may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed by, and the apparatus may also be implemented as, special purpose logic circuits, such as an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[0056] For example, processors suitable for executing computer programs include general and special purpose microprocessors, and any one or more of any type of digital computer. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The basic components of a computer are a processor that executes instructions and one or more storage devices that store instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as magnetic disks, magneto-optical disks, or optical disks, or operably coupled to receive data from a mass storage device or transfer data to a mass storage device, or both. However, a computer does not necessarily have such a device. Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, including, for example, semiconductor memory devices, such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and compact disk read-only memory (CD ROM) and digital versatile disk read-only memory (DVD-ROM) disks. The processor and memory can be supplemented by dedicated logic circuits, or incorporated in dedicated logic circuits.

[0057] Likewise, although operations are described in a particular order in the drawings, this should not be understood as requiring that such operations be performed in the particular order or order shown, or that all illustrated operations be performed, in order to achieve the desired results. In addition, the separation of various system components in the embodiments of this patent document should not be understood as requiring such separation in all embodiments.

[0058] In several embodiments provided herein, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, program segment or a part of the code contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0059] In addition, without departing from the scope of the present disclosure, the discrete or separate techniques, systems, subsystems, and methods described and illustrated in the various embodiments may be combined or integrated with other systems, modules, techniques, or methods. Other items shown or discussed as coupled may be directly connected, or may be indirectly coupled or communicated electrically, mechanically, or otherwise through some interface, device, or intermediate component. Other examples of changes, substitutions, and alterations may be determined by those skilled in the art without departing from the spirit and scope disclosed herein.

Claims

1. An efficient data transmission method, characterized in that: The following steps are involved: Encrypt the data and determine whether the data is normal after obtaining the encrypted data; Distribute the encrypted data to the corresponding server; After the data is distributed to the corresponding servers, the data already stored in the corresponding servers is transferred through one of the servers; The method for encrypting data includes: Encrypting the data, that is, obtaining an original data check value according to the original data, wherein the original data check value includes: a summary value of the original data, and the start and end bits of the original data; Construct encrypted data based on original data, original data check value and hashed data; The method for distributing the encrypted data to the corresponding server includes: Group the servers according to their network port transmission speeds, group the servers with the same network port transmission speeds into one group, and sort all the servers in the group from small to large according to the network port transmission speeds; In a server group whose network port transmission speed is lower than a preset speed, determine the health status of the servers therein; Identify the original data from the encrypted data, divide the original data according to the length of the data, record the server group corresponding to each segment of data after the division, and the position sequence of each segment of data in the original data, and add the position sequence of each segment of data to the segment of data; Each piece of data is sent to the corresponding group of servers for storage. Each group of servers determines the time required to store the corresponding data. If the server group with the highest network port transmission speed among the servers used to store the data requires the longest time, the corresponding data is further divided; The method of sending each piece of data to a server of a corresponding group for storage is to send the original data to a server group with the highest network port transmission speed, and then send each piece of data to a corresponding server group via the server group with the highest network port transmission speed, and send the fragmented files in the original data to a server group with the lowest network port transmission speed, and first store them through the server in the group with the best health status; After the server group with the highest network port transmission speed sends each segment of data to the server of the corresponding group, the server with the highest network port transmission speed transfers the data in other groups to the server group with the same network port transmission speed at every preset time interval.

2. The efficient data transmission method according to claim 1, characterized in that: The hash data is some random disordered data added to the original data, and the hash data is used to make the summary value of the encrypted data the same as the summary value of the original data.

3. The efficient data transmission method according to claim 2, characterized in that: The method for determining whether the encrypted data is normal after obtaining the encrypted data includes: After obtaining the encrypted data, first obtain the summary value of the entire encrypted data, then obtain the summary value of the original data recorded in the original data check value, and then obtain the summary value of the original data in the encrypted data. If the summary values ​​of the three are the same, it is determined that the original data is normal.

4. The efficient data transmission method according to claim 3, characterized in that: When it is necessary to retrieve data stored in the server group, the data is retrieved by verifying the first key corresponding to the position sequence of each piece of data in the original data and the preset second key at the same time.

5. A data transmission system using the efficient data transmission method according to any one of claims 1 to 4, characterized in that: include: The host computer is configured to encrypt the data, determine whether the data is normal after obtaining the encrypted data, and then distribute the encrypted data to the corresponding server.

6. A computer device comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the efficient data transmission method of claim 1.

7. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the efficient data transmission method according to claim 1 are implemented.

8. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the efficient data transmission method according to claim 1 are implemented.

Images