Priority slice file transmission method and device based on information entropy

Abstract

The application relates to a priority slice file transmission method and device based on information entropy. The method divides a target file into continuous data blocks at a sending end; based on information entropy of the data blocks, the target file is split to obtain multiple file slices with variable sizes; based on information entropy sizes of the file slices, the file slices are marked with corresponding transmission priorities; and based on the transmission priorities, the corresponding file slices are sent to a receiving end in sequence, so that the receiving end recovers the target file based on the received file slices, the importance of data can be identified based on information entropy, data slices are adaptively divided, and transmission strategies are dynamically adjusted accordingly, so that the overall transmission efficiency and reliability are improved, and the problems of poor file transmission performance and stability are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of file transmission, in particular to a priority slice file transmission method and device based on information entropy. BACKGROUND

[0002] In the modern information society, file transmission, as an important part of data communication, is widely used in cloud computing, big data transmission, remote backup, multimedia streaming service and other fields. With the explosive growth of data volume and the diversification of network environment, how to efficiently and reliably transmit files has become a problem to be solved.

[0003] Traditional file transmission methods usually adopt a fixed-size segmentation strategy, dividing files into equal-length segments for transmission. Although this method is simple to implement and easy to manage, it often fails to effectively identify the importance differences of data when faced with complex and diverse content files, resulting in low transmission efficiency, especially in network bandwidth limited scenarios, with poor user experience.

[0004] To address the problem of poor file transmission performance and stability in related technologies, no effective solutions have been proposed so far. SUMMARY

[0005] A priority slice file transmission method and device based on information entropy are provided in the present embodiment to solve the problem of poor file transmission performance and stability in related technologies.

[0006] In a first aspect, a priority slice file transmission method based on information entropy is provided in the present embodiment, applied to a sending end, and the method comprises:

[0007] dividing a target file into continuous data blocks;

[0008] based on the information entropy of the data blocks, slicing the target file to obtain a plurality of file slices with variable sizes;

[0009] based on the information entropy size of the file slices, marking the transmission priority of the file slices;

[0010] based on the transmission priority, sending the corresponding file slices to a receiving end in sequence, so that the receiving end recovers the target file based on the received file slices.

[0011] In some embodiments, based on the information entropy of the data blocks, the target file is sliced to form file slices with variable sizes, comprising:

[0012] for each data block, based on the frequency of each byte appearing in the data block, calculating the information entropy of the data block;

[0013] based on the information entropy, integrating the data blocks to form file slices of variable sizes.

[0014] In some embodiments, based on the information entropy, integrating the data blocks to form file slices of variable sizes comprises:

[0015] traversing all the data blocks;

[0016] if the information entropy of the current data block is greater than or equal to a preset first threshold, defining the current data block as a high-entropy slice alone;

[0017] if the information entropy of the current data block is less than the first threshold, merging the current data block and a plurality of data blocks following the current data block, whose information entropy is less than the first threshold, into a low-to-medium-entropy slice within a preset slice size range.

[0018] In some embodiments, before sending the file slices to a receiving end in order based on the transmission priorities, the method further comprises:

[0019] in response to a high-entropy slice retransmission request sent by the receiving end, inserting the high-entropy slice with transmission exception into the first position of the sending order to be sent to the receiving end preferentially.

[0020] In some embodiments, sending the file slices to a receiving end in order based on the transmission priorities comprises:

[0021] obtaining network condition information in real time;

[0022] based on the network condition information, the transmission priorities and the sizes of the file slices, dynamically updating the sending order of the file slices;

[0023] sending the file slices to the receiving end in order based on the sending order.

[0024] In a second aspect, the present embodiment provides a priority slice file transmission device based on information entropy, applied to a sending end, comprising:

[0025] a block dividing module configured to divide a target file into continuous data blocks;

[0026] a slicing module configured to slice the target file based on the information entropy of the data blocks to form file slices of variable sizes;

[0027] a marking module configured to mark the file slices with corresponding transmission priorities based on the sizes of the information entropy of the file slices;

[0028] The sending module is configured to send the file slices to the receiving end in sequence based on the transmission priorities, so that the receiving end recovers the target file based on the received file slices.

[0029] In a third aspect, a priority slice file transmission method based on information entropy is provided in the embodiment, and is applied to a receiving end. The method comprises:

[0030] receiving file slices transmitted by a sending end; the file slices are formed by continuous data blocks divided from a target file; and transmission priorities of the file slices are determined based on information entropy of the file slices;

[0031] sequentially splicing the file slices based on the transmission priorities from high to low; wherein splicing positions of the file slices are determined based on original positions of the file slices in the target file;

[0032] updating output data in real time based on the spliced file slices.

[0033] In some embodiments, splicing the received file slices comprises:

[0034] extracting a first hash value from the file slices; the first hash value is calculated by the sending end based on data content to be transmitted;

[0035] calculating a second hash value based on data content of the file slices;

[0036] judging whether the file slices are complete based on a comparison result of the first hash value and the second hash value;

[0037] splicing the received complete file slices.

[0038] In some embodiments, the method further comprises:

[0039] in response to a file slice transmission exception, sending a retransmission request to the sending end, and recovering partial lost data based on redundant information in the file slices adjacent to the lost file slices.

[0040] In a fourth aspect, a priority slice file transmission device based on information entropy is provided in the embodiment, and is applied to a receiving end. The device comprises:

[0041] a receiving module configured to receive file slices transmitted by a sending end; the file slices are formed by continuous data blocks divided from a target file; and transmission priorities of the file slices are determined based on information entropy of the file slices;

[0042] The recombination module is configured to sequentially splice the file slices based on the transmission priority from high to low; wherein the splicing position of the file slices is determined based on the original position of the file slices in the target file;

[0043] The output module is configured to update the output data in real time based on the spliced file slices.

[0044] Compared with the related art, the information entropy-based priority file slicing transmission method and device provided in the embodiment can divide the target file into continuous data blocks at the sending end; based on the information entropy of the data blocks, the target file is divided into multiple file slices with variable sizes; based on the information entropy size of the file slices, the transmission priority of the file slices is labeled; based on the transmission priority, the corresponding file slices are sent to the receiving end in sequence, so that the receiving end recovers the target file based on the received file slices, solves the problem of poor file transmission performance and stability, can identify the importance of data based on information entropy, adaptively divide data slices, and dynamically adjust the transmission strategy accordingly, thereby improving the overall transmission efficiency and reliability.

[0045] The details of one or more embodiments of the present application are presented in the following drawings and description to make other features, objects and advantages of the present application more apparent. BRIEF DESCRIPTION OF DRAWINGS

[0046] The drawings described herein are intended to provide further understanding of the present application, form a part of the present application, and are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0047] Figure 1 The hardware structure block diagram of the terminal for the information entropy-based priority file slicing transmission method in the embodiments of the present application;

[0048] Figure 2 The flowchart of the information entropy-based priority file slicing transmission method applied to the sending end in the embodiments of the present application;

[0049] Figure 3 The flowchart of the information entropy-based priority file slicing transmission method applied to the receiving end in the embodiments of the present application;

[0050] Figure 4 The flowchart of the information entropy-based priority file slicing transmission method in the preferred embodiments of the present application;

[0051] Figure 5 The flowchart of the entropy analysis in the preferred embodiments of the present application;

[0052] Figure 6 This is a structural block diagram of the priority slice file transfer device based on information entropy applied to the sending end in the embodiments of this application;

[0053] Figure 7 This is a structural block diagram of a priority slice file transfer device based on information entropy applied to the receiving end in an embodiment of this application.

[0054] Reference numerals: 102, processor; 104, memory; 106, transmission device; 108, input / output device; 61, block module; 62, fragment module; 63, tag module; 64, transmitting module; 71, receiving module; 72, reassembly module; 73, output module. Detailed Implementation

[0055] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0056] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.

[0057] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal for the priority-slicing file transfer method based on information entropy in this embodiment. For example... Figure 1As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0058] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the information entropy-based priority slice file transfer method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, thereby implementing the above-described method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0059] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0060] This embodiment provides a priority-based slice file transfer method based on information entropy. Figure 2 This is a flowchart of the priority-slice file transfer method based on information entropy in this embodiment, as shown below. Figure 2 As shown, this process is applied to the sending end and includes the following steps:

[0061] Step S210: Divide the target file into consecutive data blocks.

[0062] Specifically, the data block is a continuous byte sequence of any fixed size, and the data block is taken as a logical analysis unit to analyze the information complexity of the target file, and the information entropy of each data block is calculated as a specific implementation manner. If the information entropy is large, it indicates that the information complexity contained in the data block is high, and if the information entropy is small, it indicates that the information complexity contained in the data block is low.

[0063] In step S220, the target file is segmented based on the information entropy of the data block to obtain a plurality of file slices of variable sizes.

[0064] Specifically, the file slice is a transmission unit segmented during actual file transmission, and is composed of data blocks. The specific implementation manner is to combine or split the data blocks to form corresponding file slices, and the data blocks with the same or similar information entropy are selected during combination. The size of the file slice can be set according to specific transmission scenarios and task requirements, such as 100 KB or 500 KB. In addition, when the target file is segmented, the information of each file slice is recorded to ensure that the data is identifiable, reconfigurable, retransmittable and verifiable. The information of the file slice includes slice ID, slice index and original offset. The slice index represents the serial number of the slice, and the original offset represents the starting offset address of the slice in the original target file.

[0065] In step S230, the transmission priority of the file slice is marked based on the information entropy size of the file slice.

[0066] Specifically, based on the data blocks contained in the file slice, the information entropy size of the file slice can be determined, thereby reflecting the complexity of the file slice. The larger the information entropy of the file slice, the higher the complexity of the file slice, and the more key information it contains. The smaller the information entropy of the file slice, the lower the complexity of the file slice, and the less key information it contains. For example, in video data, the file slice is usually static frame data. Therefore, during transmission, a higher transmission priority is marked for the file slice with high information entropy.

[0067] In step S240, the corresponding file slice is sent to the receiving end in sequence based on the transmission priority, so that the receiving end recovers the target file based on the received file slice.

[0068] Specifically, the file slices can be sent in sequence according to the priority order, that is, the higher the priority, the earlier the sending. During streaming transmission, the transmission order can also be adjusted based on the current network state, considering the priority and file slice size, and the file slice with high information entropy and small size is preferentially sent.

[0069] In the embodiment, the target file is divided into continuous data blocks at the sending end; the target file is split based on information entropy of the data blocks to obtain multiple file slices with variable sizes; the transmission priority of the file slices corresponding to the labels is determined based on the information entropy of the file slices; and the corresponding file slices are sent to the receiving end in sequence based on the transmission priority, so that the target file is recovered at the receiving end based on the received file slices. The importance of data can be identified based on the information entropy, the data slices are adaptively divided, and the transmission strategy is dynamically adjusted accordingly, thereby improving the overall transmission efficiency and reliability. The embodiment is applicable to the transmission of files such as text, audio, and video. The file slice information entropy is not defined depending on the video coding structure (I / P / B frame), but is adaptively sliced and prioritized based on the information entropy, which is more flexible in application and is suitable for various scenarios to ensure reliable transmission of important data.

[0070] In some embodiments, the target file is split based on information entropy of the data blocks to form file slices with variable sizes, including:

[0071] In step S221, the information entropy of each data block is calculated based on the frequency of each byte in the data block.

[0072] Specifically, the calculation formula of the information entropy of the data block is as follows:

[0073] ;

[0074] wherein x i represents a possible byte value (0~255); P(x i ) represents the frequency (number of occurrences / total number of bytes) of byte x i in the data block; and H(X) represents the information entropy (bit / byte) of the data block.

[0075] If most of the bytes in a 64KB block are repeated (for example, all 0x00), the entropy value is close to 0. If the byte distribution is very average, the entropy value is close to 8, indicating that the data is complex and has poor compressibility.

[0076] In step S222, the data blocks are integrated based on the information entropy to form file slices with variable sizes.

[0077] In the embodiment, the calculation of the information entropy has low complexity, and the file slices have higher efficiency.

[0078] In some embodiments, the data blocks are integrated based on the information entropy to form file slices with variable sizes, including:

[0079] In step S222-A, all data blocks are traversed.

[0080] Step S222-B, if the information entropy of the current data block is greater than or equal to the preset first threshold, the current data block is defined as a high-entropy slice.

[0081] Step S222-C, if the information entropy of the current data block is less than the first threshold, the current data block and a plurality of data blocks following the current data block and having information entropy less than the first threshold are merged into a medium-low-entropy slice within a preset slice size range. The slice size of the medium-low-entropy slice is greater than the slice size of the high-entropy slice.

[0082] Specifically, the first threshold is set to 7, and the data block with information entropy greater than or equal to 7 is taken as a small slice (for example, 100 KB) alone. When the information entropy is less than 7, a second threshold can be further set, for example, set to 5. If the information entropy is greater than or equal to 5 and less than 7, the data blocks are merged into a medium-entropy slice with a first slice size (for example, 300 KB) as a limit. If the information entropy is less than 5, the data blocks are merged into a low-entropy slice with a second slice size (for example, 500 KB) as a limit. Therefore, in the embodiment, in terms of data complexity: high-entropy slice > medium-entropy slice > low-entropy slice; and in terms of slice size: high-entropy slice < medium-entropy slice < low-entropy slice.

[0083] In some embodiments, the data block merging logic is as follows: a buffer is initialized, all data blocks are traversed, and if the information entropy of a data block is less than the first threshold (i.e., a low-entropy block), the low-entropy block is added to the buffer. When the total length of the buffer reaches an upper limit (for example, 500 KB), a file slice is output.

[0084] In the embodiment, for a high-entropy slice, the slice is a small block to reduce the probability of packet loss in the transmission process. For a medium-low-entropy slice, the slice can be integrated into a larger block to reduce the protocol overhead caused by small packet transmission and improve the overall bandwidth utilization.

[0085] In some embodiments, before the file slices are sent to the receiving end in sequence based on the transmission priority, the following steps are further included:

[0086] Step S222-D, in response to a high-entropy slice retransmission request sent by the receiving end, a high-entropy slice with transmission exception is inserted at the first position of the sending sequence to be sent to the receiving end preferentially.

[0087] Specifically, the high-entropy slice is marked with a high priority, and the receiving end can judge that the lost file slice is a high-entropy slice according to the priority. In the transmission process, if interruption or packet loss occurs, the receiving end will automatically call the fault tolerance and recovery module to re-request the missing high-priority slice to ensure the complete transmission of important data.

[0088] In some embodiments, the file slices are sent to the receiving end in sequence based on the transmission priority, including:

[0089] Step S241, real-time network condition information is acquired.

[0090] Step S242, based on the network condition information, the transmission priority, and the size of the file slice, the sending order of the file slice is dynamically updated.

[0091] Step S243, based on the sending order, the file slice is sent to the receiving end in sequence.

[0092] Specifically, the slice is transmitted block by block using streaming technology (such as HTTP / 2 or WebSocket protocol). During transmission, the transmission order of the slice is dynamically adjusted according to the real-time network condition, ensuring that the slice with high priority and small data volume is transmitted first, thereby improving the overall transmission efficiency and user experience. Further, the transmission order of the slice is dynamically adjusted according to the real-time network condition and the feedback from the receiving end.

[0093] In this embodiment, a priority slice file transmission method based on information entropy is provided, Figure 3 is a flowchart of the priority slice file transmission method based on information entropy of this embodiment, as Figure 3 shown, the flow is applied to the receiving end and includes the following flow steps:

[0094] Step S310, receiving the file slice transmitted by the sending end; the file slice is formed by continuous data blocks divided from the target file; the transmission priority of the file slice is determined based on the information entropy of the file slice.

[0095] Specifically, the sending end can generate and send the file slice based on any of the above priority slice file transmission methods based on information entropy applied to the sending end. The received file slice is marked with a transmission priority, and the receiving end preferentially receives and processes the file slice with a higher transmission priority. Among them, the low transmission priority slice is received and processed after the high transmission priority slice is transmitted. The design of the transmission priority can be two levels, three levels, or even multiple levels of numerical priority, depending on the fineness of the system composed of the sending end and the receiving end.

[0096] Step S320, based on the order from high to low of the transmission priority, the file slices are sequentially spliced; wherein the splicing position of the file slice is determined based on the original position of the file slice in the target file.

[0097] Specifically, the receiving end checks, decodes, writes to disk / cache, and so on the received file slices in order of transmission priority from high to low, especially in real-time decoding scenarios (such as video, edge-down-edge play), the file slice marked as the highest transmission priority is prioritized for splicing and use. When splicing, for multiple file slices of the same priority, the original offset (or slice index) is sorted and reorganized, thereby ensuring that the data restored from the file slices conforms to the order of the original target file. The original offset is used to record the starting position of the slice in the original file; the slice index is obtained based on the slice sequence number.

[0098] Step S330, real-time update of output data based on the spliced file slice.

[0099] Specifically, in the scenario of video transmission, the display screen is updated in real time, and the user can see the screen update in a timely manner.

[0100] In this embodiment, the information entropy analysis technology is used to dynamically adjust the content and transmission order of the file slices, to ensure the priority of important data transmission and reception, avoid important data from causing screen freezing due to packet loss, and improve user experience.

[0101] In some embodiments, splicing the received file slices includes:

[0102] Step S321, extracting a first hash value from the file slice; the first hash value is calculated by the sending end based on the data content to be sent.

[0103] Step S322, calculating a second hash value based on the data content of the file slice.

[0104] Step S323, determining whether the file slice is complete based on the comparison result of the first hash value and the second hash value.

[0105] Specifically, if the file slice is complete, i.e., passes the integrity check, the sending end returns a transmission success result, and the subsequent step S324 is continued; if the file slice is not complete, i.e., the integrity check fails, a fault recovery mechanism is triggered, the sending end is requested to retransmit, and fault information is recorded.

[0106] Step S324, splicing the received complete file slice.

[0107] In this embodiment, in order to ensure the accuracy and integrity of the data, the received slice is verified for integrity using a hash check code (SHA-256), thereby improving the reliability of file transmission.

[0108] In some embodiments, the method further includes:

[0109] Step S340, in response to the file slice transmission exception, a retransmission request is sent to the sending end, and part of the lost data is recovered based on the redundant information in the file slices adjacent to the lost file slices.

[0110] Specifically, the file slice transmission exception includes integrity check failure, at which time the fault recovery mechanism is triggered, under which the receiving end automatically requests retransmission of the file slices, while using redundant information to recover part of the lost data, avoiding retransmission of the entire file.

[0111] In this embodiment, the efficiency and integrity of transmission are ensured, and the reliable arrival of important data is guaranteed even in poor network conditions.

[0112] The preferred embodiment will be described and explained below with the transmission task of a video file as an example. The video file to be transmitted (i.e. the target file) contains multiple high-frequency and low-frequency picture contents, and the Figure 4 The information entropy-based priority slice file transmission method of the preferred embodiment includes the following processes:

[0113] 1. Entropy analysis and slice generation: the sending end responds to the target file transmission request initiated by the user to divide the target file into continuous data blocks; the information entropy of each data block is calculated. Based on the information entropy, entropy analysis is started, see Figure 5 According to the information entropy of the data block, it is determined whether the data block belongs to a high-entropy region or a low-entropy region, so as to identify the key frames and static frames in the video file; the key frames are cut into 100KB file slices (high-entropy slices), and multiple static frames are integrated into 500KB file slices (low-entropy slices).

[0114] 2. Priority allocation: the sending end marks the transmission priority of the file slices based on the information entropy size of the file slices, including high priority and low priority; the transmission priority of the high-entropy slices is high priority, and the transmission priority of the low-entropy slices is low priority.

[0115] 3. Streaming mechanism: the WebSocket protocol is used for communication between the sending end and the receiving end. The sending end starts transmitting the file based on the order of transmission priority from high to low. If the network bandwidth is unstable during transmission, the receiving end adjusts the transmission order of the subsequent file slices according to the feedback information of the receiving end, to ensure that the high-priority file slices are always given priority.

[0116] 4. Receiving end reconstruction: After receiving the slices, the receiving end first verifies the high-priority file slices. In order to ensure the accuracy and integrity of the data, the received target slices are verified for integrity using the hash check code (SHA-256). Each file slice is attached with the corresponding hash value when sent, and the receiving end verifies by comparing the calculated hash value with the received hash value when reconstructing the data. After verification, the file slices are spliced and reorganized based on the original offset and index of the file slices, so that the user can see the screen update in time when watching the video. The low-priority file slices are processed step by step after the high-priority file slices are transmitted.

[0117] 5. Fault recovery mechanism: If some high-priority file slices are lost during transmission, the receiving end will automatically request retransmission, and at the same time, use redundant information to recover part of the lost data, avoiding retransmission of the entire file.

[0118] In the preferred embodiment, the information entropy analysis technique is used to dynamically adjust the size and content of the file slices and assign corresponding transmission priorities, ensuring that the data blocks in the high-entropy area are transmitted preferentially, significantly improving the transmission speed of large files in different network environments, reducing the risk of important data loss, and ensuring the reliable arrival of video files.

[0119] It should be noted that the steps shown in the above process or the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that described here.

[0120] In this embodiment, an information entropy-based priority slice file transmission device is also provided, which is used to implement the above embodiments and preferred embodiments, and will not be described again. The terms "module", "unit", "sub-unit" and the like used below can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware or a combination of software and hardware is also possible and contemplated.

[0121] Figure 6 is a structure block diagram of the information entropy-based priority slice file transmission device applied to the sending end of the present embodiment, as shown in Figure 6 The device includes a blocking module 61, a slicing module 62, a marking module 63, and a sending module 64.

[0122] The blocking module 61 is used to divide the target file into continuous data blocks.

[0123] The slicing module 62 is used to divide the target file based on the information entropy of the data blocks to form file slices of variable sizes.

[0124] The marking module 63 is configured to mark the transmission priority of the file slice based on the information entropy size of the file slice.

[0125] The sending module 64 is configured to send the file slices to the receiving end in sequence based on the transmission priority, so that the receiving end recovers the target file based on the received file slices.

[0126] Figure 7 FIG. 6 is a structural block diagram of an information entropy-based priority slice file transmission device applied to the receiving end according to the present embodiment. As shown in FIG. 6, the device comprises a receiving module 71, a recombination module 72, and an output module 73. Figure 7

[0127] The receiving module 71 is configured to receive the file slices transmitted by the sending end. The file slices are formed by continuous data blocks divided from the target file. The transmission priority of the file slices is determined based on the information entropy of the file slices.

[0128] The recombination module 72 is configured to sequentially splice the file slices in the order from high to low based on the transmission priority. The splicing position of the file slices is determined based on the original position of the file slices in the target file.

[0129] The output module 73 is configured to update the output data in real time based on the spliced file slices. It should be noted that each of the above modules can be a functional module or a program module, which can be implemented by software or hardware. For the modules implemented by hardware, each of the above modules can be located in the same processor, or each of the above modules can be located in different processors in any combination.

[0130] In the present embodiment, a computer device is also provided, which comprises a memory and a processor. The memory stores a computer program. The processor is configured to execute the computer program to perform the steps in any of the above method embodiments.

[0131] Optionally, the computer device can further comprise a transmission device and an input / output device. The transmission device is connected to the processor. The input / output device is connected to the processor.

[0132] It should be noted that the specific examples in the present embodiment can refer to the examples described in the above embodiments and optional implementation manners, which will not be described herein again.

[0133] ​In addition, in combination with the information entropy-based priority slice file transmission method provided in the above embodiments, a storage medium can also be provided to implement the method in the embodiment. The storage medium has a computer program stored thereon. The computer program is executed by a processor to implement any one of the information entropy-based priority slice file transmission methods in the above embodiments.

[0134] It should be understood that the specific embodiments described herein are merely exemplary and are not intended to limit the application. Based on the embodiments provided in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of the present application.

[0135] Obviously, the drawings are only some examples or embodiments of the present application, and those skilled in the art can also apply the present application to other similar situations without creative efforts. In addition, it can be understood that although the work done in the development process may be complex and long, some design, manufacture or production changes made by those skilled in the art according to the technical content disclosed in the present application are only routine technical means and should not be regarded as insufficient disclosure of the present application.

[0136] The term "embodiment" in the present application means that the specific features, structures or characteristics described in combination with the embodiments can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor does it mean independence or alternatives to other embodiments. It can be clearly or implicitly understood by those skilled in the art that the embodiments described in the present application can be combined with other embodiments without conflict.

[0137] The above-described embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of patent protection. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which all fall within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A priority-slice file transfer method based on information entropy, characterized in that, Applied to the sending end, the method includes: Divide the target file into contiguous data blocks; Based on the information entropy of the data blocks, the target file is segmented to obtain multiple file slices of variable size, including: traversing all the data blocks; if the information entropy of the current data block is greater than or equal to a preset first threshold, then the current data block is defined as a high-entropy slice; if the information entropy of the current data block is less than the first threshold, then within a preset slice size range, the current data block and multiple subsequent data blocks with information entropy less than the first threshold are merged into a medium-low entropy slice. Based on the information entropy of the file slice, the corresponding transmission priority of the file slice is marked; Based on the transmission priority, the corresponding file slices are sent to the receiving end in sequence, so that the receiving end can recover the target file based on the received file slices.

2. The priority-slice file transfer method based on information entropy according to claim 1, characterized in that, The calculation process of the information entropy of the data block includes: For each data block, the information entropy of the data block is calculated based on the frequency of each byte appearing in the data block.

3. The priority-slice file transfer method based on information entropy according to claim 1, characterized in that, Before sending the file slices to the receiving end sequentially based on the transmission priority, the process also includes: In response to the high-entropy slice retransmission request sent by the receiving end, the high-entropy slice with transmission error is inserted at the beginning of the transmission order so as to be sent to the receiving end with priority.

4. The priority-slice file transfer method based on information entropy according to claim 1, characterized in that, Based on the transmission priority, the file slices are sent to the receiving end sequentially, including: Get real-time network status information; Based on the network condition information, the transmission priority, and the size of the file slice, the sending order of the file slice is dynamically updated; The file slices are sent to the receiving end sequentially based on the sending order.

5. A priority-slice file transfer device based on information entropy, characterized in that, Applied to the transmitting end, the device includes: The chunking module is used to divide the target file into consecutive data blocks; The slicing module is used to slice the target file based on the information entropy of the data blocks to form file slices of variable size, including: traversing all the data blocks; if the information entropy of the current data block is greater than or equal to a preset first threshold, then the current data block is defined as a high-entropy slice; if the information entropy of the current data block is less than the first threshold, then within a preset slice size range, the current data block and multiple subsequent data blocks with information entropy less than the first threshold are merged into a medium-low entropy slice. The marking module is used to mark the file slice with a corresponding transmission priority based on the information entropy of the file slice; The sending module is used to send the file slices to the receiving end sequentially based on the transmission priority, so that the receiving end can recover the target file based on the received file slices.

6. A priority-slice file transfer method based on information entropy, characterized in that, Applied to the receiving end, the method includes: The system receives file slices transmitted from the sending end; the file slices are formed by dividing consecutive data blocks from a target file; the transmission priority of the file slices is determined based on the information entropy of the file slices; wherein, the process of forming the file slices from consecutive data blocks from the target file includes: traversing all the data blocks; if the information entropy of the current data block is greater than or equal to a preset first threshold, then the current data block is defined as a high-entropy slice; if the information entropy of the current data block is less than the first threshold, then within a preset slice size range, the current data block and multiple subsequent data blocks with information entropy less than the first threshold are merged into a medium-low entropy slice; The file slices are concatenated sequentially based on the transmission priority from high to low; wherein the concatenation position of the file slices is determined based on the original position of the file slices in the target file. The output data is updated in real time based on the assembled file slices.

7. The priority-slice file transfer method based on information entropy according to claim 6, characterized in that, Based on the transmission priority from high to low, the file slices are concatenated sequentially, including: The first hash value is extracted from the file slice; the first hash value is calculated by the sender based on the data content to be sent. Calculate the second hash value based on the data content of the file slice; Based on the comparison result between the first hash value and the second hash value, it is determined whether the file slice is complete; Based on the transmission priority from high to low, the complete file slices are spliced ​​together sequentially.

8. The priority-slice file transfer method based on information entropy according to claim 6, characterized in that, The method further includes: In response to a file slice transmission anomaly, a retransmission request is sent to the sending end, and some of the lost data is recovered based on the redundant information in the file slices adjacent to the lost file slice.

9. A priority-slice file transfer device based on information entropy, characterized in that, Applied to the receiving end, the device includes: A receiving module is used to receive file slices transmitted by a sending end; the file slices are formed by dividing a target file into consecutive data blocks; the transmission priority of the file slices is determined based on the information entropy of the file slices; wherein, the process of forming the file slices from consecutive data blocks divided from the target file includes: traversing all the data blocks; if the information entropy of the current data block is greater than or equal to a preset first threshold, then the current data block is defined as a high-entropy slice; if the information entropy of the current data block is less than the first threshold, then within a preset slice size range, the current data block and multiple subsequent data blocks with information entropy less than the first threshold are merged into a medium-low entropy slice; The reassembly module is used to sequentially splice the file slices according to the transmission priority from high to low; wherein the splicing position of the file slices is determined based on the original position of the file slices in the target file; The output module is used to update the output data in real time based on the spliced ​​file slices.

Images