A method implemented in a computer, a computer program, or a computer system

Abstract

To provide a method mounted in a computer, a computer program or a computer system that segment a data structure into a plurality of chunks, on the basis of identifying a sequence of bits within the respective chunks that are unique within the data structure.SOLUTION: In an approach 100, a processor segments a data structure into a plurality of initial chunks. The processor, for each of the plurality of initial chunks, determines a chunk key, on the basis of identifying a sequence of bits within the respective chunks that are unique within the data structure. The processor re-segments the data structure into a plurality of final chunks, wherein a plurality of divisions correspond to a plurality of positions of the identified sequences of bits.SELECTED DRAWING: Figure 2

Description

[Background technology]

[0001] The present invention relates to the field of data structures, and more particularly to storing data structures, and more particularly to segmenting data structures into multiple chunks based on identifying a bit sequence within each chunk that is unique within the data structure.

[0002] Many applications are used to edit very large data structures exceeding several hundred megabytes. Such data structures are handled by JavaScript. (Registered trademark) Object annotations (JSON: JavaScript) (Registered trademark) This can be an Object Notation (Object Notation) file, a Rich Text Format (RTF) file, or code. For example, modern web-based editor applications are used to edit very large data structures. Consequently, saving and reloading data structures during the editing process by these applications typically proves to be a time-consuming and computationally expensive process. [Overview of the project] [Problems that the invention aims to solve]

[0003] The present invention aims to provide a method, computer program, or computer system implemented on a computer. [Means for solving the problem]

[0004] According to one embodiment of the present invention, a computer-implemented method, a computer program product or computer program, and a computer system are provided. A processor segments a data structure into a plurality of initial chunks. For each of the plurality of initial chunks, the processor determines a chunk key based on identifying a bit sequence within each chunk that is unique within the data structure. The processor resegments the data structure into a plurality of final chunks, where the plurality of divisions correspond to a plurality of positions in the identified bit sequence.

[0005] According to one embodiment of the present invention, a method implemented on a computer, Segmenting a data structure into multiple initial chunks; For each of the multiple initial chunks, determine the chunk key based on identifying the bit sequence within each chunk that is unique within the data structure; and, Re-segmenting the data structure into multiple final chunks, where the multiple segments correspond to multiple positions in the identified bit sequence. A method including the above is provided.

[0006] According to another embodiment of the present invention, A computer program product, It comprises one or more computer-readable storage media, and program instructions collectively stored in the one or more computer-readable storage media, wherein the program instructions are Program instructions for segmenting a data structure into multiple initial chunks; A program instruction for determining a chunk key for each of the multiple initial chunks based on identifying a unique bit sequence within each chunk within the data structure; and Instructions for resegmenting the data structure into multiple final chunks, where the multiple segments correspond to multiple positions in the identified bit sequence. A computer program product including the above is provided.

[0007] According to another embodiment of the present invention, a computer system, It comprises one or more computer processors, one or more computer-readable storage media, and program instructions collectively stored in at least one of the one or more storage media for execution by at least one of the one or more computer processors, and the program instructions are Program instructions for segmenting a data structure into multiple initial chunks; A program instruction for determining a chunk key for each of the multiple initial chunks based on identifying a unique bit sequence within each chunk within the data structure; and Instructions for resegmenting the data structure into multiple final chunks, where the multiple segments correspond to multiple positions in the identified bit sequence. A computer system including the above is provided.

[0008] The accompanying drawings will be referenced only as examples, in order to better understand the present invention and to more clearly illustrate how it can be carried out. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 is a block diagram of an exemplary system in which the viewpoints of an exemplary embodiment may be implemented. [Figure 2] Figure 2 is a flowchart of an approach for storing a data structure according to one embodiment. [Figure 3A] Figure 3A is a flowchart of an approach for further processing the final chunk, following one aspect of one embodiment. [Figure 3B] Figure 3B is a flowchart of an approach for further processing the final chunk, following one aspect of one embodiment. [Figure 4]FIG. 4 is a flowchart of an approach for updating a data structure according to one perspective of one embodiment. [Figure 5] FIG. 5 is a flowchart of an approach for loading a data structure according to another embodiment. [Figure 6] FIG. 6 is a block diagram of one embodiment of a system for storing a data structure. [Figure 7] FIG. 7 is a simplified block diagram of a computer system where an approach for entering data into a data input field can be executed according to one embodiment. **DETAILED DESCRIPTION**

[0010] Even if known compression techniques are utilized, a large amount of data still needs to be transmitted and received because the data structure is opened for editing and the edits to the data structure are saved. Thus, there is a need for an approach for storage and loading that shows a smaller amount of data flow.

[0011] One way to reduce the amount of data could be to transmit a record of the changes to the data structure when the edited data structure is saved. Thus, only that record of the changes is sent to memory for storage, where the record of the changes can be used to update and save the fully up-to-date data structure. However, embodiments of the present invention recognize that while such an approach improves the performance for saving the data structure, it does not improve the performance for opening the data structure where the full data structure needs to be retrieved by an application.

[0012] In fact, multiple users may locally save the data structure. If one user edits the data structure and subsequently updates the data structure (perhaps using the record of the changes described above) in a common storage location, other users need to load the entire data structure from the common storage to obtain the latest data structure.

[0013] The present invention will be described below with reference to the drawings.

[0014] The detailed description of the invention and specific examples show exemplary embodiments of the apparatus, system and method, but are for illustrative purposes only and are not intended to limit the scope of the present invention. It should be understood that these and other features, aspects and advantages of the apparatus, system and method of the present invention will be better understood from the following detailed description of the invention, the appended claims and the accompanying drawings. It should be understood that the drawings are only schematic and are not drawn to scale. It should also be understood that the same reference numbers are used throughout the figures to indicate the same or similar parts.

[0015] Modifications to the disclosed embodiments will be understood and can be implemented by those skilled in the art in practicing the claimed invention by considering the drawings, the disclosure and the appended claims. In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. Note that when the term "configured to" is used in the claims or the detailed description of the invention, the term "configured to" is intended to be equivalent to the term "composed of".

[0016] In the context of the present application, when an embodiment of the present invention constitutes a method, it should be understood that such a method can be a process for execution by a computer, that is, a method that can be implemented on a computer. Therefore, the various steps of the method can reflect the various parts of a computer program, for example, the various parts of one or more algorithms.

[0017] Furthermore, in the context of this application, a system may be a single device or a collection of distributed devices adapted to perform one or more embodiments of the method of the present invention. For example, a system may be a personal computer (PC), a server, or a collection of PCs or servers connected via a network, such as a local area network or the Internet, which cooperate to perform at least one embodiment of the method of the present invention.

[0018] Implementations relating to this disclosure relate to various techniques, methods, schemes, or solutions, or combinations thereof, for storing data structures. In particular, the data structure is segmented to produce a final chunk of data with corresponding chunk keys. Multiple segments of the final chunk are selected / configured to correspond to multiple positions of bit sequences (unique within each of the data structure of the multiple initial chunks), where the chunk keys are based on the identified bit sequences.

[0019] By segmenting the data structure into smaller final chunks, when loading the data structure, only the final chunk that is not stored locally (e.g., in the cache) needs to be retrieved. Consequently, the amount of data that needs to be fetched is reduced, improving performance in terms of both computational complexity and time consumption.

[0020] However, simply segmenting a data structure into chunks of a fixed size often leads to problems. In particular, when bits are added or removed from the data structure, not only the chunk where the bits were added or removed, but all subsequent chunks also need to be updated. Therefore, every time the data structure is edited (even in a small way), many chunks will need to be updated. This can be especially problematic in situations where many clients store chunks locally for use, as each client needs to reload many chunks to obtain the latest data structure.

[0021] Embodiments of the present invention recognize that the final chunks and chunk keys generated by the present invention can overcome the above-mentioned problems. In particular, the data structure is first segmented into a plurality of initial chunks. Next, each initial chunk is analyzed to identify a unique bit sequence. Then, the unique bit sequence identified within each initial chunk is used as the basis for the chunk key corresponding to each of the plurality of initial chunks. Subsequently, the data structure is re-segmented at positions corresponding to multiple positions of the unique bit sequence. As a result, an array of chunk keys identifying the corresponding final chunk is obtained.

[0022] Therefore, when bits are added to or removed from the data structure during editing, only the final chunks corresponding to those added or removed bits are updated. This is because the chunk key corresponds to the beginning of each final chunk, rather than the final chunk of a specified length.

[0023] In other words, embodiments of the present invention provide a way to segment a data structure into multiple final chunks, which allows for a change "delta" applied to the final chunks, minimizing the number of final chunks to modify. Thus, locally cached final chunks can then be used to reduce the burden of loading the data structure (for example, by avoiding the need to use the entire updated data structure over a network).

[0024] Further embodiments of the proposed invention provide a computer-implemented method for loading a data structure. The method includes retrieving a plurality of final chunks and corresponding chunk keys associated with the data structure, where the plurality of divisions of the final chunks correspond to a plurality of positions of a bit sequence unique within the data structure, and the chunk keys are based on the bit sequence; comparing the chunk keys with a plurality of locally stored chunks to identify chunk keys that do not have locally stored associated chunks; and loading the final chunks associated with the identified chunk keys.

[0025] In this way, since only the final chunk that is not stored locally needs to be loaded, the burden of downloading many final chunks, or perhaps even the entire data structure, is reduced. This is made possible by segmenting the data into final chunks corresponding to chunk keys, where each final chunk is divided at positions corresponding to multiple locations of bit sequences that are unique within the data structure.

[0026] In some embodiments, the method for loading the data structure may further include combining the multiple final chunks to obtain the data structure. In this way, the most recent data structure can be obtained.

[0027] Turning to other embodiments of the proposed one or more concepts for storing the data structure, identifying the bit sequence may include sequentially scanning the data structure from the first bit of the first chunk to the last bit of the first chunk to identify the bit sequence within the chunk that is unique within the data structure and closest to the first bit.

[0028] By scanning the first chunk from beginning to end, a unique bit sequence can be identified as close as possible to the first bit. In this way, the divisions of the data structure that generate the final chunk can be made as close as possible to the size of the initial chunk. This can prove particularly advantageous when chunks of a given size are desired. In addition, scanning the first chunk from the first bit to the last bit makes it possible to efficiently and reliably identify the unique bit sequence within the data structure.

[0029] In another embodiment, a method for storing the data structure may further include receiving a record of changes, including changes made to the data structure; combining the multiple final chunks to obtain the data structure; updating the data structure based on the record of changes; and segmenting the updated data structure into a plurality of updated final chunks, where the plurality of segments correspond to the plurality of positions in the identified bit sequence.

[0030] Because only a record of changes is received, rather than a completely updated data structure, the burden of data transmission is reduced. The data contained in the record of changes may be far less than the updated data structure, especially if the changes made are relatively small compared to the size of the data structure. The updated data structure can then be obtained in a simple way by combining the record of changes with the original data structure. The already generated chunk keys can be reused to deliver the final chunk, reducing the overall processing load required to segment the updated data structure.

[0031] In some embodiments, the method for storing the data structure may further include determining, for each of the plurality of final chunks, whether the size of the final chunk is greater than a predetermined upper threshold; and segmenting the final chunk in response to determining that the size of the final chunk is greater than the predetermined upper threshold.

[0032] By effectively setting upper thresholds for the size of each of the multiple final chunks, chunks that are so large that the benefits of segmentation are diminished can be avoided. This can occur, for example, when the identified (unique) bit sequence for the first chunk is near the first bit of the chunk, and the identified (unique) bit sequence for the second consecutive chunk is near the last bit. As a result, the multiple segments of the final chunk may be such that the final chunk is very large and therefore requires further segmentation.

[0033] In a further embodiment, the method for storing the data structure may further include determining, for each of the plurality of final chunks, whether the size of the final chunk is less than a predetermined lower threshold; and, in response to determining that the size of the final chunk is less than the predetermined lower threshold, combining the final chunk with adjacent final chunks.

[0034] Similar to setting an upper threshold, setting a lower threshold can prevent chunks that are too small to reduce the benefits of segmentation. Combining adjacent chunks can result in chunks of a more appropriate size.

[0035] In an additional embodiment, the approach to storing the data structure may further include, for each of the plurality of final chunks, computing a hash of the data of the final chunk, and appending the hash and separator to the chunk key corresponding to the final chunk.

[0036] Therefore, the hash can identify the final chunk to which the chunk key corresponds.

[0037] In some embodiments, the approach to storing the data structure further includes compressing each of the plurality of final chunks, and storing the plurality of compressed final chunks and the plurality of associated chunk keys on a server.

[0038] By compressing the data structure and segmenting it into multiple chunks, more efficient data storage and transfer can be achieved.

[0039] In a further embodiment, the data structure is JavaScript (Registered trademark) This can be an object annotation (JSON) data structure, a rich text format (RTF) file, or a code file.

[0040] JSON data structures tend to be large and problematic to transfer, especially when there is a lot of saving, loading, and reloading required. While JSON data structures are easy to understand and use, they are often space-inefficient because they require many bytes to represent even small objects. Therefore, efficient storage and transfer of JSON data structures can be achieved by implementing a method for storing them by segmenting the JSON data structure.

[0041] In fact, this method can be extended to any uncompressed data structure. For example, RT files or code files may benefit from being stored using the method described above.

[0042] To elaborate further, given a very large data structure (e.g., a string 100MB in length), it may be advantageous to divide the string into manageable chunks (e.g., each 10MB). A simple solution might be to simply divide the 100MB string into 10MB chunks. However, this means that inserting or deleting any number of bits in the 100MB string will result in changes in all chunks from those bits onward. As a result, it is possible to produce chunks close to the desired length (e.g., 10MB), but at the same time, a chunking mechanism that minimizes the number of chunks that change as a result of insertions or deletions is desirable.

[0043] According to embodiments of the present invention, this segmentation can be achieved by the following steps: (i) Dividing the data structure into multiple initial chunks (of normal size); (ii) For each initial chunk, scan from left to right to find a unique bit sequence (e.g., a unique 20-byte sequence) that does not exist in any previous initial chunk; (iii) Adding each of the identified unique bit sequences to the array of chunk keys; (iv) Divide the initial data structure again at positions corresponding to multiple positions of the identified chunk key to obtain multiple final chunks; (v) If any of the resulting final chunks are smaller than the minimum size of the threshold, add them to the previous final chunk and discard the corresponding chunk key; (vi) If any of the resulting final chunks are larger than the maximum size of the threshold, divide the final chunk using the same mechanism as in steps (i) to (iv); and (vii) For each chunk key, add a separator and a hash of the chunk data of the final chunk corresponding to that chunk key. For example, MD5 or a secure hash algorithm (SHA) hash may be used.

[0044] The result of the above segmentation process is an array of chunk keys and the corresponding final chunks of data. These final data chunks can be cached locally in the browser using the chunk hash as the cache key. In fact, modern web development applications often have support for asynchronous storage that can be used to store locally cached data chunks.

[0045] By segmenting the data structure in the format described above, the final chunk and chunk key can be used in the format described below.

[0046] When the data is first saved, all final chunks are compressed and sent to the server along with their corresponding chunk keys. The server may store the final chunk in a database associated with it. The server may also store the chunk keys. Subsequently, when loading the data, the client may retrieve the chunk keys from the server. For each final chunk, the client first checks its local cache (and uses it if available). If the final chunk is not available in the local cache, the client may then retrieve it from the server and add it to the cache. The client may then decompress the final chunks and combine them to obtain the complete data structure.

[0047] Furthermore, when saving edited content to the data structure, the client can calculate a record of changes between the initial data structure and the updated version of the data structure, and then send this record of changes to the server. The server can then reconstruct the initial data structure by putting together the saved final chunks, and then apply the record of changes to obtain the updated data structure. The server can then segment the updated data structure using existing chunk keys and save the resulting updated final chunks. In other words, the server can segment the updated data structure according to steps (iv) to (vii) described above.

[0048] Therefore, this results in a set of multiple updated final chunks, some of which will match among the previous final chunks (depending on the scope of the changes). This means the client only needs to load the updated final chunk, which is the same as the unchanged final chunk, from the server. For example, inserting at the beginning of the data structure would only affect the first final chunk, while all other final chunks remain unchanged. Ultimately, this reduces the amount of data transfer required to retrieve the latest data structure, both when saving to the server and when loading by the client.

[0049] Now, looking at Figure 1, a block diagram of an exemplary system 200 is presented, in which the aspects of an exemplary embodiment may be implemented. System 200 is an example of a computer, for example, a client in a distributed processing system, in which computer-readable code or instructions that implement the processes of an exemplary embodiment of the present invention may be located. For example, system 200 may be configured to implement a data manipulation unit and a chunk key generation unit according to one embodiment.

[0050] In the illustrated example, system 200 uses a hub architecture, for example, a hub architecture that includes a northbridge and memory controller hub (NB / MCH) 202 and a southbridge and input / output (I / O) controller hub (SB / ICH) 204. Processing units 206, main memory 208, and graphics processor 210 are connected to the NB / MCH 202. The graphics processor 210 may be connected to the NB / MCH 202 through an accelerated graphics port (AGP).

[0051] In the illustrated example, the local area network (LAN) adapter 212 connects to the SB / ICH 204. The audio adapter 216, keyboard and mouse adapter 220, modem 222, read-only memory (ROM) 224, disks, such as a hard disk drive (HDD) 226, CD-ROM drive 230, universal serial bus (USB) ports and other communication ports 232, and PCI / PCIe devices 234 connect to the SB / ICH 204 via the first bus 238 and the second bus 240. PCI / PCIe devices include, for example, Ethernet adapters, add-in cards, and PC cards for notebook computers. PCI uses a cardbus controller, while PCIe does not. ROM 224 could be, for example, a basic input / output system (BIOS).

[0052] The HDD 226 and CD-ROM drive 230 are connected to the SB / ICH 204 via a second bus 240. The HDD 226 and CD-ROM drive 230 may use, for example, an Integrated Drive Electronics (IDE) or Serial Advanced Technology Attachment (SATA) interface. A Super I / O (SIO) device 236 may be connected to the SB / ICH 204.

[0053] The operating system runs in processing unit 206. This operating system coordinates and provides control over various components within system 200 in Figure 2. As a client, the operating system can be a commercially available operating system, such as an object-oriented programming system, e.g., Java. (登録商標) The programming system may run in conjunction with the operating system and may also run on system 200, and may be Java. (登録商標) Provides calls from a program or application to the operating system. Java (登録商標) and all Java (登録商標) The base trademarks and logos are trademarks or registered trademarks of Oracle and / or its affiliates.

[0054] As a server, System 200 is, for example, Advanced Interactive Executive (AIX). (登録商標) ) Operating system or Linux (登録商標) IBM, which runs the operating system (登録商標) eServer (商標)The system p can be a computer system. The system 200 can be a symmetric multiprocessor (SMP) system that includes multiple processors in the processing unit 206. Alternatively, a single-processor system can be used. IBM (登録商標) , eServer (商標) and AIX (商標) are trademarks of International Business Machines Corporation in the United States and / or other countries. Registered trademark Linux (登録商標) is used under sublicense from the Linux (Registered Trademark) Foundation, which is an exclusive licensee of Linus Torvalds, the owner of the trademark worldwide. (登録商標) Foundation)

[0055] The operating system, programming system, and application or program instructions can be located in a storage device, such as HDD 226, and loaded into the main memory 208 for execution by the processing unit 206. Similarly, one or more message processing programs according to one embodiment can be adapted to be stored by the storage device or the main memory 208 or a combination thereof.

[0056] The process of an exemplary embodiment of the present invention is executed by the processing unit 206 using program code that can be used by a computer, and the program code can be located in a memory, such as the main memory 208, ROM 224, or in one or more peripheral devices 226 and 230.

[0057] A bus system, for example, the first bus 238 or the second bus 240 shown in Figure 1, may comprise one or more buses. Of course, the bus system may be implemented using any type of communication fabric or architecture that provides data transfer between different components or devices connected to the fabric or architecture. A communication unit, for example, the modem 222 or network adapter 212 in Figure 1, may comprise one or more devices used to send and receive data. Memory may be, for example, main memory 208, ROM 224, or cache, for example, the cache seen in NB / MCH 202 in Figure 1.

[0058] Those skilled in the art will understand that the hardware in Figure 1 may vary depending on the implementation. Other internal hardware or peripheral devices, such as flash memory, equivalent non-volatile memory, or optical disc drives, may be used in addition to or instead of the hardware shown in Figure 1. Furthermore, the processes of the exemplary embodiments may be applied to multiprocessor data processing systems other than those described above without departing from the spirit and scope of the invention.

[0059] Furthermore, System 200 can take the form of any of several different data processing systems, such as a client computing device, a server computing device, a tablet computer, a laptop computer, a telephone or other communication device, a personal digital assistant (PDA), etc. In some exemplary embodiments, System 200 may be a portable computing device configured with flash memory to provide non-volatile memory for storing operating system files or user-generated data or a combination thereof. Thus, System 200 can essentially be any known or later developed data processing system without architectural limitations.

[0060] Referring now to Figure 2, a flowchart 100 of an approach for storing a data structure according to an exemplary embodiment is shown.

[0061] In step 102, the processor segments the data structure into a plurality of initial chunks. This segmentation may be performed at regular intervals so that the data structure is divided into regular chunks (e.g., 10MB), or it may be selected individually. The size of the chunks may be set to a predetermined value, or it may be selected by the user.

[0062] The data structure can be any data structure such that inserting or deleting data does not alter the data structure before or after the insertion / deletion. For example, the data structure could be an uncompressed JSON data structure, a code file, or a text file. However, the data structure may not be a .doc file.

[0063] In step 104, for each of the multiple initial chunks, the processor identifies a bit sequence within the chunk that is unique within the data structure. The bit sequence may be set to a predetermined length (e.g., 20 bytes).

[0064] Identifying the bit sequence involves sequentially scanning the data structure from the first bit of the initial chunk to the last bit of the initial chunk to identify the bit sequence within the chunk that is unique within the data structure and closest to the first bit. However, alternative methods for searching for a (unique) sequence of bits may be used. The bit sequence may simply be a sequence of bits that does not exist in any previous chunk.

[0065] In step 106, for each of the plurality of initial chunks, the processor determines a chunk key based on the identified bit sequence within that initial chunk.

[0066] In this way, the chunk key can be used to uniquely identify a chunk of the data structure. The chunk key may simply be the identified bit sequence, or it may be another representation of the identified bit sequence, for example, a compressed version of the identified bit sequence. Finally, the chunk key may be stored in an array of chunks.

[0067] In step 108, the processor resegments the data structure into multiple final chunks, where each chunk corresponds to a position in the identified bit sequence.

[0068] In other words, multiple final chunks of the data structure are obtained, each of which has a corresponding chunk key. When data is inserted into or deleted from any individual final chunk, the other final chunks remain unchanged. As a result, when the data structure is edited, modified, or otherwise altered, only a portion of the final chunks may be modified. This ultimately has the effect that when the updated version of the data structure is downloaded to a client that has an older version of the data structure stored locally, only the modified chunks need to be downloaded.

[0069] In some embodiments, approach 100 may further include steps 110-112 or steps 114-116 or a combination thereof.

[0070] In step 110, for each of the multiple final chunks, the processor calculates a hash of the data in the final chunk. For example, the hash of the data may be the MD5 hash of the data, the SHA hash of the data, or any known hashing technique.

[0071] In step 112, for each of the multiple final chunks, the processor adds the hash and separator to the chunk key corresponding to the final chunk.

[0072] Therefore, the chunk key array can be obtained using the hash of the final chunk data. This information can be cached locally. In web development applications, this information can be cached locally in the browser.

[0073] In step 114, the processor compresses each of the multiple final chunks. The final chunks may be compressed individually to improve the speed and efficiency of transfer and storage. For example, the final chunks may be compressed using LZW compression or other known compression techniques.

[0074] In step 116, the processor stores the multiple compressed final chunks and the associated multiple chunk keys. In other words, the multiple compressed final chunks and chunk keys may be stored in a server that is commonly accessible for use by many clients. In this way, the data structure may be loaded by multiple clients as needed. Updates are sent to the server, and the clients can access the latest data structure from the server (by fetching the updated final chunks).

[0075] Figures 3A and 3B show flowcharts of approaches for further processing the final chunks, following one aspect of an exemplary embodiment. Both approaches are aimed at delivering final chunks of a desired size. More specifically, Figure 3A ensures that each final chunk does not exceed a predetermined upper threshold, and Figure 3B ensures that each final chunk satisfies at least a predetermined lower threshold.

[0076] In step 120, the processor determines whether the size of the final chunk is greater than a predetermined upper threshold. If it determines that the size of the final chunk is not greater than the predetermined upper threshold, the method terminates.

[0077] In step 122, if the processor determines that the size of the final chunk is greater than the predetermined upper threshold, it segments the final chunk. The final chunk may be segmented using the approach described in relation to Figure 2, generating a number of quasi-final chunks having corresponding chunk keys.

[0078] Consequently, the final chunk becomes excessively large, thus suppressing or losing the benefits associated with segmenting the data structure into multiple chunks.

[0079] In step 130, the processor determines whether the size of the final chunk is less than a predetermined lower threshold. Depending on whether it determines that the size of the final chunk is not less than the predetermined lower threshold, the approach terminates.

[0080] In step 132, if the size of the final chunk is determined to be below a predetermined lower threshold, the processor combines the final chunk with an adjacent final chunk. In some embodiments, the adjacent final chunk may be a previous final chunk. In this case, the chunk key associated with the final chunk that was determined to be too small may be discarded. For final chunks that are excessively small, the burden of recombination and the increased complexity in individual transmissions outweighs the benefits of splitting the data structure into multiple chunks.

[0081] Referring now to Figure 4, a flowchart of approach 140 for updating (editing) the data structure is shown, following one aspect of an exemplary embodiment.

[0082] In step 142, the processor receives a record of changes, which includes the changes made to the data structure. The record of changes may include a plurality of bits to be added and their corresponding insertion positions, a plurality of bits to be deleted and their corresponding positions, and a plurality of bits to be modified and their corresponding positions.

[0083] Thus, the computational and bandwidth requirements for transmitting a completely updated data structure are avoided by transmitting only the changes. This can be particularly beneficial when only a small number of changes have been made to the data structure. However, this requires that the older version of the data structure be available to the recipient.

[0084] In step 144, the processor combines the multiple final chunks to obtain the data structure. In other words, the older version of the data structure is obtained by combining the stored final chunks. This may require decompressing the final chunks.

[0085] In step 146, the processor updates the data structure based on the record of the change.

[0086] The record of changes may provide a simple list of updates made to the data structure. Thus, bits may be added to or removed from the data structure, and bits may be modified. This provides a completely up-to-date data structure from the previous one.

[0087] In step 148, the processor segments the updated data structure into multiple updated final chunks, where each segment corresponds to a position in the identified bit sequence.

[0088] If the updated data structure is resegmented according to multiple positions in a previously identified bit sequence (reflected in the chunk key), the updated final chunk may be retrieved (because the final chunk contains added bits, deleted bits, updated bits, or a combination thereof). The unchanged final chunk may also be retrieved because it does not contain added bits, deleted bits, updated bits, or a combination thereof. Therefore, if a client with a previous version of the data structure attempts to load the updated data structure, it may only need to load those changed final chunks.

[0089] Figure 5 shows a flowchart of approach 300 for loading a data structure according to another exemplary embodiment.

[0090] In step 302, the processor retrieves a plurality of final chunks and corresponding chunk keys associated with the data structure. The plurality of divisions of the final chunks correspond to a plurality of positions of a bit sequence that is unique within the data structure, and the chunk keys are based on the bit sequence. In other words, the data structure is stored according to one of the methods described with respect to Figures 2-4.

[0091] In step 304, the processor compares the chunk key with several locally stored chunks to identify chunk keys that do not have an associated locally stored chunk.

[0092] In other words, the locally stored chunk may correspond to a previous version of the data structure. If there is a chunk key that is not associated with a locally stored chunk, this means that the locally stored data structure is an older version. The identified chunk key indicates which last chunk needs to be loaded to obtain the latest version of the data structure.

[0093] In step 306, the processor loads the final chunk associated with the identified chunk key.

[0094] In this way, the loading of the entire data structure can be avoided, and thus the amount of data flow through the network to which this method of storing and loading data structures is applied is reduced.

[0095] Some embodiments of approach 300 may further include step 308, in which the processor combines the multiple final chunks to obtain the data structure.

[0096] Therefore, a completely updated data structure can be loaded while minimizing the amount of data that flows through the system for storing, loading, and updating the data structure.

[0097] Referring now to Figure 6, a simplified block diagram of system 400 for storing data structures is shown. System 400 comprises at least a data manipulation unit 410 and a chunk key generation unit 420. Optionally, system 400 may also include an interface unit 430.

[0098] Firstly, the data manipulation unit 410 is configured to segment the data structure into a plurality of initial chunks. The chunk key generation unit 420 is configured to determine a chunk key (for each of the plurality of initial chunks) based on identifying an identifier for a bit sequence within the chunk that is unique within the data structure.

[0099] Furthermore, the data manipulation unit 410 is configured to resegment the data structure into multiple final chunks, where the multiple segments correspond to multiple positions in the identified bit sequence.

[0100] In some embodiments, the chunk key generation unit 420 is configured to identify a bit sequence within a chunk that is unique within the data structure and closest to the first bit, by sequentially scanning the data structure from the first bit of the first chunk to the last bit of the first chunk.

[0101] Furthermore, the system 400 may further include an interface unit 430. The interface unit 430 may be configured to receive a record of changes, including changes made to a data structure. In this case, the data manipulation unit 410 is further configured to take the data structure by combining the multiple final chunks, update the data structure based on the record of changes, and segment the updated data structure into multiple updated final chunks, where the segments correspond to the multiple positions of the identified bit sequence.

[0102] As a further example, as shown in Figure 7, the embodiment may include a computer system 70 that can form part of a networked system 7. For example, a system for editing images to be displayed by a display device may be implemented by the computer system 70. The components of the computer system / server 70 include, but are not limited to, a processor or processing unit 71, system memory 74, and a bus 90 that connects various system components, such as various system components including the system memory 74, to the processing unit 71.

[0103] The system memory 74 may include a computer system-readable medium in the form of volatile memory, such as random access memory (RAM) or cache memory 76 or a combination thereof. The computer system / server 70 may further include other removable / non-removable, volatile / non-volatile computer system storage media 77. In such examples, each may be connected to the bus 90 by one or more data medium interfaces. The memory 74 may contain at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the proposed embodiment. For example, the memory 74 may contain a computer program product having a program executable by a processing unit 71 to cause the system to perform a method of storing data structures according to the proposed embodiment.

[0104] A program / utility 78 having a set (at least one) of program modules 79 may be stored in memory 74. The program modules 79 generally perform functions or methodologies, or combinations thereof, of the embodiments proposed for storing data structures.

[0105] The computer system / server 70 may also communicate with one or more external devices 80, such as a keyboard, pointing device, display 85, etc., which enable a user to interact with the computer system / server 70, or any device (e.g., a network card, modem, etc.) that enables the computer system / server 70 to communicate with one or more other computing devices, or a combination thereof. Such communication can be carried out via the input / output (I / O) interface 72. Furthermore, the computer system / server 70 may communicate via the network adapter 73 with one or more networks, such as a local area network (LAN), a general wide area network (WAN), or a public network (e.g., the Internet), or a combination thereof (e.g., to receive a record of changes, including changes made to data structures).

[0106] In the context of this application, when embodiments of the present invention constitute a method, it should be understood that such a method is a process for execution by a computer, i.e., a method implementable on a computer. Therefore, the various steps of the method reflect various parts of a computer program, for example, various parts of one or more algorithms.

[0107] The present invention may, at any possible level of technical detail of integration, be a system, method, or computer program or computer program product or a combination thereof. The computer program product may encompass one or more computer-readable storage media having computer-readable program instructions for causing a processor to execute the aspects of the present invention.

[0108] The computer-readable storage medium can be a tangible device capable of holding and storing instructions for use by an instruction execution device. The computer-readable storage medium may be, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. A non-exhaustive list of more specific examples of the computer-readable storage medium includes: portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disk (DVD), memory sticks, floppy disks, mechanically encoded devices such as punch cards or grooved structures on which instructions are recorded, or any suitable combination thereof. As used herein, a computer-readable storage medium should not be interpreted as a transient signal itself, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses passing through optical fiber cables), or electrical signals transmitted via wires.

[0109] The computer-readable program instructions described herein may be downloaded from a computer-readable storage medium to individual computing devices / processing devices, or to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, or a wireless network, or a combination thereof. The network may consist of copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, or edge servers, or a combination thereof. A network adapter card or network interface in each computing device / processing device receives computer-readable program instructions from the network and transmits them for storage in a computer-readable storage medium within the individual computing device / processing device.

[0110] Computer-readable program instructions for performing the operation of the present invention include assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, configuration data for integrated circuits, or one or more programming languages, such as object-oriented programming languages ​​(e.g., Smalltalk). (Registered trademark)The source code or object code may be written in any combination of a conventional procedural programming language (e.g., the "C" programming language or a similar programming language), such as C++, etc. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, partially as a standalone software package on the user's computer, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer via any type of network, such as a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), may execute computer-readable program instructions by personalizing the electronic circuit by utilizing state information of the computer-readable program instructions in order to implement the aspects of the present invention.

[0111] The aspects of the present invention are described herein with reference to flowcharts, block diagrams, or combinations thereof of methods, apparatus (systems), and computer programs or computer program products according to embodiments of the present invention. It should be understood that each block in such flowcharts, block diagrams, or combinations thereof, and any combination of blocks in such flowcharts, block diagrams, or combinations thereof, can be implemented by computer-readable program instructions.

[0112] These computer-readable program instructions can be provided to a computer processor or other programmable data processing device to create a machine, such that instructions executed via the processor of the computer or other programmable data processing device generate means for implementing functions / operations specified in one or more blocks of the flowchart or block diagram or a combination thereof. These computer-readable program instructions can also be stored in a computer-readable storage medium that can instruct a computer-programmable data processing device or other device or a combination thereof to function in a particular manner, such that the stored instructions include a product containing instructions that implement the functional / operational aspects specified in one or more blocks of the flowchart or block diagram or a combination thereof.

[0113] The computer-readable program instructions may also be loaded onto the computer, other programmable data processing device, or other device such that instructions executed on the computer, other programmable data processing device, or other device implement the functions / operations specified in one or more blocks of the flowchart or block diagram or combination thereof, thereby causing a series of operational steps on the computer, other programmable device, or other device to generate a computer-implemented process.

[0114] The flowcharts and block diagrams in the drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer programs or computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent a module, segment, or part thereof of instructions, which includes one or more executable instructions for implementing one or more specified logical functions. In some alternative implementations, the functions shown in the blocks may occur in a different order than that shown in the drawings. For example, two consecutively shown blocks may actually be achieved as a single step executed simultaneously, substantially simultaneously, partially or entirely in a temporally overlapping manner, depending on the functions involved, or the blocks may be executed in reverse order. It should also be noted that each block in the block diagram or flowchart or a combination thereof, and any combination of blocks in the block diagram or flowchart or a combination thereof, may be implemented by a special-purpose hardware-based system that performs the specified functions or operations, or by a combination of special-purpose hardware and computer instructions.

[0115] The descriptions of various embodiments of the present invention are provided for illustrative purposes only and are not intended to be exhaustive or to limit the disclosed embodiments. Similarly, examples of features or functions of the embodiments of the present invention described herein, whether used in a particular embodiment description or described as examples, are not intended to limit the embodiments of the present invention described herein or to limit the disclosure to the examples described herein. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terms used herein have been selected to describe the principles of the embodiments, their practical application to or improvement over technologies seen in the market, or to enable those skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method implemented in a computer, Segmenting a data structure into multiple initial chunks; For each of the plurality of initial chunks, a chunk key is determined based on identifying a bit sequence within each chunk that is unique within the data structure; and The data structure is resegmented into multiple final chunks by dividing it at positions corresponding to multiple locations of the chunk key. The method, including the method described above.

2. The computer-implemented method according to claim 1, wherein identifying the bit sequence includes sequentially scanning the data structure from the first bit of each chunk to the last bit of each chunk to identify a bit sequence in the chunk that is unique within the data structure and closest to the first bit.

3. To receive a record of changes, including changes made to the aforementioned data structure; Combining the aforementioned multiple final chunks to obtain the aforementioned data structure; Updating the data structure based on the aforementioned record of the changes; and, Segmenting the updated data structure into a plurality of updated final chunks, where the plurality of segments correspond to the plurality of positions in the identified bit sequence. A computer-implemented method according to claim 1, further comprising:

4. For each of the aforementioned multiple final chunks, it is determined that the size of each individual final chunk is greater than a predetermined upper threshold; and In response to determining that the size of each of the final chunks is greater than the predetermined upper threshold, the individual final chunks are segmented. A computer-implemented method according to claim 1, further comprising:

5. For each of the aforementioned multiple final chunks, it is determined that the size of each individual final chunk is smaller than a predetermined lower threshold; and In response to determining that the size of each of the final chunks is smaller than the predetermined lower threshold, the individual final chunks are combined with adjacent final chunks. A computer-implemented method according to claim 1, further comprising:

6. For each of the aforementioned multiple final chunks, Calculating a hash of one of the data from each of the final chunks; and, Adding the aforementioned hash and separator to the chunk key corresponding to each of the final chunks. A computer-implemented method according to claim 1, further comprising:

7. Compressing each of the aforementioned multiple final chunks; and, The aforementioned multiple compressed final chunks and the associated multiple chunk keys are stored on the server. A computer-implemented method according to claim 1, further comprising:

8. The computer-implemented method according to claim 1, wherein the data structure is selected from the group consisting of JavaScript® object annotation (JSON) data structures, rich text format (RTF) files, and code files.

9. By comparing the chunk key with a plurality of locally stored chunks, identify chunk keys that do not have an associated locally stored chunk; and, Loading the individual final chunks associated with the identified chunk key. A computer-implemented method according to claim 1, further comprising:

10. A computer-implemented method according to claim 1, further comprising combining the plurality of final chunks to obtain the data structure.

11. Retrieving the aforementioned multiple final chunks and the corresponding chunk keys associated with the data structure, where, The multiple divisions of the final chunk correspond to multiple positions of a bit sequence that is unique within the data structure; and, The chunk key is based on the bit sequence; The chunk key is compared with a plurality of locally stored chunks to identify chunk keys that do not have an associated locally stored chunk; and, Load the last chunk associated with the identified chunk key. A computer-implemented method according to claim 1, further comprising:

12. The method implemented in a computer according to claim 1, wherein the chunk key is based on the identified bit sequence.

13. The determined chunk key is stored in the chuck array. A computer-implemented method according to claim 1, comprising adding each of the identified unique bit sequences to the array of chunk keys.

14. The computer-implemented method according to claim 1, wherein the resegmentation obtains an array of chunk keys that identify the corresponding final chunk.

15. The computer-implemented method according to claim 1, wherein each of the final chunks has a corresponding chunk key.

16. The computer-implemented method according to claim 6, wherein the final chunk is cached locally on the browser using the hash as the cache key.

17. The computer-implemented method according to claim 6, wherein the array of chunk keys is obtained using the hash of the last chunk.

18. The computer-implemented method according to claim 17, wherein the array of chunk keys is cached locally in the browser.

19. A computer program that causes a processor to perform each step of the method according to any one of claims 1 to 18.

20. A computer system, The computer system comprises one or more computer processors and one or more computer-readable storage media, wherein the computer program described in claim 19 is stored in the one or more computer-readable storage media, and the one or more computer processors execute the computer program.

Images