An information storage method, a storage device, and a storage medium

Abstract

The embodiment of the application discloses an information storage method, a storage device and a storage medium. The information storage method is applied to the storage device, and the storage device comprises a message queue. The information processing method comprises the following steps: in the case that a file storage request transmitted by a terminal is received, a file calling instruction corresponding to the file storage request is generated by using the message queue according to a storage request response sequence; the file calling instruction is transmitted to the terminal, so that the file calling instruction is used to acquire a to-be-stored file corresponding to a file transmission instruction from the terminal; and the to-be-stored file is stored.

Description

Technical Field

[0001] This application relates to the field of information storage technology, and in particular to an information storage method, terminal, device, and storage medium. Background Technology

[0002] As the business system architecture evolves, the number of business processes increases, leading to a growing number of business files and creating numerous problems in their storage.

[0003] In existing technologies, when a terminal obtains storage information, it directly sends an information transmission notification to the storage system and simultaneously calls the information transmission interface to transmit the storage information directly to the storage system. When multiple terminals send multiple pieces of storage information to the storage system at the same time, the storage system needs to perform storage procedures on multiple pieces of storage information simultaneously. This can easily cause the storage system to crash, thereby reducing the speed of file storage. Summary of the Invention

[0004] To address the aforementioned technical problems, embodiments of this application aim to provide an information storage method, terminal, device, and storage medium that can improve the speed at which storage devices store files to be stored.

[0005] The technical solution of this application is implemented as follows:

[0006] This application provides an information storage method applied to a storage device, the storage device including a message queue, comprising:

[0007] Upon receiving a file storage request transmitted from a terminal, the message queue is used to generate a file retrieval instruction corresponding to the file storage request in the order of storage request responses.

[0008] The file retrieval instruction is transmitted to the terminal to retrieve the file to be stored corresponding to the file transfer instruction from the terminal using the file retrieval instruction.

[0009] Store the file to be stored.

[0010] This application embodiment provides another information storage method, applied to a terminal, including:

[0011] Upon obtaining the file to be stored, a file storage request corresponding to the file to be stored is sent to the storage device;

[0012] Upon receiving a file retrieval instruction corresponding to the file storage request generated by the storage device using a message queue in the order of storage request responses, the file to be stored is transmitted to the storage device so that the storage device can store the file to be stored using the message queue.

[0013] This application embodiment provides a storage device, the storage device including a message queue, the device comprising:

[0014] A generation unit is used to generate a file call instruction corresponding to the file storage request in accordance with the storage request response order using the message queue when a file storage request is received from the terminal.

[0015] The first transmission unit is used to transmit the file retrieval instruction to the terminal, so as to obtain the file to be stored corresponding to the file transmission instruction from the terminal using the file retrieval instruction;

[0016] A storage unit is used to store the file to be stored.

[0017] This application provides a terminal, the terminal comprising:

[0018] The sending unit is used to send a file storage request corresponding to the file to be stored to the storage device when the file to be stored is obtained;

[0019] The second transmission unit is used to transmit the file to be stored to the storage device when it receives a file call instruction corresponding to the file storage request generated by the storage device using a message queue in accordance with the storage request response order, so that the storage device can use the message queue to store the file to be stored.

[0020] This application embodiment also provides a storage device, the device comprising:

[0021] The system comprises a first memory, a first processor, and a first communication bus. The first memory communicates with the first processor via the first communication bus. The first memory stores a program of information storage that can be executed by the first processor. When the program of information storage is executed, the first processor executes an information storage method applied to the storage device.

[0022] This application embodiment also provides a terminal, the terminal comprising:

[0023] The second memory, the second processor, and the second communication bus are connected. The second memory communicates with the second processor via the second communication bus. The second memory stores a program of information storage that can be executed by the second processor. When the program of information storage is executed, the information storage method applied to the terminal is executed by the second processor.

[0024] This application provides a storage medium storing a computer program for use in storage devices and terminals. The computer program, when executed by a first processor, implements an information storage method applied to the storage device; and when executed by a second processor, it implements an information storage method applied to the terminal.

[0025] This application provides an information storage method, terminal, device, and storage medium. The information storage method is applied to a storage device, which includes a message queue. The information storage method includes: upon receiving a file storage request transmitted from a terminal, generating a file retrieval instruction corresponding to the file storage request using the message queue according to the storage request response order; transmitting the file retrieval instruction to the terminal to retrieve the file to be stored corresponding to the file transmission instruction from the terminal using the file retrieval instruction; and storing the file to be stored. Using the above method, the storage device uses a message queue to generate a file retrieval instruction corresponding to the file storage request according to the storage request response order, and retrieves the file to be stored from the terminal using the file retrieval instruction for storage. When the number of files to be stored is large, the storage order of the files is controlled by using a message queue according to the storage request response order, reducing the risk of storage device downtime and improving the speed of storing the files. Attached Figure Description

[0026] Figure 1 A schematic diagram of an information storage structure in the prior art provided in this application embodiment. Figure 1 ;

[0027] Figure 2 A schematic diagram of an information storage method in the prior art provided in this application embodiment. Figure 2 ;

[0028] Figure 3 A flowchart of an information storage method provided in this application embodiment Figure 1 ;

[0029] Figure 4 A schematic diagram illustrating an exemplary information storage method provided in this application embodiment;

[0030] Figure 5 A flowchart of an information storage method provided in this application embodiment Figure 2 ;

[0031] Figure 6 A schematic diagram illustrating an exemplary information storage method provided in this application embodiment;

[0032] Figure 7 A schematic diagram illustrating an exemplary hash value determination method provided in this application embodiment;

[0033] Figure 8A schematic diagram of the composition structure of a storage device provided in an embodiment of this application. Figure 1 ;

[0034] Figure 9 A schematic diagram of the composition structure of a storage device provided in an embodiment of this application. Figure 2 ;

[0035] Figure 10 A schematic diagram of the composition structure of a terminal provided in this application embodiment. Figure 1 ;

[0036] Figure 11 A schematic diagram of the composition structure of a terminal provided in this application embodiment. Figure 2 . Detailed Implementation

[0037] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0038] In the existing technical architecture, the front-end business application performs file uploads, and the back-end business services handle the data flow as follows: Figure 1 As shown: The backend business service serves as the traffic entry point, posing a risk of business node congestion. In existing technologies, all related algorithms are concentrated on the business node, which causes significant problems with the service's computing and storage resources, primarily including:

[0039] System TPS (Transactions Per Second) equals the number of transactions per unit of time. Since a business application often involves numerous associated attachments, high-concurrency scenarios require guaranteed computing and storage resources. Furthermore, file read / write operations are highly resource-intensive, not only subject to system read / write handle limitations but also posing a risk of memory overflow. In existing solutions, business data and file read / write operations occur simultaneously and are centralized within the business system. File duplication is determined based on hash values, making the speed and accuracy of hash calculation crucial factors in file similarity detection. However, for large files, reading the entire file content to calculate the hash also puts additional memory pressure on the business system.

[0040] A schematic diagram of file storage in the prior art, such as Figure 2 As shown, during the evolution of business system architecture, with the increase in business complexity, the traditional monolithic service architecture will inevitably evolve into a distributed system structure. Businesses involving the uploading and downloading of attachments will also face the following challenges:

[0041] When attachments are shared between systems, the file processing time and bandwidth usage vary depending on the attachment size, posing a risk of blocking business traffic. In high-concurrency service scenarios, untimely processing of file stream data can easily lead to insufficient service memory, resulting in service downtime. During service reconstruction, historical data migration faces challenges, and the disk resources consumed increase over time, making file copying between servers more difficult.

[0042] The problems existing in the prior art can be solved using the methods in Embodiment 1 and Embodiment 2.

[0043] Example 1

[0044] This application provides an information storage method applied to a storage device, which includes a message queue. Figure 3 A flowchart of an information storage method provided in this application embodiment Figure 1 ,like Figure 3 As shown, information storage methods may include:

[0045] S101. Upon receiving a file storage request transmitted from the terminal, generate a file call instruction corresponding to the file storage request using a message queue in the order of storage request responses.

[0046] The information storage method provided in this application embodiment is applicable to scenarios where files to be stored are transmitted from a terminal.

[0047] In the embodiments of this application, the storage device can be implemented in various forms. For example, the storage device described in this application may include devices such as mobile phones, cameras, tablet computers, laptops, handheld computers, personal digital assistants (PDAs), portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, and devices such as digital TVs and desktop computers.

[0048] In the embodiments of this application, the storage device can be Storage, which is a general term for storage systems.

[0049] In this embodiment of the application, the message queue includes at least one storage request. The message queue processes the at least one storage request sequentially according to the storage request response order to obtain at least one file call instruction corresponding to the at least one storage request. The at least one file call instruction includes a file call instruction corresponding to the storage request.

[0050] In this embodiment, the storage request response order can be information configured in the storage device; the storage request response order can also be information received by the storage device before it generates the file call instruction corresponding to the file storage request according to the storage request response order using a message queue; the storage request response order can also be information obtained by the storage device in other ways; the specific order can be determined according to the actual situation, and this embodiment does not limit it.

[0051] In this embodiment, the problems of file reading and writing hindering business threads, slow processing speed, and poor security caused by copying files between related subsystems can be solved by increasing the storage capacity of the storage device.

[0052] S102. Transmit a file retrieval command to the terminal to obtain the file to be stored corresponding to the file transfer command from the terminal using the file retrieval command.

[0053] In this embodiment, after the storage device generates a file retrieval instruction corresponding to the file storage request according to the storage request response order using a message queue, the storage device can transmit the file retrieval instruction to the terminal to obtain the file to be stored corresponding to the file transfer instruction from the terminal using the file retrieval instruction.

[0054] S103. Store the file to be stored.

[0055] In this embodiment, the storage device transmits a file retrieval instruction to the terminal. After obtaining the file to be stored corresponding to the file transfer instruction from the terminal using the file retrieval instruction, the storage device can store the file to be stored.

[0056] In this embodiment, the storage device may directly store the file to be stored upon receiving it from the terminal; the storage device may also store the file to be stored using a message queue upon receiving it; or the storage device may store the file to be stored according to the storage time determined by the message queue upon receiving it. The specific method by which the storage device stores the file to be stored can be determined according to the actual situation, and this embodiment does not limit it.

[0057] In this embodiment of the application, the process of a storage device storing a file to be stored includes: the storage device acquiring a stored file; the storage device determining a similarity value between the file to be stored and the stored file; and storing the file to be stored if the similarity value is less than a similarity threshold.

[0058] In this embodiment, the similarity threshold can be a threshold configured in the storage device; the similarity threshold can also be a threshold received by the storage device before the similarity value is determined to be less than the similarity threshold; the similarity threshold can also be a threshold obtained by the storage device in other ways; the specific way the storage device obtains the similarity threshold can be determined according to the actual situation, and this embodiment does not limit it.

[0059] In this embodiment of the application, the similarity threshold can be a percentage; the similarity threshold can also be other forms of threshold; the specific threshold can be determined according to the actual situation, and this embodiment of the application does not limit it.

[0060] In this embodiment of the application, the storage file obtained by the storage device (The file to be stored or the file already stored) is shown in formula (1):

[0061] (1)

[0062] in, For storing file formats, For the file size of the stored file, For storing the hash value of the file, This is used to store the relationship between files and file formats, file sizes, and hash values.

[0063] Specifically, the storage device obtains the file to be stored. As shown in formula (2); the stored file obtained by the storage device As shown in formula (3):

[0064] (2)

[0065] (3)

[0066] It should be noted that, The first file format of the file to be stored The first file size of the file to be stored, This is the first hash value of the file to be stored. For the second file format of the already stored file, The second file size is the amount of the stored file. The second hash value of the stored file.

[0067] In this embodiment, the process by which a storage device determines the similarity value between a file to be stored and a file already stored includes: the storage device acquiring a first file capacity, a first file format, and a first hash value corresponding to the file to be stored, and a second file capacity, a second file format, and a second hash value corresponding to the file already stored; the storage device determining capacity difference information between the first file capacity and the second file capacity; the storage device determining format difference information between the first file format and the second file format; the storage device determining hash difference information between the first hash value and the second hash value; and the storage device determining a similarity value based on the capacity difference information, the format difference information, and the hash difference information.

[0068] In this embodiment of the application, when the file size of the stored files (the file to be stored and the file already stored) is greater than or equal to a preset capacity threshold, the storage device will first group the file content of the stored files to obtain multiple groups of file content. The storage device will then determine the hash corresponding to each of the multiple groups of file content to obtain multiple hash values. After that, the multiple hashes will be used as the hash values ​​corresponding to the stored files (the file to be stored and the file already stored).

[0069] In this embodiment of the application, the multiple hash values ​​can be 10 hash values; the multiple hash values ​​can also be 1024 hash values; the number of multiple hash values ​​can also be other values; the number of multiple hash values ​​can be determined according to the actual situation, and this embodiment of the application does not limit it.

[0070] In this embodiment of the application, the storage device can determine the capacity difference information between the first file capacity and the second file capacity by: the storage device determining the capacity difference between the first file capacity and the second file capacity, and then the storage device determining the square of the capacity difference to obtain the capacity difference information.

[0071] In this embodiment of the application, the storage device may determine the format difference information between the first file format and the second file format by: determining the format difference value between the first file format and the second file format, and then determining the square of the format difference value to obtain the format difference information.

[0072] In this embodiment, if there are multiple first hash values ​​and multiple second hash values, the storage device determines the hash difference information between the first hash values ​​and the second hash values ​​as follows: the storage device determines the differences between the multiple first hash values ​​and the multiple second hash values ​​respectively, obtaining multiple differences; then, the storage device determines the squares of the multiple differences respectively, obtaining multiple square values; then, the storage device determines the sum of the multiple square values, obtaining a total square value; finally, the storage device takes the square root of the total square value to obtain the hash difference information. If there is only one first hash value and only one second hash value, the storage device determines the hash difference information between the first hash value and the second hash value as follows: the storage device determines the hash difference value between the first hash value and the second hash value; then, the storage device determines that the hash difference value is the hash difference information.

[0073] In this embodiment of the application, the storage device determines the similarity value based on the capacity difference information, format difference information, and hash difference information by determining the sum of the capacity difference information, format difference information, and hash difference information to obtain the sum information; the storage device then takes the square root of the sum information to obtain the similarity value.

[0074] In this embodiment, the preset capacity threshold can be a capacity threshold configured in the storage device; the preset capacity threshold can also be a capacity threshold transmitted to the storage device by other devices; the preset capacity threshold can also be a capacity threshold obtained by the storage device in other ways. The specific way in which the storage device obtains the preset capacity threshold can be determined according to the actual situation, and this embodiment does not limit it.

[0075] It should be noted that the preset capacity threshold can be a threshold of 5k; the preset capacity threshold can also be a threshold of 1M; the preset capacity threshold can also be a threshold of other values; the specific value of the preset capacity threshold can be determined according to the actual situation, and this application embodiment does not limit it.

[0076] In this embodiment of the application, the process by which the storage device determines a similarity value based on capacity difference information, format difference information, and hash difference information includes: the storage device determining a file capacity coefficient based on a first file capacity and a second file capacity; the storage device determining a hash value coefficient based on a first hash value and a second hash value; the storage device determining a file format coefficient based on the file capacity coefficient and the hash value coefficient; and the storage device determining a similarity value based on the file capacity coefficient, capacity difference information, hash value coefficient, hash difference information, file format coefficient, and format difference information.

[0077] In this embodiment of the application, the storage device determines the file capacity coefficient based on the first file capacity and the second file capacity in the following ways: the storage device selects the target file capacity with the largest file capacity value from the first file capacity and the second file capacity; then, the storage device determines the product between the target file capacity and the preset target file capacity coefficient; then, the storage device determines the square value of the product; finally, the storage device divides the preset coefficient by the square value to obtain the file capacity coefficient.

[0078] It should be noted that the preset target file capacity coefficient can be 3; the preset target file capacity coefficient can also be 5; the preset target file capacity coefficient can also be other values; the specific value can be determined according to the actual situation, and this application embodiment does not limit it.

[0079] It should be noted that the preset coefficient can be 1; the preset coefficient can also be 3; the preset coefficient can also be other values; the specific preset coefficient can be determined according to the actual situation, and this application embodiment does not limit it.

[0080] In this embodiment, if there are multiple first hash values ​​and multiple second hash values, the storage device determines the hash value coefficient based on the first and second hash values ​​in the following ways: the storage device determines the squares of multiple first hash values ​​to obtain multiple first hash square values; the storage device determines the sum of multiple first hash square values ​​to obtain a total first hash square value; the storage device takes the square root of the total first hash square value to obtain a first square value; the storage device determines the squares of multiple second hash values ​​to obtain multiple second hash square values; the storage device determines the sum of multiple second hash square values ​​to obtain a total second hash square value; the storage device takes the square root of the total second hash square value to obtain a second square value; the storage device determines the target square value with the largest value from the first and second square values; the storage device determines the product of the target square value and a preset target square value coefficient to obtain a target product; the storage device divides the target product by the preset coefficient to obtain the hash value coefficient.

[0081] It should be noted that the preset target square root value coefficient can be 3; the preset target square root value coefficient can also be 5; the preset target square root value coefficient can also be other values; the specific preset target square root value coefficient can be determined according to the actual situation, and this application embodiment does not limit it.

[0082] In this embodiment of the application, the storage device determines the file format coefficient based on the file capacity coefficient and the hash value coefficient by summing the file capacity coefficient and the hash value coefficient to obtain the sum coefficient; then, the storage device determines the difference between the preset value and the sum coefficient to obtain the file format coefficient.

[0083] It should be noted that the preset value can be 1 / 3; the preset value can also be 1 / 2; the preset value can also be other values; the specific preset value can be determined according to the actual situation, and this application embodiment does not limit it.

[0084] In this embodiment, the storage device determines the similarity value based on the file capacity coefficient, capacity difference information, hash value coefficient, hash difference information, file format coefficient, and format difference information in the following ways: the storage device determines the product between the file capacity coefficient and the capacity difference information to obtain a first product; the storage device determines the product between the hash value coefficient and the hash difference information to obtain a second product; the storage device determines the product between the file format coefficient and the format difference information to obtain a third product; the storage device determines the sum of the first product, the second product, and the third product to obtain a total product; and the storage device takes the square root of the total product to obtain the similarity value.

[0085] In this embodiment of the application, the multiple hash values ​​(first hash value) corresponding to the file to be stored can be The multiple hash values ​​(second hash values) corresponding to the stored file can be... .

[0086] In this embodiment, the storage device determines the similarity value between the file to be stored and the file already stored. The method is as shown in formula (4):

[0087] (4)

[0088] The first file has a size of The second file size is The first file format is The second file format is File size factor As shown in formula (5), the hash value coefficient As shown in formula (6), the file format coefficient As shown in formula (7):

[0089] (5)

[0090] It should be noted that 3 is the preset target file size factor; 1 is the preset factor.

[0091] (6)

[0092] It should be noted that 3 is the preset target square root value coefficient; 1 is the preset coefficient.

[0093] (7)

[0094] It should be noted that 1 / 3 is a preset value.

[0095] It is understandable that the storage device determines the file capacity coefficient using the first file capacity and the second file capacity; determines the hash value coefficient using the first hash value and the second hash value; determines the file format coefficient using the file capacity coefficient and the hash value coefficient; and determines the similarity value based on the file capacity coefficient, capacity difference information, hash value coefficient, hash difference information, file format coefficient, and format difference information, thereby improving the accuracy of determining the similarity value between the file to be stored and the already stored file.

[0096] In this embodiment of the application, after the storage device stores the file to be stored, the storage device also generates storage success information corresponding to the file to be stored; the storage device transmits the storage success information to the terminal.

[0097] In this embodiment, the successful storage information can be text information; it can also be character information; it can also be other forms of information; the specific form of the successful storage information can be determined according to the actual situation, and this embodiment does not limit it.

[0098] In this embodiment, the storage device may use a message queue to transmit storage success information to the terminal; the storage device may also directly transmit storage success information to the terminal; or the storage device may transmit storage success information to the terminal through other means; the specific method by which the storage device transmits storage success information to the terminal can be determined according to the actual situation, and this embodiment does not limit it.

[0099] In this embodiment, when storing files, an asynchronous storage method is adopted. This allows the files to be uploaded only once and transferred to the storage device, enabling different systems to download them from multiple locations. This avoids repeated file streams within various related subsystems, which consumes memory and degrades service performance. During the asynchronous storage process, a message queue is introduced to handle asynchronous operations and flow control. Specifically, as shown... Figure 4 As shown: The terminal (first terminal and / or second terminal) transmits a file storage request to the storage device; the storage device utilizes a message queue ( Figure 4 In the MQ (Message Queue), the storage device generates file retrieval instructions corresponding to the file storage request according to the order of storage request response; the storage device transmits the file retrieval instructions to the terminal to retrieve the file to be stored corresponding to the file transfer instructions from the terminal and store the file to be stored; or the storage device retrieves the stored files in the storage device to the terminal through the message queue.

[0100] Understandably, storage devices use message queues to generate file access instructions corresponding to file storage requests in the order of storage request responses. They then use these instructions to retrieve the files to be stored from the terminal. When there are a large number of files to be stored, the storage order can be controlled by using message queues to control the storage order of the files, thereby reducing the risk of storage device downtime and improving the speed of storing the files.

[0101] Example 2

[0102] This application embodiment further provides an information storage method applied to a terminal. Figure 5 A flowchart of an information storage method provided in this application embodiment Figure 2 ,like Figure 5 As shown, information storage methods may include:

[0103] S201. If the file to be stored is obtained, send a file storage request corresponding to the file to be stored to the storage device.

[0104] The information storage method provided in this application embodiment is applicable to scenarios where files to be stored are transmitted from a terminal.

[0105] In the embodiments of this application, the terminal can be implemented in various forms. For example, the terminal described in this application may include devices such as mobile phones, cameras, tablet computers, laptops, handheld computers, personal digital assistants (PDAs), portable media players (PMPs), navigation devices, wearable devices, smart bracelets, pedometers, and devices such as digital TVs and desktop computers.

[0106] In this embodiment, the file to be stored can be a file generated in the terminal; the file to be stored can also be a file transmitted to the terminal from other devices; the file to be stored can also be a file obtained by the terminal in other ways; the specific way the terminal obtains the file to be stored can be determined according to the actual situation, and this embodiment does not limit it.

[0107] In this embodiment, the file to be stored can be a file obtained by the terminal at a preset time interval; the file to be stored can also be a file obtained by the terminal at an indefinite time interval; the file to be stored can also be a file obtained by the terminal when it receives a file acquisition instruction; the specific way the terminal obtains the file to be stored can be determined according to the actual situation, and this embodiment does not limit it.

[0108] In this embodiment of the application, the process of a terminal sending a file storage request corresponding to a file to be stored to a storage device includes: the terminal obtaining the file content of the file to be stored; the terminal determining a first hash value of the file to be stored based on the file content; the terminal adding the first hash value to the initial file storage request to obtain the file storage request; and the terminal sending the file storage request to the storage device.

[0109] In this embodiment of the application, the process of the terminal obtaining the file content of the file to be stored further includes: the terminal determining the first file size of the file to be stored; correspondingly, the process of the terminal determining the first hash value of the file to be stored based on the file content includes: when the first file size is greater than or equal to a preset capacity threshold, the terminal groups the file content to obtain multiple groups of files; the terminal determines the hash value of each of the multiple groups of files to obtain multiple hash values; and the terminal uses the multiple hash values ​​as the first hash value.

[0110] For example, if there are 5 hash values, the process of the terminal using these 5 hash values ​​as the first hash value can be as follows: the terminal considers all 5 hash values ​​as the first hash value, that is, the first hash value includes 5 hash values; if there are 10 hash values, the process of the terminal using these 10 hash values ​​as the first hash value can be as follows: the terminal considers all 10 hash values ​​as the first hash value, that is, the first hash value includes 10 hash values; the specific number of hash values ​​can be determined according to the actual situation, and this application embodiment does not limit this.

[0111] In this embodiment, the preset capacity threshold can be a threshold configured in the terminal; the preset capacity threshold can also be a threshold received by the terminal before comparing the capacity of the first file with the preset capacity threshold; the preset capacity threshold can also be a threshold obtained by the terminal in other ways; the specific way the terminal obtains the preset capacity threshold can be determined according to the actual situation, and this embodiment does not limit it.

[0112] It should be noted that the preset capacity threshold can be 5k, 1M, or other capacity values; the specific value of the preset capacity threshold can be determined according to the actual situation, and this application embodiment does not limit it.

[0113] In this embodiment of the application, the process of the terminal determining the first hash value of the file to be stored based on the file content includes: when the first file size is less than a preset size threshold, the terminal determines the first hash value of the file to be stored based on the file content.

[0114] S202. Upon receiving a file retrieval instruction corresponding to a file storage request generated by the storage device using a message queue in accordance with the storage request response order, the storage device transmits the file to be stored to the storage device so that the storage device can store the file to be stored using the message queue.

[0115] In this embodiment, after the terminal sends a file storage request corresponding to the file to be stored to the storage device, the terminal, upon receiving a file retrieval instruction corresponding to the file storage request generated by the storage device using a message queue in accordance with the storage request response order, transmits the file to be stored to the storage device so that the storage device can store the file to be stored using the message queue.

[0116] In this embodiment of the application, the storage procedure for the file to be stored is changed to an asynchronous file stream method, such as... Figure 6 As shown: Middleware is introduced to decouple business logic and files to be stored. Terminals only need to send file requests to the message queue without waiting for file processing to complete. The storage device handles file retrieval and saving uniformly. By controlling the speed of the storage device's message queue processing, problems such as file read / write operations hindering business threads, slow processing speeds due to file copying between related subsystems, and poor security are addressed.

[0117] In this embodiment of the application, when the size of the first file is greater than or equal to a preset size threshold, the terminal groups the file content to obtain multiple groups of files, specifically as follows: Figure 7 As shown: If the file to be stored is a File, the file format is txt, and the file size is 1048576 (equal to 1Mb), then when calculating the file hash, a sampler can first be used to group the data into 64-bit groups, resulting in 16384 groups. Then, each group is cyclically sampled from 1 to 64, ultimately yielding 16384 data points, resulting in 16384 hash values ​​(sampled hash).

[0118] Understandably, when a terminal obtains a file to be stored, it sends a file storage request to the storage device. By receiving the file call instruction corresponding to the file storage request generated by the storage device using a message queue according to the storage request response order, the terminal transmits the file to be stored to the message queue. The message queue is then used to store the file to be stored on the storage device. When the number of files to be stored is large, the storage order of the files is controlled by using the message queue according to the storage request response order, which reduces the risk of storage device downtime and improves the speed of storing the files.

[0119] Example 3

[0120] Based on the same inventive concept as Embodiment 1, this application provides a storage device 1, corresponding to an information storage method applied in the storage device; Figure 8 A schematic diagram of the composition structure of a storage device provided in an embodiment of this application. Figure 1 The storage device 1 may include:

[0121] The generation unit 11 is used to generate a file call instruction corresponding to the file storage request in the order of the storage request response using the message queue when a file storage request is received from the terminal.

[0122] The first transmission unit 12 is used to transmit the file retrieval instruction to the terminal, so as to obtain the file to be stored corresponding to the file transmission instruction from the terminal using the file retrieval instruction;

[0123] Storage unit 13 is used to store the file to be stored.

[0124] In some embodiments of this application, the device further includes a first acquisition unit and a first determination unit;

[0125] The first acquisition unit is used to acquire the stored files in the storage device;

[0126] The first determining unit is used to determine the similarity value between the file to be stored and the already stored file;

[0127] The storage unit 13 is used to store the file to be stored when the similarity value is less than the similarity threshold.

[0128] In some embodiments of this application, the first acquisition unit is used to acquire the first file capacity, first file format and first hash value corresponding to the file to be stored, and the second file capacity, second file format and second hash value corresponding to the stored file;

[0129] The first determining unit is configured to determine capacity difference information between the first file capacity and the second file capacity; determine format difference information between the first file format and the second file format; determine hash difference information between the first hash value and the second hash value; and determine the similarity value based on the capacity difference information, the format difference information, and the hash difference information.

[0130] In some embodiments of this application, the first determining unit is configured to determine a file capacity coefficient based on the first file capacity and the second file capacity; determine a hash value coefficient based on the first hash value and the second hash value; determine a file format coefficient based on the file capacity coefficient and the hash value coefficient; and determine the similarity value based on the file capacity coefficient, the capacity difference information, the hash value coefficient, the hash difference information, the file format coefficient, and the format difference information.

[0131] In some embodiments of this application, the generation unit 11 is used to generate storage success information corresponding to the file to be stored;

[0132] The first transmission unit 12 is used to transmit the storage success information to the terminal.

[0133] It should be noted that, in practical applications, the aforementioned generation unit 11, first transmission unit 12, and storage unit 13 can be implemented by the first processor 14 on the storage device 1, specifically by a CPU (Central Processing Unit), MPU (Microprocessor Unit), DSP (Digital Signal Processor), or Field Programmable Gate Array (FPGA), etc.; the aforementioned data storage can be implemented by the first memory 15 on the storage device 1.

[0134] This application embodiment also provides a storage device 1, such as... Figure 9 As shown, the storage device 1 includes: a first processor 14, a first memory 15, and a first communication bus 16. The first memory 15 communicates with the first processor 14 through the first communication bus 16. The first memory 15 stores programs executable by the first processor 14. When the program is executed, the first processor 14 executes an information storage method applied to the storage device.

[0135] In practical applications, the first memory 15 can be volatile memory, such as random-access memory (RAM); or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD) or solid-state drive (SSD); or a combination of the above types of memory, and provide instructions and data to the first processor 14.

[0136] This application provides a computer-readable storage medium having a computer program thereon, which, when executed by a first processor 14, implements the information storage method as described above.

[0137] Understandably, storage devices use message queues to generate file access instructions corresponding to file storage requests in the order of storage request responses. They then use these instructions to retrieve the files to be stored from the terminal. When there are a large number of files to be stored, the storage order can be controlled by using message queues to control the storage order of the files, thereby reducing the risk of storage device downtime and improving the speed of storing the files.

[0138] Example 4

[0139] Based on the same inventive concept as Embodiment 2, this application provides a terminal 2, corresponding to an information storage method applied in the terminal; Figure 10 A schematic diagram of the composition structure of a terminal provided in this application embodiment. Figure 1 The terminal 2 may include:

[0140] The sending unit 21 is used to send a file storage request corresponding to the file to be stored to the storage device when the file to be stored is obtained;

[0141] The second transmission unit 22 is used to transmit the file to be stored to the storage device when it receives a file call instruction corresponding to the file storage request generated by the storage device using a message queue in accordance with the storage request response order, so that the storage device can use the message queue to store the file to be stored.

[0142] In some embodiments of this application, the device further includes a second acquisition unit, an addition unit, and a second determination unit;

[0143] The second acquisition unit is used to acquire the file content of the file to be stored;

[0144] The second determining unit is used to determine the first hash value of the file to be stored based on the file content;

[0145] The adding unit is used to add the first hash value to the initial file storage request to obtain the file storage request;

[0146] Sending unit 21 is used to send the file storage request to the storage device.

[0147] In some embodiments of this application, the second determining unit is used to determine the first file size of the file to be stored;

[0148] Correspondingly, the terminal also includes a grouping unit;

[0149] The grouping unit is used to group the file content into multiple groups of files when the first file size is greater than or equal to a preset size threshold.

[0150] The second determining unit is used to determine the hash values ​​of the multiple sets of files respectively, to obtain multiple hash values; and to use the multiple hash values ​​as the first hash value.

[0151] In some embodiments of this application, the second determining unit is used to determine the first hash value of the file to be stored based on the file content when the first file size is less than the preset capacity threshold.

[0152] It should be noted that, in practical applications, the aforementioned sending unit 21 and the second transmission unit 22 can be implemented by the second processor 23 on the terminal 2, specifically by a CPU (Central Processing Unit), MPU (Microprocessor Unit), DSP (Digital Signal Processor), or FPGA (Field Programmable Gate Array), etc.; the aforementioned data storage can be implemented by the second memory 24 on the terminal 2.

[0153] This application embodiment also provides a terminal 2, such as Figure 11 As shown, the terminal 2 includes: a second processor 23, a second memory 24, and a second communication bus 25. The second memory 24 communicates with the second processor 23 through the second communication bus 25. The second memory 24 stores programs executable by the second processor 23. When the program is executed, the second processor 23 executes an information storage method applied to the terminal.

[0154] In practical applications, the second memory 24 can be volatile memory, such as random-access memory (RAM); or non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD) or solid-state drive (SSD); or a combination of the above types of memory, and provide instructions and data to the second processor 23.

[0155] This application provides a computer-readable storage medium having a computer program thereon, which, when executed by a second processor 23, implements the information storage method as described above.

[0156] Understandably, when a terminal obtains a file to be stored, it sends a file storage request to the storage device. By receiving the file call instruction corresponding to the file storage request generated by the storage device using a message queue according to the storage request response order, the terminal transmits the file to be stored to the message queue. The message queue is then used to store the file to be stored on the storage device. When the number of files to be stored is large, the storage order of the files is controlled by using the message queue according to the storage request response order, which reduces the risk of storage device downtime and improves the speed of storing the files.

[0157] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0158] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0159] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0160] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0161] The above description is merely a preferred embodiment of this application and is not intended to limit the scope of protection of this application.

Claims

1. An information storage method, characterized in that, Applied to a storage device, the storage device including a message queue, the method includes: Upon receiving a file storage request transmitted from a terminal, the message queue is used to generate a file retrieval instruction corresponding to the file storage request in the order of storage request responses. The file retrieval instruction is transmitted to the terminal to retrieve the file to be stored corresponding to the file transfer instruction from the terminal using the file retrieval instruction. Storing the file to be stored includes: acquiring a file already stored in the storage device; acquiring a first file capacity, a first file format, and a first hash value corresponding to the file to be stored, and a second file capacity, a second file format, and a second hash value corresponding to the already stored file; determining capacity difference information between the first file capacity and the second file capacity; determining format difference information between the first file format and the second file format; determining hash difference information between the first hash value and the second hash value; determining a file capacity coefficient based on the first file capacity and the second file capacity; determining a hash value coefficient based on the first hash value and the second hash value; determining a file format coefficient based on the file capacity coefficient and the hash value coefficient; determining the product between the file capacity coefficient and the capacity difference information to obtain a first product; determining the product between the hash value coefficient and the hash difference information to obtain a second product; determining the product between the file format coefficient and the format difference information to obtain a third product; determining the sum of the first product, the second product, and the third product to obtain a total product; taking the square root of the total product to obtain a similarity value; and storing the file to be stored if the similarity value is less than a similarity threshold.

2. A storage device, characterized in that, The storage device is used to perform the method of claim 1, including a message queue, the device comprising: The generation unit is used to generate a file call instruction corresponding to the file storage request in accordance with the storage request response order using the message queue when a file storage request is received from the terminal. The first transmission unit is used to transmit the file retrieval instruction to the terminal, so as to obtain the file to be stored corresponding to the file transmission instruction from the terminal using the file retrieval instruction; A storage unit is used to store the file to be stored.

3. A storage device, characterized in that, The device includes: The system comprises a first memory, a first processor, and a first communication bus. The first memory communicates with the first processor via the first communication bus. The first memory stores a program that can be executed by the first processor. When the program is executed, the method described in claim 1 is performed by the first processor.

4. A storage medium storing a computer program thereon, used in terminals and storage devices, characterized in that, When the computer program is executed by the first processor, it implements the method of claim 1.

Images