Data processing method, device, server and storage medium

Abstract

The embodiment of the present invention discloses a data processing method, device, server and storage medium, wherein the method comprises: in response to a storage request for target slice data, determining a data identifier of target multimedia data to which the target slice data belongs, wherein the data identifier is associated with multiple slice data of the target multimedia data; obtaining slice management information associated with the data identifier; if it is determined according to the slice management information that there are unstored slice data among the multiple slice data, the target slice data is stored in a slice cache area; if it is determined according to the slice management information that the slice data arranged before the target slice data has been stored, determining a target storage area for storing the slice data arranged before the target slice data, and storing the target slice data stored in the slice cache area in the target storage area, thereby improving the accuracy of storing multimedia data.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, device, server and storage medium. Background Art

[0002] With the continuous development of computer technology, the amount of multimedia data uploaded to the server for storage is getting larger and larger. In order to facilitate uploading multimedia data to the server for storage, the client can first slice the multimedia data with a large amount of data, and then upload the sliced ​​data to the server for storage. At present, when uploading the sliced ​​data of multimedia data to the server for storage, most of them are uploaded to the server for storage after uploading the sliced ​​data to the server for storage, and then upload another sliced ​​data to the server for storage. Since the client needs to upload a lot of multimedia data to the server for storage, the server may not be able to effectively distinguish the received sliced ​​data, resulting in errors in the stored multimedia data. Therefore, how to ensure the accuracy of the server's storage of multimedia data has become a current research hotspot. Summary of the invention

[0003] The embodiments of the present invention provide a data processing method, device, server and storage medium, which can improve the accuracy of storing multimedia data.

[0004] On the one hand, an embodiment of the present invention provides a data processing method, including:

[0005] In response to a storage request for target slice data, determining a data identifier of target multimedia data to which the target slice data belongs, wherein the data identifier is associated with a plurality of slice data of the target multimedia data;

[0006] Acquire shard management information associated with the data identifier;

[0007] If it is determined according to the shard management information that among the plurality of shard data, there is shard data that has not been stored in the shard data arranged before the target shard data, then the target shard data is stored in the shard buffer area;

[0008] If it is determined according to the slice management information that the slice data arranged before the target slice data has been stored, a target storage area for storing the slice data arranged before the target slice data is determined, and the target slice data stored in the slice buffer area is stored in the target storage area.

[0009] In another aspect, an embodiment of the present invention provides a data processing device, comprising:

[0010] a determining unit, configured to determine, in response to a storage request for target slice data, a data identifier of target multimedia data to which the target slice data belongs, wherein the data identifier is associated with a plurality of slice data of the target multimedia data;

[0011] An acquisition unit, used to acquire the shard management information associated with the data identifier;

[0012] a storage unit, configured to store the target slice data in a slice buffer area when it is determined according to the slice management information that there is unstored slice data in the slice data arranged before the target slice data among the plurality of slice data;

[0013] The determining unit is further configured to determine a target storage area for storing the slice data arranged before the target slice data if it is determined according to the slice management information that the slice data arranged before the target slice data has been stored;

[0014] The storage unit is further configured to store the target shard data stored in the shard buffer area into the target storage area.

[0015] In another aspect, an embodiment of the present invention provides an intelligent terminal, including a processor, an input device, an output device, and a memory, wherein the processor, the input device, the output device, and the memory are interconnected, wherein the memory is used to store a computer program that supports the terminal to execute the above method, the computer program includes program instructions, and the processor is configured to call the program instructions to execute the following steps:

[0016] In response to a storage request for target slice data, determining a data identifier of target multimedia data to which the target slice data belongs, wherein the data identifier is associated with a plurality of slice data of the target multimedia data;

[0017] Acquire shard management information associated with the data identifier;

[0018] If it is determined according to the shard management information that among the plurality of shard data, there is shard data that has not been stored in the shard data arranged before the target shard data, then the target shard data is stored in the shard buffer area;

[0019] If it is determined according to the slice management information that the slice data arranged before the target slice data has been stored, a target storage area for storing the slice data arranged before the target slice data is determined, and the target slice data stored in the slice buffer area is stored in the target storage area.

[0020] On the other hand, an embodiment of the present invention provides a computer-readable storage medium, in which program instructions are stored. When the program instructions are executed by a processor, they are used to execute the data processing method as described in the first aspect.

[0021] In an embodiment of the present invention, after the client performs segmentation processing on the target segmented data to be stored, the client can upload the obtained multiple segmented data to the server concurrently. After receiving the storage request for the target segmented data from the client, the server can determine the data identifier of the target multimedia data to which the target segmented data belongs, and can obtain the segment management information associated with the data identifier based on the data identifier, so as to determine whether the segment data arranged before the target segmented data among the multiple segmented data associated with the data identifier of the target multimedia data has been stored in the target storage area according to the segment management information. When the server determines that the slice data arranged before the target slice data has not been stored in the target storage area, the target slice data can be stored in the slice cache area first. Furthermore, if the server determines that the slice data arranged before the target slice data has been stored in the target storage area based on the slice management information, it means that the target slice data is the slice data that currently needs to be stored. The target slice data stored in the slice cache area can be stored in the target storage area, thereby ensuring the integrity of the stored multimedia data, and at the same time enabling the server to receive the slice data uploaded by the client while storing the received slice data, thereby improving the storage efficiency of the multimedia data. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0023] Figure 1 is a schematic diagram of a data processing system provided by an embodiment of the present invention;

[0024] Figure 2 is a schematic flow chart of a data processing method provided by an embodiment of the present invention;

[0025] Figure 3a It is a schematic diagram of multiple fragmented data received concurrently provided by an embodiment of the present invention;

[0026] Figure 3b is a schematic diagram of a data processing method provided by an embodiment of the present invention;

[0027] Figure 3cis a schematic diagram of a data processing method provided by an embodiment of the present invention;

[0028] Figure 4 is a schematic flow chart of a data processing method provided by an embodiment of the present invention;

[0029] Figure 5 is a schematic diagram of a data processing method provided by an embodiment of the present invention;

[0030] Figure 6 is a schematic block diagram of a data processing device provided by an embodiment of the present invention;

[0031] Figure 7 It is a schematic block diagram of a server of a data processing method provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The embodiment of the present invention proposes a data processing method, which enables the server to store the fragment data (such as the target fragment data) of the target multimedia data uploaded out of order by the client in the target storage area according to the display order of the target multimedia data, so that the server can realize the concurrent out of order reception of the fragment data of the multimedia data when the target storage area for storing the multimedia data does not support out of order writing, thereby effectively improving the storage efficiency of the multimedia data. In one embodiment, the data processing method can be applied to Figure 1 In the data processing system shown, the data processing system includes a client 10 and a server 11. The client 10 can send segmented data of multimedia data to the server 11 based on the communication connection established with the server 11, wherein the client 10 can receive multimedia data to be stored uploaded by the user, and thus can upload the received multimedia data to be stored to the server 11 for storage based on the communication connection established with the server 11. In one embodiment, when the client 10 establishes a communication connection with the server 11, the client 10 can select the same access point of the server 11 for access, and establish multiple connections with the server 11 to concurrently send multiple segmented data to the server based on the established multiple connections. It can be understood that in order to improve the storage efficiency of multimedia data, after obtaining the multimedia data to be stored, the client 10 can divide the multimedia data into multiple fragments and upload the multiple fragments obtained by division to the server 11 concurrently. After receiving the concurrently uploaded fragment data, the server 11 can respectively determine whether the fragment data before each fragment data (such as the target fragment data) has been stored in the target storage area, thereby realizing the storage of each fragment data.

[0033] In one embodiment, the server 11 can continuously acquire the shard data from the client 10 based on the communication connection with the client 10. While acquiring the shard data from the client 10, the server 11 can also write the acquired shard data into the underlying file system (i.e., the target storage area) for storage in the display order of the corresponding multimedia data to achieve the storage of the multimedia data, so that the server 11 can receive the shard data transmitted by the client 10 while writing the shard data into the target storage area in order for storage, thereby achieving the out-of-order reception and sequential storage of the shard data, thereby effectively improving the storage efficiency of the multimedia data by the server 11. In one embodiment, the shard storage area and the target storage area can be two different storage areas of the server, or the shard storage area can be a storage area in the server, and the target storage area is a storage area corresponding to other storage engines, such as the target storage area is a file storage system in other storage engines.

[0034] In one embodiment, when a client uploads any multimedia data (such as target multimedia data) to a server for storage, if the target multimedia data has a large data volume, the client may first perform segmentation processing on the target multimedia data to obtain multiple segmentation data of the target multimedia data. Furthermore, the client may concurrently upload the obtained multiple segmentation data to the server based on the connection relationship established with the server. When the client uploads the segmentation data to the server, it may first generate a corresponding storage request based on the segmentation data, and send the storage request to the server, wherein the storage request generated based on the segmentation data includes: attribute information of the segmentation data, such as the target multimedia data to which the segmentation data belongs, and the offset value of the segmentation data corresponding to the target multimedia data, etc. In one embodiment, multimedia data with a data volume greater than or equal to 20 megabytes (Mega byte, Mb) can be regarded as multimedia data with a large data volume. The data volume of the target multimedia data can be, for example, 30M or 50M or 2 gigabytes (Giga byte, Gb), etc. The data volume of the sliced ​​data obtained by slicing the target multimedia data by the client can be, for example, 512 kilobytes (Kilo byte, Kb), or 218Kb, etc. The target multimedia data can be document data, file data, audio data, video data, etc.

[0035] In one embodiment, after receiving a storage request for target slice data sent by a client, the server can determine the data identifier corresponding to the target multimedia data to which the target slice data belongs based on the attribute information of the target slice data in the storage request. After the server determines the data identifier corresponding to the target multimedia data, it can obtain the slice management information associated with the data identifier, so as to determine based on the slice management information whether the slice data before the target slice data among the multiple slice data corresponding to the target multimedia data has been stored in the target storage area, wherein the target slice data is randomly selected from the multiple slice data and sent to the server after the client divides the target multimedia data into multiple slice data and disrupts the arrangement order of the multiple slice data corresponding to the target multimedia data. In one embodiment, if the server determines, based on the shard management information associated with the data identifier, that there is unstored shard data in the shard data arranged before the target shard data, in order not to affect the server's subsequent reception of storage requests for shard data from the client, the server may first store the target shard data in the shard cache area, so that the target shard data waits in the shard cache area to be stored in the target storage area; or, if the server determines, based on the shard management information, that all shard data arranged before the target shard data have been stored in the target storage area, the server may directly store the target shard data in the target storage area.

[0036] In one embodiment, if the server determines, based on the shard management information, that the shard data before the target shard data has not been stored in the target storage area, after the server stores the target shard data in the shard cache area, the server may perform real-time detection of the storage status of the shard data before the target shard data in the target storage area, and after the server detects that the shard data before the target shard data has been stored in the target storage area, transfer the target shard data stored in the shard cache area to the target storage area, thereby completing the storage of the target shard data.

[0037] See also Figure 2 , is a schematic flow chart of a data processing method provided by an embodiment of the present invention, such as Figure 2 As shown, the method may include:

[0038] S201 : In response to a storage request for target slice data, determine a data identifier of target multimedia data to which the target slice data belongs, wherein the data identifier is associated with a plurality of slice data of the target multimedia data.

[0039] In one embodiment, after receiving the target multimedia data uploaded by the user, the client may store the target multimedia data in the target storage area. Specifically, after receiving the target multimedia data, the client may first segment the target multimedia data to obtain multiple segment data corresponding to the target multimedia data, so that the multiple segment data can be uploaded to the server concurrently. When the client sends the segment data to the server, it can first generate a corresponding storage request based on the segment data, so that the client can send the storage request to the server together with the segment data, and correspondingly, the server can receive the storage request for the segment data from the client. In one embodiment, after the client processes the target multimedia data to obtain multiple fragmented data, it uses a concurrent upload method to send the fragmented data to the server. Therefore, the storage request for the fragmented data obtained by the server also includes the storage requests corresponding to each fragmented data uploaded to the server by the client. After receiving the storage requests corresponding to each fragmented data uploaded by the client, the server can process the storage requests corresponding to each fragmented data in sequence based on the order in which the storage requests corresponding to each fragmented data are received, and can use the storage request for the fragmented data received first as the storage request for the target fragmented data.

[0040] In one embodiment, if the client performs segmentation processing on the target multimedia data to obtain segmented data, the segmented data respectively include: Figure 3a As shown in the shard data 1 to shard data n, the client can generate corresponding storage requests based on the shard data 1 to shard data n obtained by shard processing, and concurrently send the storage requests corresponding to the shard data to the server, such as Figure 3a As shown, the client can send storage requests generated based on shard data 2, shard data 3 and shard data 9 to the server. If the storage request for the shard data first received by the server is the storage request corresponding to shard data 9, shard data 9 can be used as the target shard data, and the storage request corresponding to shard data 9 can be responded to.

[0041] In one embodiment, after determining the storage request of the target slice data, the server may further determine the data identifier of the target multimedia data to which the target slice data belongs, wherein the storage request of the target slice data includes the attribute information of the target slice data, and the attribute information may be, for example, the slice identifier of the target slice data, and the slice identifier includes: the data range of the target slice data, wherein the data range includes: the starting data of the target slice data corresponds to the starting offset value of the target multimedia data, i.e., range_start, and the ending data in the target slice data corresponds to the ending offset value of the target multimedia data, i.e., range_end. In one embodiment, the slice identifier also includes: the slice number of the target multimedia data corresponding to the target slice data, such as the slice number may be, for example, slice-1, which indicates that the target slice is the first slice of the target multimedia data, or the slice number may also include the data identifier of the target multimedia data, such as the target multimedia data is a document named doc1, and the slice number corresponding to the target multimedia data may also be doc1-slice1. It can be understood that when the server determines the data identifier of the target multimedia data to which the target fragment data belongs, it can first obtain the attribute information of the target fragment data from the storage request, and the attribute information includes the fragment identifier of the target fragment data, such as the above-mentioned fragment number, etc. Further, the server can determine the data identifier of the target multimedia data to which the target fragment data belongs based on the fragment identifier.

[0042] In one embodiment, since the target multimedia data is pre-divided into multiple different slices by the client, the data identifier of the target multimedia data is associated with each slice data of the target multimedia data, that is, based on the data identifier of the target multimedia data, the server can determine multiple slice data of the target multimedia data. Then, when the server receives a storage request for different multimedia data sent by the client, the corresponding slice data can be determined based on the data identifier of the different multimedia data, so as to avoid the situation where the multimedia data to which the slice data belongs cannot be distinguished when multiple slice data from different multimedia data are received, thereby improving the accuracy of the server in distinguishing the corresponding slice data of the multimedia data. After the server determines the data identifier of the target multimedia data to which the target slice data belongs, it can determine the slice management information for the target multimedia data based on the data identifier, thereby realizing the storage of the target slice data, that is, turning to execute step S202.

[0043] S202, obtaining shard management information associated with the data identifier;

[0044] S203: If it is determined according to the shard management information that there is unstored shard data among the plurality of shard data, the target shard data is stored in the shard cache area.

[0045] In step S202 and step S203, after determining the data identifier of the target multimedia data to which the target slice data belongs, the server can obtain the slice management information associated with the data identifier, so as to determine whether the slice data arranged before the target slice data among the multiple slice data of the target multimedia data has been stored in the target storage area based on the slice relationship information, wherein, if the slice data arranged before the target slice data has been stored in the target storage area, it means that the target slice data is exactly the slice data to be stored in the target storage area, and the target slice can be stored in the target storage area; and if the slice data arranged before the target slice data has not been stored in the target storage area, since the target storage area does not support out-of-order storage of multimedia data, the target storage area cannot currently store the target slice data, and the server can temporarily store the target slice data in a temporary slice cache area, so that the target slice data can wait in the temporary slice cache area to be stored in the target storage area.

[0046] In one embodiment, when the server acquires the slice management information, it may acquire it by querying the metadata of the target multimedia data, wherein if the server acquires the metadata of the target multimedia data, the slice management information of the target multimedia data may be determined according to the metadata, and if the server cannot acquire the metadata of the target multimedia data, the server cannot determine the slice management information of the target multimedia data. In one embodiment, the metadata of the target multimedia data records the slice data that has been stored in the target storage area among the multiple slice data of the target multimedia data, and the data amount of the slice data that has been stored in the target storage area; the slice management information includes: the offset value (offset) corresponding to the slice data that has been stored in the target storage area among the multiple slice data of the target multimedia data, and the slice information of the slice data that has been stored in the target storage area, for example, the slice data that has been stored in the target storage area, and the slice identifier of the slice data, etc.

[0047] In one embodiment, when the server obtains the slice management information associated with the data identifier, it may first query the local area of ​​the server, and by querying the metadata of the target multimedia data, determine whether the slice management information of the target multimedia data is obtained according to whether the metadata of the target multimedia data is obtained. In one embodiment, if the server fails to successfully obtain the metadata of the target multimedia data based on the query of the metadata of the target multimedia data in the local area of ​​the server, the metadata of the target multimedia data may be obtained from the remote storage engine associated with the server. It is understandable that the server can obtain the slice management information of the target multimedia data based on the query of the metadata of the target multimedia data stored locally and the query of the metadata of the target multimedia data in the remote storage engine. In one embodiment, the slice management information is generated after at least one slice data of the target multimedia data has been stored in the target storage area. It is understandable that the slice management information is generated after the first slice data of the target multimedia data is stored in the target storage area.

[0048] S204: If it is determined according to the shard management information that the shard data arranged before the target shard data has been stored, determine a target storage area for storing the shard data arranged before the target shard data, and store the target shard data stored in the shard cache area in the target storage area.

[0049] In one embodiment, if the server determines that there is shard data before the target shard data that has not been stored in the target storage area based on the acquired shard management information, the target shard data may be first stored in the shard cache area. Further, after the server stores the target shard data in the shard cache area, if it is determined based on a re-query of the shard management information that the shard data arranged before the target shard data has been stored, the server may transfer the target shard data stored in the shard cache area to the target storage area to achieve storage of the target shard data. In one embodiment, the shard cache area is a shard storage area local to the server for temporarily caching shard data, and the target storage area may be a local storage area of ​​the server or a storage area in a remote storage engine. When the target storage area is a storage area in a remote storage engine, the target storage area may be, for example, a self-developed distributed file storage system (Tencent File System, TFS). In one embodiment, the shard management information is associated with the data identifier of the shard data. After receiving a storage request for the shard data, the server can create a corresponding shard management information for different shard data, and associate the created shard management information with the corresponding shard, so that the server can determine whether the previous shard data has been stored based on the shard management information corresponding to the shard data.

[0050] In one embodiment, the server includes a front-end access module (Front) and a central processing module (Zone). The front-end access module is used to verify the received storage request to determine whether the storage request is legal, and if the storage request is legal, send the storage request to the central processing module, which processes the storage request. The server proposed in the embodiment of the present invention is a server including the front-end access module and the central processing module, or is called a support vector regression machine (SVR). Figure 3b As shown, the client can send a storage request for the target slice data to the request management end of the server (i.e., the above-mentioned front-end access module). After the request management end verifies the target slice data, it can request the task management end (i.e., the above-mentioned central processing module) to search for the slice management information based on the storage request, and set parameters (set param) so that the task management end can search for the slice management information corresponding to the target multimedia data based on the set parameters, wherein the parameter can be, for example, the data identifier of the target multimedia data, etc.

[0051] In one embodiment, the server Figure 3bThe server shown adopts a layered architecture design, which specifically includes request management, task management, and cache management auxiliary modules. The storage engine can be a storage engine in the server or a storage engine in other servers, wherein the storage engine can include a file storage system and a metadata recording module, wherein the file storage system is used to divide the target storage area to store the sliced ​​data, and the metadata recording module is used to record the metadata of the target multimedia data and the metadata (or attribute information) of the sliced ​​data. The request management module is used to receive the storage request for the sliced ​​data uploaded by the client, and feedback the storage result information to the client after storing the sliced ​​data. It can be understood that the request management module mainly processes:

[0052] ① Receive storage requests for shard data;

[0053] ②Call other modules to store shard data;

[0054] ③ Feedback the storage results of the sharded data to the client.

[0055] In one embodiment, the task management module is used to interact with the underlying target storage area, is responsible for storing the shard data in the target storage area, and returns the storage result to the caller through a callback mechanism. In one embodiment, the cache management auxiliary module is used to divide the shard cache area and store the shard data in the shard cache area. The cache auxiliary module can be specifically used to:

[0056] ① Save shard data;

[0057] ② Check the fragment identifier of the fragment data (such as the offset value offset corresponding to the fragment data after it is stored in the target storage area, the offset value range (including the starting offset value range_start and the ending offset value range_end) of the fragment data corresponding to the target multimedia data, and the fragment size (file_size)) to determine whether the fragment identifier is legal information;

[0058] ③ Generate a bitmap management field, such as bitmap_str, to record the shard data stored in the target storage area;

[0059] ④ Provide the server with information such as whether the cache in the shard cache area is full and whether there is shard data that can be stored in the target storage area.

[0060] In one embodiment, the server may further include a main logic template and a temporary buffer, wherein the main logic template is used to:

[0061] ① Receive the fragmented data sent by the request management module;

[0062] ②Store the shard data in the shard cache;

[0063] ③Store the shard data in the shard cache area to the target storage area in order;

[0064] ④After storing the shard data each time, return the storage result to the request management module;

[0065] ⑤Store metadata in the storage engine.

[0066] In one embodiment, the temporary buffer is used to assist the cache management auxiliary module in temporarily storing the shard data. The temporary buffer uses a shard storage area of ​​limited size to store the shard data received by the server before the metadata is queried.

[0067] Among them, since the client adopts a multi-connection concurrent upload method when uploading the sliced ​​data to the server, the server may receive a storage request for the sliced ​​data sent by the client based on other connections after the first slice of the multimedia data is stored in the target storage area and before the metadata is queried. The server needs to obtain the metadata corresponding to the multimedia data before storing the sliced ​​data. This is because if the range_end of the sliced ​​data is less than the offset corresponding to the stored sliced ​​data, it means that the sliced ​​data is duplicated and cannot be stored duplicately. Based on the query of the metadata, the server can determine the way to manage the sliced ​​data, that is, whether to directly store the sliced ​​data or store the sliced ​​data in the slice cache area. In one embodiment, Figure 3b The storage process of the target multimedia data by the server shown is as follows:

[0068] s10, the client segments the target multimedia data to obtain a plurality of segmented data. Furthermore, the client may generate a corresponding storage request based on one or part of the segmented data among the plurality of segmented data, and concurrently send the storage request to the server SVR so that the server stores the part of the segmented data, thereby improving the server's efficiency in receiving the segmented data.

[0069] s11, when the server receives the data storage request uploaded by the client, it calls the request management module to receive the data storage requests uploaded concurrently by the client. After the server calls the request management module and receives multiple data storage requests, when processing the multiple data storage requests received, it can process them sequentially based on the order in which the storage requests are received by the server.

[0070] s12, after the request management module is called by the server and obtains multiple data storage requests uploaded concurrently from the client, it can first determine the first storage request obtained from the received storage requests, so that the request management module can send a query request for shard management information query to the task management module. After receiving the query request for searching shard management information, the task management module can query the local cache from the cache management.

[0071] s13, if the task management module in the server does not find the storage record of the shard data corresponding to the first received storage request (for example, the above-mentioned target shard data) in the local cache management, the shard data corresponding to the first obtained storage request can be stored in a temporary shard cache area.

[0072] s14, after storing the target slice data in the slice cache area, the task management module in the server can further query the metadata of the target multimedia data, so as to obtain the slice management information associated with the data identifier of the target multimedia data based on the query of the metadata, wherein the task manager queries the metadata of the target multimedia data from the storage engine, or the task manager can also determine the metadata of the target multimedia data based on local search.

[0073] s15, after the task management module included in the server obtains the first received storage request, since the corresponding fragment data is the first fragment data uploaded by the client to the server, the task management module cannot obtain the metadata of the target multimedia data.

[0074] s16, when the task management module in the server fails to obtain the metadata of the target multimedia data, the request management module in the server, when subsequently receiving a storage request for the slice data from the client, stores the attribute information of the slice data (or the metadata of the slice data) and the slice data in a temporary slice cache area. Further, the cache management module in the server can subsequently trigger the cache management module to manage the slice data stored in the slice cache area, so as to transfer the slice data in the slice cache area to the storage engine, specifically, to store the slice data in the file storage system of the storage engine.

[0075] s17, when the cache management module in the server transfers the shard data stored in the shard cache area to the storage engine, it can take out the shard data one by one from the shard cache area according to the display order of the target multimedia data, and store them in the target cache area. That is, the cache management module needs to first obtain the first shard data (that is, the first slice) of the target multimedia data from the shard cache area, and store the first slice in the target cache area. Furthermore, the cache management module can sequentially obtain the shard data after the first shard data from the shard cache area and store them in the target cache area.

[0076] In one embodiment, after processing the storage request for the received target slice data and storing the target slice data requested to be stored by the storage request in the slice cache area or the target storage area, the server can obtain storage result information and return the storage result information to the client, wherein the storage result information includes the offset value corresponding to the slice data stored in multiple slice data of the target multimedia data, and the offset value range corresponding to the slice data that can be received by the target storage area. The client can upload the slice data of the target multimedia data that is not stored subsequently to the server for storage based on the offset value returned by the server. In one embodiment, if the server first stores the target slice data in the slice cache area, when the server stores the target slice data in the slice cache area into the target storage area, it can be determined based on the offset corresponding to the slice data stored in the target storage area and the offset corresponding to the target slice data whether the display order of the target multimedia data corresponding to the stored slice data is the next slice of the display data of the target multimedia data; if so, the target slice data is stored in the target storage area, and after the target slice data is stored in the target storage area, the offset corresponding to the slice data stored in the target storage is updated.

[0077] s18, when the cache management module in the server stores the shard data in the shard cache area into the target storage area, if it receives a storage request for other shard data sent by the client, it can store the other shard data in the shard cache area, and then store it in the target storage area later. In one embodiment, when the server stores the shard data in the shard cache area into the target storage area, if it detects that the next shard of the shard data stored in the target storage area does not exist in the shard cache area, the shard data in the shard cache area will not be stored. Among them, after the cache management module stores any shard data, it will feedback the corresponding storage information to the client to inform the client that the corresponding shard data has been stored in the storage engine.

[0078] s19, for the tail data of the target multimedia data, after the server receives the storage request for the tail data of the target multimedia data uploaded by the client, the server can obtain the upload request connection corresponding to the tail data, and after storing the tail data in the target storage area, based on the upload connection, return the Uniform Resource Locator (URL) information of the target multimedia data to the client through the request connection of the last fragment.

[0079] In one embodiment, if a server originally used to process the target multimedia data experiences a single machine failure, a machine restart, or other abnormality, resulting in a change in the server processing the target multimedia data, the changed server needs to first restore the metadata of the multimedia data, and then store the slice data in the target storage area, such as Figure 3c As shown, the server can receive a storage request for a new target segment data sent by the client. Since the remote storage engine has stored other segment data of the target multimedia data, the server can obtain the metadata of the target multimedia data from the remote storage engine. Figure 3b As for the process of storing the target multimedia data by the server shown in FIG. Figure 3c The process of storing the target multimedia data by the server shown in FIG. 1 is the same as the process of obtaining the fragmented data of the target multimedia data from the client. The difference is that Figure 3c In the process shown, when the server obtains metadata from the remote storage engine, it can successfully obtain the metadata of the target multimedia data, so that the server can initialize the shard cache area based on the metadata and store the obtained shard data in the initialized shard cache area. When the shard data in the restored shard cache area is subsequently stored in the storage engine, the above process can also be referred to. Figure 3b The steps in, namely, the above-mentioned steps s17 to s19.

[0080] In an embodiment of the present invention, after the client performs segmentation processing on the target segmented data to be stored, the client can upload the obtained multiple segmented data to the server concurrently. After receiving the storage request for the target segmented data from the client, the server can determine the data identifier of the target multimedia data to which the target segmented data belongs, and can obtain the segment management information associated with the data identifier based on the data identifier, so as to determine whether the segment data arranged before the target segmented data among the multiple segmented data associated with the data identifier of the target multimedia data has been stored in the target storage area according to the segment management information. When the server determines that the slice data arranged before the target slice data has not been stored in the target storage area, the target slice data can be stored in the slice cache area first. Furthermore, if the server determines that the slice data arranged before the target slice data has been stored in the target storage area based on the slice management information, it means that the target slice data is the slice data that currently needs to be stored. The target slice data stored in the slice cache area can be stored in the target storage area, thereby ensuring the integrity of the stored multimedia data, and at the same time enabling the server to receive the slice data uploaded by the client while storing the received slice data, thereby improving the storage efficiency of the multimedia data.

[0081] See also Figure 4 , is a schematic flow chart of a data processing method provided by another embodiment of the present invention, such as Figure 4 As shown, the method may include:

[0082] S401 : In response to a storage request for target slice data, determine a data identifier of target multimedia data to which the target slice data belongs, wherein the data identifier is associated with a plurality of slice data of the target multimedia data.

[0083] In one embodiment, the specific implementation of step S401 can refer to the specific implementation of step S201 in the above embodiment, which will not be described in detail here.

[0084] S402: query metadata of the target multimedia data according to the data identifier, where the metadata records fragment management information of the fragment data associated with the data identifier.

[0085] S403: If the metadata is successfully obtained according to the query result, the shard management information is determined according to the obtained metadata.

[0086] In step S402 and step S403, the steps of acquiring the slice management information by the server in the above embodiment are specifically refined. Specifically, when the server determines the slice management information associated with the data identifier, it can first query the metadata of the target multimedia data according to the data identifier, so as to acquire the slice management information based on the query result. If the query result is that the metadata is acquired, the server can determine the slice management information according to the acquired metadata, or if the query result is that the metadata is not acquired, the server can take the target slice data as the first slice data received and store the target slice data in the slice cache area, wherein the server's acquisition of metadata includes: acquiring metadata based on the server locally, or acquiring metadata based on a remote storage engine. In one embodiment, the slice management information associated with the data identifier of the multimedia data is generated by the server after storing the first slice data (or the first slice data) of the target multimedia data. It can be understood that after receiving multiple slice data uploaded by the client, the server can store the slice data in sequence in the target storage area based on the display order of the slice data corresponding to the target multimedia data, and after the server stores the target multimedia data corresponding to the first slice data displayed first in the display order in the target storage area, it generates metadata of the target multimedia data and records the slice management information for storing the first slice data, such as storing the offset value corresponding to the first slice data.

[0087] In one embodiment, after the server determines the slice management information according to the obtained metadata, if the server obtains a storage request for other slice data associated with the data identifier during the query of the metadata of the target multimedia data according to the data identifier, in order to improve the server's efficiency in receiving the slice data, the access may store the other slice data in the slice cache area, wherein the other slice data is the slice data other than the target slice data among the multiple slice data associated with the data identifier. For example, if the target multimedia data is as described above Figure 3a The target multimedia data shown is fragment data 9. When the server queries the metadata of the target multimedia data according to the data identifier, if a storage request for other fragment data, such as fragment data 7, is obtained, the server can first store fragment data 7 in the fragment cache area.

[0088] In one embodiment, the slice management information is used to indicate whether the slice data before the target slice data has been stored in the target storage area. After obtaining the slice management information, the server can also determine the storage identifier corresponding to the slice data stored in the target storage area and the slice identifier of the target multimedia data based on the slice management information recorded in the metadata, wherein the storage identifier includes: the slice data stored in the target storage area corresponds to the first offset value of the multimedia data, and the slice identifier includes: the target multimedia data corresponds to the second offset value of the target multimedia data. Furthermore, the server can determine whether the slice data arranged before the target slice data has been stored in the target storage area based on the storage identifier and the slice identifier. In a specific implementation, if the server determines that the first offset value is equal to the second offset value, it is determined that the shard data arranged before the target shard data has been stored in the target storage area; or, if the server determines that the first offset value is less than the second offset value, it is determined that the shard data arranged before the target shard data has not been stored in the target storage area; or, if the server determines that the first offset value is greater than the second offset value, it is determined that the target shard data is a duplicate shard data that has been stored in the target storage area, and the server may delete the target shard data to avoid duplicate storage of the target shard data.

[0089] In one embodiment, the shard management information is triggered based on the status identifier of the server. Specifically, the shard management information is obtained when the status identifier is detected to be a metadata query identifier, wherein the status identifier of the server includes: an initial wait identifier (STATE_WAIT_UPLOAD_BLOCK), and the initial wait identifier is used to wait for obtaining the shard management information after receiving a storage request for the target shard data, that is, under the initial wait identifier, the server is waiting for a storage request for the shard data sent by the client. In one embodiment, the corresponding status identifier of the server in the initial state is STATE_WAIT_UPLOAD_BLOCK, and the request module in the server under this status identifier can trigger the main logic module to execute the next step. Alternatively, the state identifier also includes a metadata query identifier (STATE_WAIT_GET_META), which is used to instruct the server to query metadata and obtain the shard management information. The server under the indication of the state identifier waits for metadata query. When the server receives the first shard data sent by the client, it converts the state STATE_WAIT_UPLOAD_BLOCK into STATE_WAIT_GET_META. The server in the metadata query identifier state will query metadata, that is, obtain the offset value corresponding to the shard data stored in the target storage area. After the server obtains the metadata, it can jump to the query result return state.

[0090] In one embodiment, the state identifier also includes a metadata query result return identifier (STATE_ON_GET_META), which is used to indicate the query result return status of the server. The query result return identifier is used to indicate the server to determine whether the shard data arranged before the target shard data has been stored in the target cache area based on the shard management information. The server can generate a temporary shard cache area for storing the shard data based on the result returned after querying the metadata, and can query whether there is shard data stored in the target storage area. When the server detects that there is shard data stored in the target storage area, it can jump to the shard storage state indicated by the shard storage identifier (STATE_RSP_UPLOAD_BLOCK) to store the shard data in the target storage area, or, when there is no shard data that can be stored in the target storage area, it can jump to the state indicated by the STATE_WAIT_UPLOAD_BLOCK identifier to wait for the arrival of storage requests corresponding to other shard data from the client. In one embodiment, the state identifier also includes a shard storage identifier (STATE_REQ_UPLOAD_BLOCK), which is used to instruct the server to store the target shard data in the target cache area. The identifier is used to trigger the server to write the shard data into the target storage area (such as TFS) for storage. When the cache management object of the server has been generated and the current state is STATE_WAIT_UPLOAD_BLOCK, and there is shard data stored in the target storage area, the current state of the server is switched to the state indicated by the STATE_REQ_UPLOAD_BLOCK identifier. After the server writes the shard data into the target storage area, it can further jump to the state indicated by the STATE_RSP_UPLOAD_BLOCK identifier. In addition, after the server writes the server shard data into the target storage area, it can also start the update state for metadata, that is, the state indicated by STATE_RSP_UPLOAD_BLOCK, so as to update the metadata, and after the update is completed, it can jump to the state indicated by the STATE_ON_SET_META identifier.

[0091] In one embodiment, after acquiring the shard management information, the server may determine whether the shard data before the target shard data has been stored in the target storage area based on the shard management information, that is, turn to execute step S404.

[0092] S404: if it is determined according to the slice management information that there is unstored slice data among the plurality of slice data, the target slice data is stored in a slice cache area.

[0093] S405: If it is determined according to the shard management information that the shard data arranged before the target shard data has been stored, determine a target storage area for storing the shard data arranged before the target shard data, and store the target shard data stored in the shard cache area in the target storage area.

[0094] In step S404 and step S405, after storing the target shard data in the target storage area, the server may first obtain storage result information, and thus may send the storage identifier to the client, so that the client sends the shard data arranged after the target shard data based on the storage identifier, wherein the storage result information includes: a storage identifier corresponding to the target shard data after storing the target shard data in the target storage area, the storage identifier including the offset value of the shard data to be stored, the shard number of the stored shard data, etc. After receiving the storage identifier, the client may determine which uploaded shard data has been stored, so that when uploading shard data subsequently, the stored shard data will no longer be repeatedly sent to the server. In one embodiment, based on the storage requests for segmented data continuously sent by the client to the server, the server may receive a storage request for the tail data of the target multimedia data, wherein the tail data is the last data slice of the target multimedia data. After receiving the storage request for the tail data, the server may determine the request path for initiating the segmented data storage request, thereby storing the tail data in the target storage area and then feeding back the storage result information generated after storing the tail data to the client based on the request path, wherein the storage result information includes URL information for the target multimedia data, etc.

[0095] In one embodiment, see Figure 5The schematic diagram shown is when the server stores the target shard data. After the server obtains the target shard data according to the storage request of the target shard data, it can first call the metadata query code (STATE_WAIT_GET_META) to query the metadata. Further, the server can enter the query result return state, that is, the state indicated by STATE_ON_GET_META. While the server is waiting for the query result for the metadata, if other shard data is obtained, the other shard data can be stored in the shard cache area. When the server queries the metadata, if the query response information is not obtained, it jumps to the timeout return state STATE_TIMEOUT. In one embodiment, if the server obtains the metadata, it can process the shard data in the shard cache area to store the shard data in the shard cache area in the target storage area. Specifically, when the server stores the shard data in the shard cache area in the target storage area, it needs to first determine whether there is shard data in the shard cache area that can be stored in the target storage area, that is, whether there is a subsequent piece of data of the shard data stored in the target storage area. If the server determines that there is such data in the shard cache area, the server jumps to the shard data storage state, that is, the state indicated by STATE_RSP_UPLOAD_BLOCK, and stores the shard data in the target storage area. Furthermore, the server may jump to the state indicated by the STATE_RSP_UPLOAD_BLOCK shard storage identifier to obtain the storage identifier corresponding to the storage of the shard data. In addition, when the server stores the shard data in the target storage area, if it detects the arrival of other shard data, it may store the other shard data in the shard buffer area, and return the storage result information after storing the shard data to indicate whether the server has completed the storage of the shard data. In one embodiment, if the server determines that there is no shard data that can be stored in the disk, the server switches to the initial waiting state to wait for other shard data, and after obtaining other shard data, continues to determine whether there is shard data that can be stored in the disk.

[0096] In an embodiment of the present invention, after receiving a storage request from a client for target shard data, the server can determine the data identifier of the target multimedia data to which the target shard data belongs, and can query the metadata of the target multimedia data according to the data identifier to determine the shard management information corresponding to the target multimedia data. If the server determines according to the shard management information that there are unstored shard data in the shard data arranged before the target shard data among the multiple shard data, the target shard data is stored in the shard cache area, or if it is determined according to the shard management information that the shard data arranged before the target shard data has been stored, the shard data stored in the shard cache area can be transferred to the target storage area, so that the server can improve the storage efficiency of the shard data to the disk efficiency of the target storage area under the network condition of a pure intranet, thereby effectively improving the storage efficiency of the server.

[0097] Based on the description of the above data processing method embodiment, the present invention also proposes a data processing device, which may be a computer program (including program code) running on the above server. The data processing device may be used to execute the following steps: Figure 2 and Figure 4 For the data processing method described, see Figure 6 The data processing device includes: a determining unit 601, an acquiring unit 602 and a storage unit 603.

[0098] A determining unit 601 is configured to determine, in response to a storage request for target slice data, a data identifier of target multimedia data to which the target slice data belongs, wherein the data identifier is associated with a plurality of slice data of the target multimedia data;

[0099] An acquisition unit 602 is used to acquire the slice management information associated with the data identifier;

[0100] The storage unit 603 is configured to store the target slice data in a slice buffer area when it is determined according to the slice management information that there is unstored slice data in the slice data arranged before the target slice data among the plurality of slice data;

[0101] The determining unit 601 is further configured to determine a target storage area for storing the slice data arranged before the target slice data if it is determined according to the slice management information that the slice data arranged before the target slice data has been stored;

[0102] The storage unit 603 is further configured to store the target shard data stored in the shard cache area into the target storage area.

[0103] In one embodiment, the acquisition unit 602 is specifically configured to:

[0104] querying metadata of the target multimedia data according to the data identifier, wherein the metadata records slice management information of the slice data associated with the data identifier;

[0105] If the metadata is successfully obtained according to the query result, the shard management information is determined according to the obtained metadata.

[0106] In one embodiment, the storage unit 603 is further configured to store the target shard data in a shard cache area if the metadata cannot be obtained according to the query result.

[0107] In one embodiment, the storage unit 603 is further configured to store the other fragment data in the fragment cache area if a storage request for other fragment data associated with the data identifier is obtained during the query of the metadata of the target multimedia data according to the data identifier;

[0108] The other fragment data are fragment data other than the target fragment data among the multiple fragment data associated with the data identifier.

[0109] In one embodiment, the shard management information is used to indicate whether the shard data before the target shard data has been stored in the target storage area.

[0110] The determining unit 601 is further configured to determine the storage identifier corresponding to the fragment data stored in the target storage area and the fragment identifier of the target multimedia data according to the fragment management information recorded in the metadata;

[0111] The determining unit 601 is further configured to determine, based on the storage identifier and the fragment identifier, whether the fragment data arranged before the target fragment data has been stored in the target storage area.

[0112] In one embodiment, the storage identifier includes: the first offset value of the multimedia data corresponding to the fragment data stored in the target storage area, and the fragment identifier includes: the second offset value of the target multimedia data corresponding to the target multimedia data; the determination unit 601 is specifically used to:

[0113] If the first offset value is equal to the second offset value, it is determined that the slice data arranged before the target slice data has been stored in the target storage area;

[0114] If the first offset value is smaller than the second offset value, it is determined that the fragment data arranged before the target fragment data is not stored in the target storage area;

[0115] If the first offset value is greater than the second offset value, it is determined that the target slice data is duplicate slice data that has been stored in the target storage area, and the target slice data is deleted.

[0116] In one embodiment, the storage request includes attribute information of the target shard data, and the determining unit 601 is specifically configured to:

[0117] Acquire attribute information of the target shard data from the storage request, the attribute information including a shard identifier of the target shard data;

[0118] According to the slice identifier, a data identifier of the target multimedia data to which the target slice data belongs is determined.

[0119] In one embodiment, the device further includes: a sending unit 604.

[0120] The acquisition unit 602 is further configured to acquire storage result information, wherein the storage result information includes: a storage identifier corresponding to storing the target shard data in the target storage area, wherein the storage identifier includes an offset value of the shard data to be stored;

[0121] The sending unit 604 is configured to send the storage identifier to the client, so that the client sends the fragment data arranged after the target fragment data based on the storage identifier.

[0122] In one embodiment, the determining unit 601 is further configured to determine, in response to a storage request for tail data, a request path corresponding to the storage request, wherein the tail data is the last fragment data of the target multimedia data;

[0123] The storage unit 603 is further configured to store the tail data in the target storage area, and send storage result information generated after storing the tail data to the client based on the request path.

[0124] In one embodiment, the shard management information is acquired based on a status indicator of the server;

[0125] The state identifier includes an initial waiting identifier, and the initial waiting identifier is used to wait for obtaining the slice management information after receiving a storage request for the target slice data.

[0126] In one embodiment, the status identifier further includes a metadata query identifier, and the metadata query identifier is used to instruct the server to query metadata and obtain the shard management information.

[0127] In one embodiment, the status identifier also includes a metadata query result return identifier, and the query result return identifier is used to instruct the server to determine whether the shard data arranged before the target shard data has been stored in the target cache area based on the shard management information.

[0128] In one embodiment, the state identifier further includes a shard storage identifier, and the shard storage identifier is used to instruct the server to store the target shard data in the target cache area.

[0129] In an embodiment of the present invention, after the client performs segmentation processing on the target segmented data to be stored, the client can upload the obtained multiple segmented data concurrently. After receiving the storage request for the target segmented data from the client, the determination unit 601 can determine the data identifier of the target multimedia data to which the target segmented data belongs, and the determination unit 601 can obtain the segment management information associated with the data identifier based on the data identifier, so as to determine whether the segment data arranged before the target segmented data among the multiple segmented data associated with the data identifier of the target multimedia data has been stored in the target storage area according to the segment management information. When the determination unit 601 determines that the slice data arranged before the target slice data has not been stored in the target storage area, the storage unit 603 may first store the target slice data in the slice cache area. Further, if the determination unit 601 determines that the slice data arranged before the target slice data has been stored in the target storage area based on the slice management information, it means that the target slice data is the slice data that currently needs to be stored. The storage unit 603 may store the target slice data stored in the slice cache area in the target storage area, thereby ensuring the integrity of the stored multimedia data, and at the same time realizing the storage of the received slice data while receiving the slice data uploaded by the client, thereby improving the storage efficiency of the multimedia data.

[0130] See also Figure 7 , is a schematic block diagram of the structure of a server provided in an embodiment of the present invention. The server may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms. The above-mentioned client may be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, etc., but is not limited to this. The client and the server may be directly or indirectly connected via wired or wireless communication, which is not limited in this application. In one embodiment, Figure 7The server in the embodiment shown may include: one or more processors 701; one or more input devices 702, one or more output devices 703 and a memory 704. The processor 701, the input device 702, the output device 703 and the memory 704 are connected via a bus 705. The memory 704 is used to store a computer program, the computer program includes program instructions, and the processor 701 is used to execute the program instructions stored in the memory 704.

[0131] The memory 704 may include a volatile memory, such as a random-access memory (RAM); the memory 704 may also include a non-volatile memory, such as a flash memory, a solid-state drive (SSD), etc.; the memory 704 may also include a combination of the above types of memory.

[0132] The processor 701 may be a central processing unit (CPU). The processor 701 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), etc. The PLD may be a field-programmable gate array (FPGA), a generic array logic (GAL), etc. The processor 701 may also be a combination of the above structures.

[0133] In the embodiment of the present invention, the memory 704 is used to store a computer program, the computer program includes program instructions, and the processor 701 is used to execute the program instructions stored in the memory 704 to implement the above-mentioned Figure 2 and Figure 4 The steps of the corresponding method in .

[0134] In one embodiment, the processor 701 is configured to call the program instructions to execute:

[0135] In response to a storage request for target slice data, determining a data identifier of target multimedia data to which the target slice data belongs, wherein the data identifier is associated with a plurality of slice data of the target multimedia data;

[0136] Acquire the shard management information associated with the data identifier, and when it is determined according to the shard management information that among the plurality of shard data, there is shard data that has not been stored in the shard data arranged before the target shard data, store the target shard data in the shard cache area;

[0137] If it is determined according to the slice management information that the slice data arranged before the target slice data has been stored, a target storage area for storing the slice data arranged before the target slice data is determined, and the target slice data stored in the slice buffer area is stored in the target storage area.

[0138] In one embodiment, the processor 701 is configured to call the program instructions to execute:

[0139] querying metadata of the target multimedia data according to the data identifier, wherein the metadata records slice management information of the slice data associated with the data identifier;

[0140] If the metadata is successfully obtained according to the query result, the shard management information is determined according to the obtained metadata.

[0141] In one embodiment, the processor 701 is configured to call the program instructions to execute:

[0142] If the metadata fails to be obtained according to the query result, the target shard data is stored in the shard cache area.

[0143] In one embodiment, the processor 701 is configured to call the program instructions to execute:

[0144] In the process of querying the metadata of the target multimedia data according to the data identifier, if a storage request for other fragment data associated with the data identifier is obtained, the other fragment data is stored in the fragment cache area;

[0145] The other fragment data are fragment data other than the target fragment data among the multiple fragment data associated with the data identifier.

[0146] In one embodiment, the shard management information is used to indicate whether shard data before the target shard data has been stored in the target storage area, and the processor 701 is configured to call the program instruction to execute:

[0147] Determine, according to the slice management information recorded in the metadata, the storage identifier corresponding to the slice data stored in the target storage area and the slice identifier of the target multimedia data;

[0148] According to the storage identifier and the slice identifier, it is determined whether the slice data arranged before the target slice data has been stored in the target storage area.

[0149] In one embodiment, the storage identifier includes: the first offset value of the multimedia data corresponding to the slice data stored in the target storage area, and the slice identifier includes: the second offset value of the target multimedia data corresponding to the target multimedia data; the processor 701 is configured to call the program instruction to execute:

[0150] If the first offset value is equal to the second offset value, it is determined that the slice data arranged before the target slice data has been stored in the target storage area;

[0151] If the first offset value is smaller than the second offset value, it is determined that the fragment data arranged before the target fragment data is not stored in the target storage area;

[0152] If the first offset value is greater than the second offset value, it is determined that the target slice data is duplicate slice data that has been stored in the target storage area, and the target slice data is deleted.

[0153] In one embodiment, the storage request includes attribute information of the target shard data, and the processor 701 is configured to call the program instruction to execute:

[0154] Acquire attribute information of the target shard data from the storage request, the attribute information including a shard identifier of the target shard data;

[0155] According to the slice identifier, a data identifier of the target multimedia data to which the target slice data belongs is determined.

[0156] In one embodiment, the processor 701 is configured to call the program instructions to execute:

[0157] Acquire storage result information, the storage result information including: a storage identifier corresponding to storing the target shard data in the target storage area, the storage identifier including an offset value of the shard data to be stored;

[0158] The storage identifier is sent to the client, so that the client sends the fragment data arranged after the target fragment data based on the storage identifier.

[0159] In one embodiment, the processor 701 is configured to call the program instructions to execute:

[0160] In response to a storage request for tail data, determining a request path corresponding to the storage request, the tail data being the last fragment data of the target multimedia data;

[0161] The tail data is stored in the target storage area, and storage result information generated after storing the tail data is sent to the client based on the request path.

[0162] In one embodiment, the shard management information is acquired based on a status indicator of the server;

[0163] The state identifier includes an initial waiting identifier, and the initial waiting identifier is used to wait for obtaining the slice management information after receiving a storage request for the target slice data.

[0164] In one embodiment, the status identifier further includes a metadata query identifier, and the metadata query identifier is used to instruct the server to query metadata and obtain the shard management information.

[0165] In one embodiment, the status identifier also includes a metadata query result return identifier, and the query result return identifier is used to instruct the server to determine whether the shard data arranged before the target shard data has been stored in the target cache area based on the shard management information.

[0166] In one embodiment, the state identifier further includes a shard storage identifier, and the shard storage identifier is used to instruct the server to store the target shard data in the target cache area.

[0167] The embodiment of the present invention provides a computer program product or a computer program, the computer program product or the computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the above-mentioned Figure 2 or Figure 4 The method embodiment shown in the figure. Wherein, the computer-readable storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0168] What is disclosed above is only a partial embodiment of the present invention, which certainly cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of implementing the above embodiments and making equivalent changes according to the claims of the present invention still fall within the scope of the invention.

Claims

1. A data processing method, characterized in that: include: In response to a storage request for target slice data, query metadata of target multimedia data to which the target slice data belongs, and enter a query result waiting state of the server until the metadata is obtained, and determine a data identifier of the target multimedia data to which the target slice data belongs according to the metadata, wherein the data identifier is associated with a plurality of slice data of the target multimedia data; wherein other slice data obtained during the metadata query process is stored in a slice cache area; Acquire shard management information associated with the data identifier; If it is determined according to the shard management information that there is unstored shard data among the multiple shard data, the target shard data is stored in the shard cache area; wherein, when there is no shard data that can be stored in the target storage area in the shard cache area, the server jumps to a waiting state for the query result; If it is determined according to the shard management information that the shard data arranged before the target shard data has been stored, a target storage area for storing the shard data arranged before the target shard data is determined, the server jumps into the shard data storage state, obtains the storage identifier of the target shard data, and stores the target shard data stored in the shard cache area into the target storage area according to the storage identifier.

2. The method according to claim 1, characterized in that: The acquiring of the shard management information associated with the data identifier includes: querying metadata of the target multimedia data according to the data identifier, wherein the metadata records slice management information of the slice data associated with the data identifier; If the metadata is successfully obtained according to the query result, the shard management information is determined according to the obtained metadata.

3. The method according to claim 2, characterized in that The method further comprises: If the metadata fails to be obtained according to the query result, the target shard data is stored in the shard cache area.

4. The method according to claim 2, characterized in that: The method further comprises: In the process of querying the metadata of the target multimedia data according to the data identifier, if a storage request for other fragment data associated with the data identifier is obtained, the other fragment data is stored in the fragment cache area; The other fragment data are fragment data other than the target fragment data among the multiple fragment data associated with the data identifier.

5. The method according to claim 2, characterized in that: The shard management information is used to indicate whether the shard data before the target shard data has been stored in the target storage area, and the method further includes: Determine, according to the slice management information recorded in the metadata, the storage identifier corresponding to the slice data stored in the target storage area and the slice identifier of the target multimedia data; According to the storage identifier and the slice identifier, it is determined whether the slice data arranged before the target slice data has been stored in the target storage area.

6. The method according to claim 5, characterized in that The storage identifier includes: the first offset value of the multimedia data corresponding to the slice data stored in the target storage area, and the slice identifier includes: the second offset value of the target multimedia data corresponding to the target multimedia data; and determining whether the slice data arranged before the target slice data has been stored in the target storage area according to the storage identifier and the slice identifier includes: If the first offset value is equal to the second offset value, it is determined that the slice data arranged before the target slice data has been stored in the target storage area; If the first offset value is smaller than the second offset value, it is determined that the fragment data arranged before the target fragment data is not stored in the target storage area; If the first offset value is greater than the second offset value, it is determined that the target slice data is duplicate slice data that has been stored in the target storage area, and the target slice data is deleted.

7. The method according to claim 1, characterized in that The storage request includes attribute information of the target fragment data, and the step of determining a data identifier of the target multimedia data to which the target fragment data belongs includes: Acquire attribute information of the target shard data from the storage request, the attribute information including a shard identifier of the target shard data; According to the slice identifier, a data identifier of the target multimedia data to which the target slice data belongs is determined.

8. The method according to claim 1, characterized in that: After storing the target shard data stored in the shard cache area in the target storage area, the method further includes: Acquire storage result information, the storage result information including: a storage identifier corresponding to storing the target shard data in the target storage area, the storage identifier including an offset value of the shard data to be stored; The storage identifier is sent to the client, so that the client sends the fragment data arranged after the target fragment data based on the storage identifier.

9. The method according to claim 1, characterized in that: The method further comprises: In response to a storage request for tail data, determining a request path corresponding to the storage request, the tail data being the last fragment data of the target multimedia data; The tail data is stored in the target storage area, and storage result information generated after storing the tail data is sent to the client based on the request path.

10. The method according to claim 1, characterized in that The shard management information is acquired based on the triggering of the server status identifier; The state identifier includes an initial waiting identifier, and the initial waiting identifier is used to wait for obtaining the slice management information after receiving a storage request for the target slice data.

11. The method according to claim 10, characterized in that The status identifier also includes a metadata query identifier, and the metadata query identifier is used to instruct the server to query metadata and obtain the slice management information.

12. The method according to claim 10, characterized in that The status identifier also includes a metadata query result return identifier, and the query result return identifier is used to instruct the server to determine whether the shard data arranged before the target shard data has been stored in the target cache area based on the shard management information.

13. The method according to claim 10, characterized in that The state identifier also includes a shard storage identifier, and the shard storage identifier is used to instruct the server to store the target shard data in a target cache area.

14. A data processing device, characterized in that: include: A determination unit, configured to query metadata of target multimedia data to which the target slice data belongs in response to a storage request for target slice data, and enter a query result waiting state of a server until the metadata is obtained, and determine a data identifier of the target multimedia data to which the target slice data belongs based on the metadata, wherein the data identifier is associated with a plurality of slice data of the target multimedia data; wherein other slice data obtained during the metadata query process is stored in a slice cache area; An acquisition unit, used to acquire the shard management information associated with the data identifier; A storage unit, configured to store the target shard data in a shard cache area when it is determined according to the shard management information that there is unstored shard data in the shard data arranged before the target shard data among the plurality of shard data; wherein when there is no shard data that can be stored in the target storage area in the shard cache area, the server jumps to a waiting state for the query result; The determining unit is further configured to determine a target storage area for storing the slice data arranged before the target slice data if it is determined according to the slice management information that the slice data arranged before the target slice data has been stored; the server jumps into a slice data storage state and obtains a storage identifier of the target slice data, The storage unit is further configured to store the target shard data stored in the shard cache area into the target storage area according to the storage identifier.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the method according to any one of claims 1 to 13.

Images