Data processing method and electronic equipment
By using the pointer group management strategy of the data buffer in the first electronic device, the problem of data access delay in the network file system is solved, the data access speed and user experience are improved, and efficient data processing and memory utilization are achieved.
Patent Information
- Application Number
- CN202410943469.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-15
- Publication Date
- 2024-12-06
AI Technical Summary
There is network transmission delay in accessing files from other devices in the network file system, which affects real-time and efficiency.
By establishing a pointer group of data buffers in the first electronic device, including a first pointer and a second pointer, for managing a plurality of preset sized data blocks, and determining the management strategy of the pointer group according to the data access method of the target application, Including whether the first pointer is moved to free the location of the data block, a data buffer with a loop structure is adopted to maximize memory utilization, and reduce data processing overhead and delay.
Improves the speed and responsiveness of data access, adapts to the different access needs of target applications, reduces loading time, and improves user experience and system performance.
Smart Images

Figure CN119088770A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of data access, and in particular to a data processing method and an electronic device. Background Art
[0002] The network file system allows users to access files on other devices in the network file system through local applications (such as players, text editors, etc.) just like accessing local files, such as in multi-device wireless interconnection scenarios. However, since there is a problem of network transmission delay when accessing files on other devices, it is necessary to improve the real-time and efficiency of accessing files on interconnected devices. Summary of the invention
[0003] The present disclosure provides a data processing method, an electronic device and a storage medium to at least solve the above technical problems existing in the prior art.
[0004] According to a first aspect of the present disclosure, a data processing method is provided, which is applied to a first electronic device, wherein the first electronic device includes a data buffer capable of caching a plurality of data blocks of a preset size from a target file of a second electronic device, the method comprising:
[0005] Establishing a pointer group of the data buffer, wherein the pointer group includes a first pointer and a second pointer, the first pointer indicates the head position of the multiple data blocks stored in the data buffer; the second pointer indicates the tail position of the multiple data blocks stored in the data buffer;
[0006] Determine a data access mode of a target application to a plurality of data blocks in the data buffer;
[0007] A management strategy for the pointer group is determined based on the data access mode, and the management strategy includes whether to move the first pointer to release the location of the accessed data block.
[0008] According to a second aspect of the present disclosure, there is provided a first electronic device, the first electronic device comprising: a data buffer capable of caching a plurality of data blocks of a preset size from a target file of a second electronic device;
[0009] A first processing module is used to establish a pointer group of the data buffer, wherein the pointer group includes a first pointer and a second pointer, the first pointer indicates the head position of the multiple data blocks stored in the data buffer; the second pointer indicates the tail position of the multiple data blocks stored in the data buffer;
[0010] A second processing module is used to determine the data access mode of the target application to the multiple data blocks in the data buffer;
[0011] The third processing module is used to determine a management strategy for the pointer group based on the data access mode, where the management strategy includes whether to move the first pointer to release the position of the data block accessed by the target application.
[0012] According to a third aspect of the present disclosure, there is provided an electronic device, including:
[0013] a data buffer capable of caching a plurality of data blocks of a preset size from a target file of a second electronic device;
[0014] at least one processor; and
[0015] a memory communicatively connected to the at least one processor; wherein,
[0016] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present disclosure.
[0017] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which:
[0019] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.
[0020] Figure 1 A schematic diagram of an implementation flow of a data processing method according to an embodiment of the present disclosure;
[0021] Figure 2 A schematic diagram of a data buffer area according to an embodiment of the present disclosure;
[0022] Figure 3 A schematic diagram of a first pointer and a second pointer according to an embodiment of the present disclosure;
[0023] Figure 4 is a schematic diagram of another first pointer and a second pointer according to an embodiment of the present disclosure;
[0024] Figure 5 A schematic diagram of moving a first pointer according to an embodiment of the present disclosure;
[0025] Figure 6 A schematic diagram of a first pointer, a second pointer, and an intermediate point according to an embodiment of the present disclosure;
[0026] Figure 7 is a schematic diagram of another method of moving a first pointer according to an embodiment of the present disclosure;
[0027] Figure 8 A schematic diagram of an implementation flow of a data access method according to an embodiment of the present disclosure;
[0028] Fig. 9 is a schematic structural diagram of a first electronic device according to an embodiment of the present disclosure;
[0029] Fig.10 The figure is a schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0030] In order to make the purpose, features, and advantages of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.
[0031] In the following description, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.
[0032] In the following description, the terms "first\second" involved are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present disclosure described herein can be implemented in an order other than that illustrated or described herein.
[0033] Unless otherwise defined, all technical and scientific terms used in this disclosure have the same meaning as those commonly understood by those skilled in the art to which this disclosure belongs. The terms used in this disclosure are only for the purpose of describing the embodiments of this disclosure and are not intended to limit this disclosure.
[0034] It should be understood that in the various embodiments of the present disclosure, the size of the serial number of each implementation process does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present disclosure.
[0035] like Figure 1As shown, an embodiment of the present disclosure provides a data processing method, which is applied to a first electronic device, wherein the first electronic device includes a data buffer capable of caching a plurality of data blocks of a preset size from a target file of a second electronic device, and the method includes:
[0036] Step 101: Establish a pointer group of the data buffer, wherein the pointer group includes a first pointer and a second pointer, the first pointer indicates the head position of multiple data blocks stored in the data buffer; the second pointer indicates the tail position of multiple data blocks stored in the data buffer;
[0037] Step 102, determining a data access mode of a target application to a plurality of data blocks in the data buffer;
[0038] Step 103: Determine a management strategy for the pointer group based on the data access mode, where the management strategy includes whether to move the first pointer to release the location of the accessed data block.
[0039] In some embodiments, the first electronic device establishes an Internet connection with the second electronic device. In some embodiments, the Internet can be a combination of a wired network and a wireless connection. In some embodiments, it can be a separate wireless network connection, such as a WIFI connection, a Bluetooth connection, etc.
[0040] In some embodiments, the number of data blocks that can be stored in the data buffer and the size of the data blocks can be set based on actual conditions, for example, can be set in combination with the performance of the first electronic device, the remaining storage space, etc.
[0041] The data buffer size is recorded as: N x length, where N represents the number of data blocks that can be stored, and length represents the size or length of the data block. N is greater than or equal to 1. For example, Figure 2 As shown, an example of a data buffer is provided, where N is 200 and length is 128kb; the data buffer size is: 200x128kb, that is, the data buffer can store 200 data blocks at the same time. Here, the size of N and length is only an example, and there is no limitation on the size of N and length.
[0042] Figure 2 In the example, the offset refers to the offset of the starting position (x) of the data block in the data buffer or target file relative to the entire storage space. This is because the data block does not necessarily start from the starting position in its storage area (e.g., the data block is not necessarily 0-128kb, 128-256kb), but starts from a certain offset (e.g., the data block can be 2-130kb, 3-131kb, etc.).
[0043] Specifically, the data buffer adopts a byte array with a circular structure, which may also be called a circular buffer, a ring buffer, etc. When storing data blocks, the first bits of each data block are stored adjacently.
[0044] The first pointer in the pointer group indicates the head position of all the multiple data blocks stored in the data buffer, and the second pointer indicates the tail position of all the multiple data blocks stored in the data buffer; and when there is free space in the data buffer, the free space will be used to continue to store data blocks.
[0045] like Figure 3 and Figure 4 As shown, two situations of a first pointer and a second pointer are provided.
[0046] like Figure 3 As shown, HEAD indicates the position of the first pointer, TAIL indicates the position of the second pointer, the area between HEAD and TAIL is the saved data block, and the blank area is the free space of the data buffer.
[0047] like Figure 4 As shown in the figure, the data buffer uses a byte array with a circular structure. Its internal data structure allows data to be stored in a circular manner. That is, if the end (tail) of the data buffer is reached when saving a data block, it can return to the starting position (head) of the data buffer to continue storing, forming a circular utilization process. Figure 3 similar, Figure 4 In another case, the data block between HEAD and TAIL is saved, and the blank area between TAIL and HEAD is the free space of the data buffer.
[0048] In the disclosed embodiment, a data buffer with a circular structure is used and the data buffer is cyclically utilized to maximize the utilization of memory space; during the loop process, the copying or moving operations of the data block can be avoided by managing the first pointer and the second pointer, thereby reducing the overhead and delay of data processing, and further improving the speed and response performance of data access.
[0049] In some embodiments, determining the data access mode of the target application to the multiple data blocks in the data buffer includes:
[0050] Determine whether the read position of the target application is the same as the position indicated by the first pointer; the read position is the position of the data block accessed by the target application in the data buffer;
[0051] If the read position is different from the position of the first pointer, determining that the data access mode is the first mode;
[0052] If the read position is the same as the position of the first pointer, it is determined that the data access mode is the second mode.
[0053] Here, the target application is the upper-layer application of the data buffer, which can acquire and present the data blocks of the data buffer. For example, the data blocks of the video files saved in the data buffer, the target application can be a video player. For another example, the data blocks of the text files saved in the data buffer, the target application can be a text browser.
[0054] The target application may access the data of the multiple data blocks in the data buffer in various ways, such as sequential access and random (disordered) access.
[0055] Sequential access refers to the method of reading from the data buffer in the order in which the data blocks are stored in the data buffer. The operation process of sequential access is linear, that is, the order of reading is consistent with the order in which the data blocks are stored in the data buffer.
[0056] Random (out-of-order) access refers to the way that data blocks at any location can be read in any order as needed. Unlike sequential access, which requires accessing data blocks one by one, random access allows you to locate and operate on data blocks in the target file by specifying an offset or index in the target file.
[0057] Here, the first pointer indicates the head position of multiple data blocks stored in the data buffer. If the read position of the target application is different from the position indicated by the first pointer, and the read position points to the position of a data block between the first pointer and the second pointer, then the data access mode is determined to be random access, that is, the first mode; if the read position is the same as the position of the first pointer, that is, the read position points to the position of the first data block in the data buffer, then the data access mode is determined to be sequential access, that is, the second mode.
[0058] In some embodiments, determining a management strategy of the pointer group based on the data access mode includes:
[0059] If the data access mode is the first mode, managing the first pointer based on a first strategy, the first strategy including determining whether to move the position of the first pointer based on the read position;
[0060] If the data access mode is the second mode, the first pointer is managed based on a second strategy, and the second strategy includes moving the position of the first pointer to the next data block to release the position of the accessed data block.
[0061] Here, a plurality of strategies for managing the pointer group are preset, including: a first strategy for the first method and a second strategy for the second method.
[0062] If it is determined that the data access mode is sequential access (second mode), based on the second strategy, after sending the accessed data block to the target application, the position of the first pointer is moved to the next data block to release the position of the accessed data block stored in the data buffer.
[0063] Specifically, if the target application performs data access in sequence, that is, the read position is equal to the position indicated by the first pointer, the data buffer directly provides the data block and moves the first pointer to the next data block to release the position of the accessed data block. Here, the position of the first pointer can be moved according to the following formula HEAD = (HEAD + readSize) % TOTAL_SIZE; where readSize represents the position of the accessed data block, TOTAL_SIZE represents the size of the data buffer, and % represents the modulo operator, which ensures that HEAD circulates in the data buffer. Figure 5 As shown, an example of moving the first pointer is provided. Assuming that data block 1 is the accessed data block, the data buffer provides data block 1 and moves the first pointer to the next data block, namely data block 2, to release the position of the accessed data block, namely data block 1.
[0064] If it is determined that the data access mode is random access (first mode), based on the first strategy, it can be determined whether to move the position of the first pointer based on the reading position.
[0065] In the disclosed embodiment, different management strategies are adopted for different data access methods, which can adapt to different access requirements of the target application (such as the first method and the second method), and then efficiently utilize the buffered data to provide the target application with fast, accurate and smooth data, reduce loading time, and improve user experience and overall system performance.
[0066] In some embodiments, determining whether to move the position of the first pointer based on the read position includes:
[0067] determining whether the distance between the read position and the position of the first pointer is less than a first threshold;
[0068] If the distance is less than a first threshold, keeping the position of the first pointer unchanged;
[0069] If the distance is greater than or equal to the first threshold, the position of the first pointer is moved according to the first threshold and the data block sent to the target application until the distance between the position of the first pointer and the reading position is the first distance.
[0070] Here, the first threshold can be determined according to the device state, user settings, etc. If the data access method is random access, it can be processed in multiple situations:
[0071] If the distance between the read position and the position of the first pointer is less than the first threshold, the data buffer will return the data and keep the first pointer position unchanged without releasing the cache space. Here, considering that the randomly read content is very close to the first pointer, there are enough data blocks cached at the back to be used for subsequent requests by the target application, and there is no need to release the cache area to buffer new data temporarily. At the same time, it can avoid the situation where the user wants to view the previous buffered data but cannot obtain the data block in time.
[0072] If the distance between the read position and the position of the first pointer is greater than or equal to the first threshold, the data buffer will return the data and move the first pointer forward so that the distance between the position of the first pointer and the read position is the first distance, and the first distance may be the first threshold. Here, considering that the distance between the read position and the position of the first pointer is far, if the target application continues to read the following content, the probability that the data buffer does not contain a data block is greatly increased. Therefore, part of the cache is released to leave space to save new data blocks for users to view.
[0073] In the disclosed embodiment, when the data access method is the first method, the position of the first pointer is reasonably adjusted in combination with specific access requirements, that is, taking into account that the user may adjust the content to be accessed, and considering whether the saved data blocks are sufficient to provide to the target application, thereby improving the user experience.
[0074] In some embodiments, determining whether to move the position of the first pointer based on the read position includes:
[0075] determining a midpoint position between the position of the first pointer and the position of the second pointer;
[0076] If the reading position is between the position of the first pointer and the midpoint position, keeping the position of the first pointer unchanged;
[0077] If the reading position is between the midpoint position and the position of the second pointer, the position of the first pointer is moved until the distance between the position of the first pointer and the reading position is a second distance.
[0078] Here, if the target application does not access data in sequence, that is, the read position is not equal to the position indicated by the first pointer, multiple situations can be processed. Figure 5 Specific instructions.
[0079] If the read position is at the first pointer position (i.e. Figure 6 HEAD in) and the midpoint position (i.e. Figure 6 The data buffer will directly provide the data block and keep the position of the first pointer unchanged.
[0080] If the reading position is between the midpoint position and the position of the second pointer ( Figure 6 The first pointer is located between the TAIL in the data buffer, that is, in the B area, and the data buffer corresponding to the first pointer moves forward in a circular manner until the distance between the position of the first pointer and the reading position is exactly the second distance, where the second distance may be TOTAL_SIZE / 2, and TOTAL_SIZE indicates the size of the data block stored in the data buffer. Figure 7 As shown, an example of moving the first pointer is provided. Before the movement, the reading position is located in area B. After the movement, the distance between the position of the first pointer and the reading position is exactly the second distance.
[0081] In the disclosed embodiment, when the data access method is the first method, the position of the first pointer is reasonably adjusted based on the specific access requirements, the position of the first pointer, and the position of the second pointer, while taking into account issues such as the possibility that the user may adjust the content to be accessed and whether the saved data blocks are sufficient, thereby improving the user experience.
[0082] In some embodiments, the determining the management strategy of the pointer group based on the data access mode further includes:
[0083] Determine the size of the free space in the data buffer;
[0084] Determine the number of concurrent threads for sending request data blocks to the second electronic device based on the size of the free space, wherein each thread corresponds to a data block transmission request service;
[0085] In response to the free space in the data buffer receiving the data block corresponding to the thread, the position of the second pointer is moved.
[0086] Here, the data buffer can continue to obtain data blocks of the target file until the end of the target file is reached. In the disclosed solution, it can be determined whether to continue to obtain and save data blocks based on the size of the free space, and it can also be determined how many data blocks to obtain. If it is determined to continue to obtain data blocks, a new concurrent thread can be created to continue to concurrently obtain remote data blocks to maintain the filling state of the data buffer.
[0087] After receiving the data block of the corresponding thread, the received data block is saved in the data buffer, and the position of the second pointer is moved, specifically, the position of the second pointer is moved to the end of all the saved data blocks.
[0088] Here, determining the number of concurrent threads for sending request data blocks to the second electronic device based on the size of the free space may include:
[0089] Determining the number of data blocks that can be stored in the free space based on the free space size and a preset size of the data block;
[0090] Determine the number of concurrent threads based on the number of data blocks that can be saved.
[0091] Here, based on the free space size and the preset size of the data block, the free space size can be divided by the preset size of the data block to obtain the number of data blocks that can be stored. Assuming that the determined free space can store 5 data blocks, the number of concurrent threads can be at least 1-5.
[0092] In the disclosed embodiment, based on the size of the free space, it is determined in time whether to continue to concurrently acquire data blocks to maintain the filling state of the data buffer, thereby ensuring the continuity and smoothness of data acquisition by the target application.
[0093] In some embodiments, determining the free space size of the data buffer includes:
[0094] determining the size of the free space in the data buffer according to a third distance between the position of the second pointer and the position of the first pointer;
[0095] If the free space size is greater than or equal to a second threshold, determining to start the concurrent thread requesting the data block from the second electronic device;
[0096] If the free space is less than the second threshold or the target file has ended, the concurrent thread is not started.
[0097] Here, the second threshold value can be set based on actual conditions, a predetermined size of a data block, etc. For example, the second threshold value can be a predetermined size of a data block, that is, the free space can satisfy one data block. In combination with the above example, assuming that the data buffer size is: 200x128kb, the second threshold value can be 128kb.
[0098] In the disclosed embodiment, whether to obtain a data block is determined according to the size of the free space, thereby avoiding the problem that the data buffer cannot be saved after obtaining the data block from the second electronic device, and reducing the waste of network resources.
[0099] In some embodiments, determining a data access mode of a target application to a plurality of data blocks in the data buffer, and determining a management strategy of the pointer group based on the data access mode includes:
[0100] If the data buffer does not include the data block that the target application needs to access, determining that the data access mode is a third mode;
[0101] Determining that the management strategy is a third strategy corresponding to the third mode;
[0102] Wherein, the third strategy includes:
[0103] Acquire target data from the second electronic device, delete the cached data blocks in the data buffer and cache the acquired target data, wherein the target data at least includes: data blocks that the target application needs to access;
[0104] The pointer group of the data buffer is re-established according to the cached target data.
[0105] Here, a plurality of policies for managing pointer groups are pre-set, and also include: a third policy for a third mode.
[0106] If the read position is not within the range of the data blocks saved in the data buffer, or, during video playback, if the user drags the progress bar and other operations cause a large deviation between the cached data and the read demand, the data rebuffering mechanism can be triggered to ensure smooth playback. Specifically, the data buffer can be reset, and the data can be rebuffered from the new position, and the pointer group of the data buffer can be reestablished based on the cached data. The rebuffered data can at least include the data blocks that the target application needs to access, so that the data blocks can be provided to the target application in a timely manner.
[0107] In the disclosed embodiment, different access requirements of target applications are fully taken into consideration, and new data can be timely retrieved and cached through the third strategy to provide it to the target application, thereby reducing loading time and improving user experience.
[0108] In some embodiments, the method further comprises:
[0109] Using multiple threads to concurrently send multiple network data requests to the second electronic device;
[0110] A data block of a preset size sent by the second electronic device in response to each network data request is received.
[0111] Here, the first electronic device can use multiple threads to send multiple network data requests to the second electronic device in parallel, and receive multiple data blocks of preset sizes. In the case of multiple concurrent network data requests, the return order of the data blocks may not be linear, but the second pointer will move forward according to the position of the data blocks actually received, each time moving the size of a data block.
[0112] In the disclosed embodiment, multiple data blocks can be obtained simultaneously by concurrently requesting multiple network data, thereby improving the speed of data buffering.
[0113] Through the method provided by the present disclosure, the first electronic device can use concurrent multithreading to simultaneously obtain multiple data blocks from the second electronic device and save them in the data buffer, thereby improving the speed of data buffering. At the same time, by managing the pointer group of the data buffer, the use efficiency of the data buffer is optimized, ensuring the continuity and stability of the data cache. In addition, the management strategy can adapt to the access requirements of the upper-level target application (including sequential, disordered, etc.), efficiently utilize the buffered data, reduce loading time while not occupying too much bandwidth, and provide the target application with a fast, accurate and smooth cross-end file data stream, thereby greatly improving the user experience and the overall performance of the system.
[0114] Figure 8 A flowchart of a data access method provided by an embodiment of the present disclosure; Figure 8 As shown, the data access method is applied to a first electronic device, the first electronic device includes a first application, a second application, and a data buffer capable of caching a plurality of data blocks of a preset size from a target file of a second electronic device; the data access method includes:
[0115] Step 801: A first application sends an access request to a second application.
[0116] Here, the first application may be: a video player (Video Player), an image browser (Photoexplorer), a text browser (File explorer), etc.
[0117] The access request may include the following information: the target file to be accessed, the size of the data block to be accessed, and the offset of the data block to be accessed.
[0118] Here, the second application is an Internet application (APP) installed or possessed by the first electronic device. The first electronic device can establish an Internet connection with the second electronic device (the second electronic device can also have an Internet APP installed) through the second application, and can obtain data and communicate from the second electronic device through the Internet.
[0119] Here, the second application may have a network connection management capability, and may be able to manage its network connection with other devices (such as the second electronic device);
[0120] The second application may also have file transfer capabilities, and can create multiple threads to obtain multiple data blocks of the target file from the second electronic device, and save the obtained multiple data blocks in the data buffer in sequence; establish a pointer group for the data buffer; determine the data access method based on the access request, and determine the management strategy for the pointer group; provide data blocks to the first application, etc.
[0121] The pointer group includes a first pointer and a second pointer; the first pointer indicates the head position of the multiple data blocks stored in the data buffer; the second pointer indicates the tail position of the multiple data blocks stored in the data buffer.
[0122] Step 802, the second application determines the data access mode according to the access request; if the data access mode is the first mode, proceed to step 803; if the data access mode is the second mode, proceed to step 804; if the data access mode is the third mode, proceed to step 805.
[0123] Specifically, determining the data access mode according to the access request includes:
[0124] Determine, according to the access request, the position of the data block accessed by the first application in the data buffer, which is recorded as the read position;
[0125] Determine whether the read position is between the position of the first pointer and the position of the second pointer, and whether the read position is the same as the position of the first pointer;
[0126] If the read position is between the position of the first pointer and the position of the second pointer and is different from the position of the first pointer, then determining that the data access mode is the first mode;
[0127] If the read position is the same as the position of the first pointer, determining the data access mode to be the second mode;
[0128] If the read position is not between the position of the first pointer and the position of the second pointer (ie, the data stored in the data buffer does not include the data block that the first application needs to access), the data access mode is determined to be the third mode.
[0129] Step 803: Manage the pointer group based on the first strategy corresponding to the first method.
[0130] Wherein, based on the first strategy corresponding to the first method, managing the pointer group includes:
[0131] If the distance between the read position and the position of the first pointer is less than the first threshold, the second application will return the data and keep the position of the first pointer unchanged without releasing the cache space;
[0132] If the distance between the reading position and the position of the first pointer is greater than or equal to the first threshold, the second application will return the data block and move the first pointer forward so that the distance between the position of the first pointer and the reading position is the first distance, which may be the first threshold.
[0133] Step 804: Manage the pointer group based on the second strategy corresponding to the second method.
[0134] Wherein, based on the second strategy corresponding to the second method, managing the pointer group includes:
[0135] The accessed data block is sent to the first application to provide the first application with the data block it needs, and the position of the first pointer is moved to the next data block to release the position of the accessed data block stored in the data buffer.
[0136] Here, the position of the first pointer can be moved according to the following formula: HEAD = (HEAD + readSize) % TOTAL_SIZE; wherein HEAD is the position of the first pointer, readSize represents the position of the accessed data block, TOTAL_SIZE represents the size of the data buffer, and % represents the modulo operator, which ensures that HEAD circulates within the data buffer.
[0137] Step 805: Manage the pointer group based on the third strategy corresponding to the third method.
[0138] The third strategy corresponding to the third method is used to manage the pointer group, including:
[0139] Reset the data buffer, start buffering data (including at least the data block requested by the first application) from the requested read position, and re-establish the position of the first pointer and the position of the second pointer;
[0140] The data block requested by the first application is sent to the first application, and the position of the first pointer is updated again.
[0141] In some embodiments, the method may further include:
[0142] The second application determines the size of the free space in the data buffer;
[0143] Determine the number of concurrent threads for sending request data blocks to the second electronic device based on the size of the free space, wherein each thread corresponds to a data block transmission request service;
[0144] In response to the free space in the data buffer receiving the data block corresponding to the thread, the position of the second pointer is moved.
[0145] Here, the second application can also determine in time whether to continue to concurrently obtain data blocks based on the size of the free space to maintain the filling state of the data buffer, thereby ensuring the continuity and smoothness of the target application in obtaining data.
[0146] Fig. 9 A schematic diagram of the structure of a first electronic device provided in an embodiment of the present disclosure; Fig. 9 As shown, the first electronic device includes: a data buffer capable of caching a plurality of data blocks of preset sizes from a target file of the second electronic device;
[0147] A first processing module is used to establish a pointer group of the data buffer, wherein the pointer group includes a first pointer and a second pointer, the first pointer indicates the head position of the multiple data blocks stored in the data buffer; the second pointer indicates the tail position of the multiple data blocks stored in the data buffer;
[0148] A second processing module is used to determine the data access mode of the target application to the multiple data blocks in the data buffer;
[0149] The third processing module is used to determine a management strategy for the pointer group based on the data access mode, where the management strategy includes whether to move the first pointer to release the position of the data block accessed by the target application.
[0150] In some embodiments, the second processing module is used to determine whether the read position of the target application is the same as the position indicated by the first pointer; the read position is the position of the data block accessed by the target application in the data buffer;
[0151] If the read position is different from the position of the first pointer, determining that the data access mode is the first mode;
[0152] If the read position is the same as the position of the first pointer, it is determined that the data access mode is the second mode.
[0153] In some embodiments, the third processing module is used to manage the first pointer based on a first strategy if the data access mode is the first mode, and the first strategy includes determining whether to move the position of the first pointer based on the read position;
[0154] If the data access mode is the second mode, the first pointer is managed based on a second strategy, and the second strategy includes moving the position of the first pointer to the next data block to release the position of the accessed data block.
[0155] In some embodiments, the third processing module is used to determine whether the distance between the read position and the position of the first pointer is less than a first threshold;
[0156] If the distance is less than a first threshold, keeping the position of the first pointer unchanged;
[0157] If the distance is greater than or equal to the first threshold, the position of the first pointer is moved according to the first threshold and the data block sent to the target application until the distance between the position of the first pointer and the reading position is the first distance.
[0158] In some embodiments, the third processing module is used to determine a midpoint position between the position of the first pointer and the position of the second pointer;
[0159] If the reading position is between the position of the first pointer and the midpoint position, keeping the position of the first pointer unchanged;
[0160] If the reading position is between the midpoint position and the position of the second pointer, the position of the first pointer is moved until the distance between the position of the first pointer and the reading position is a second distance.
[0161] In some embodiments, the third processing module is further used to determine the size of the free space in the data buffer;
[0162] Determine the number of concurrent threads for sending request data blocks to the second electronic device based on the size of the free space, wherein each thread corresponds to a data block transmission request service;
[0163] In response to the free space in the data buffer receiving the data block corresponding to the thread, the position of the second pointer is moved.
[0164] In some embodiments, the third processing module is further used to determine the size of the free space of the data buffer according to a third distance between the position of the second pointer and the position of the first pointer;
[0165] If the free space size is greater than or equal to a second threshold, determining to start the concurrent thread requesting the data block from the second electronic device;
[0166] If the free space is less than the second threshold or the target file has ended, the concurrent thread is not started.
[0167] In some embodiments, the third processing module is used to determine that the data access mode is a third mode if the data buffer does not include the data block that the target application needs to access;
[0168] Determining that the management strategy is a third strategy corresponding to the third mode;
[0169] Wherein, the third strategy includes:
[0170] Acquire target data from the second electronic device, delete the cached data blocks in the data buffer and cache the acquired target data, wherein the target data at least includes: data blocks that the target application needs to access;
[0171] The pointer group of the data buffer is re-established according to the cached target data.
[0172] In some embodiments, the electronic device further comprises: a fourth processing module, configured to concurrently send multiple network data requests to the second electronic device using multiple threads;
[0173] A data block of a preset size sent by the second electronic device in response to each network data request is received.
[0174] It is understandable that when the first electronic device provided in the above embodiment implements the corresponding data processing method, the above processing can be assigned to different program modules as needed to complete all or part of the processing described above. In addition, the device provided in the above embodiment and the embodiment of the corresponding method belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.
[0175] An embodiment of the present disclosure provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will be triggered to execute the data processing method provided by the embodiment of the present disclosure.
[0176] In some embodiments, the computer-readable storage medium may be a ferroelectric random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disk, or a CD-ROM; it may also be various devices including one or any combination of the above memories.
[0177] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, model, subroutine or other unit suitable for use in a computing environment.
[0178] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.
[0179] An embodiment of the present disclosure provides a computer program product, which includes a computer program / instructions. When the computer program / instructions are executed by a processor, the data processing method described in the present disclosure is implemented.
[0180] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure; Fig.10 As shown, the electronic device 100 includes: a processor 1001 and a memory 1002 for storing a computer program that can be run on the processor; when the processor 1001 is used to run the computer program, it executes the data processing method provided by the embodiment of the present disclosure.
[0181] The electronic device 100 further includes a data buffer capable of caching a plurality of data blocks of a preset size from a target file of the second electronic device.
[0182] In actual application, the electronic device 100 may further include: at least one network interface 1003. The various components in the electronic device 100 are coupled together via a bus system 1004. It is understood that the bus system 1004 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 1004 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Fig.10 In the figure, various buses are marked as bus system 1004. There may be at least one processor 1001. The network interface 1003 is used for wired or wireless communication between the electronic device 100 and other devices.
[0183] The memory 1002 in the embodiment of the present disclosure is used to store various types of data to support the operation of the electronic device 100 .
[0184] The method disclosed in the above embodiment of the present disclosure can be applied to the processor 1001, or implemented by the processor 1001. The processor 1001 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit in the processor 1001 or the instruction in the form of software. The above processor 1001 can be a general processor, a digital signal processor (DSP, DiGital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 1001 can implement or execute the disclosed methods, steps and logic block diagrams in the embodiment of the present disclosure. The general processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiment of the present disclosure can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor can be combined to execute. The software module can be located in a storage medium, which is located in the memory 1002. The processor 1001 reads the information in the memory 1002 and completes the steps of the above method in combination with its hardware.
[0185] In some embodiments, the electronic device 100 can be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD), field programmable gate array (FPGA), general processor, controller, microcontroller (MCU), microprocessor, or other electronic components to execute the aforementioned method.
[0186] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0187] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
[0188] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.
Claims
1. A data processing method, applied to a first electronic device, wherein the first electronic device includes a data buffer capable of caching a plurality of data blocks of a preset size from a target file of a second electronic device, the method comprising: Establishing a pointer group of the data buffer, wherein the pointer group includes a first pointer and a second pointer, the first pointer indicates the head position of the multiple data blocks stored in the data buffer; the second pointer indicates the tail position of the multiple data blocks stored in the data buffer; Determine a data access mode of a target application to a plurality of data blocks in the data buffer; A management strategy for the pointer group is determined based on the data access mode, and the management strategy includes whether to move the first pointer to release the location of the accessed data block.
2. The method according to claim 1, characterized in that Determining a data access mode of a target application to a plurality of data blocks in the data buffer includes: Determine whether the read position of the target application is the same as the position indicated by the first pointer; the read position is the position of the data block accessed by the target application in the data buffer; If the read position is different from the position of the first pointer, determining that the data access mode is the first mode; If the read position is the same as the position of the first pointer, it is determined that the data access mode is the second mode.
3. The method according to claim 1 or 2, characterized in that: Determining a management strategy of the pointer group based on the data access mode includes: If the data access mode is the first mode, managing the first pointer based on a first strategy, the first strategy including determining whether to move the position of the first pointer based on the read position; If the data access mode is the second mode, the first pointer is managed based on a second strategy, and the second strategy includes moving the position of the first pointer to the next data block to release the position of the accessed data block.
4. The method according to claim 3, characterized in that Determining whether to move the position of the first pointer based on the read position includes: determining whether the distance between the read position and the position of the first pointer is less than a first threshold; If the distance is less than a first threshold, keeping the position of the first pointer unchanged; If the distance is greater than or equal to the first threshold, the position of the first pointer is moved according to the first threshold and the data block sent to the target application until the distance between the position of the first pointer and the reading position is the first distance.
5. The method according to claim 3, characterized in that: Determining whether to move the position of the first pointer based on the read position includes: determining a midpoint position between the position of the first pointer and the position of the second pointer; If the reading position is between the position of the first pointer and the midpoint position, keeping the position of the first pointer unchanged; If the reading position is between the midpoint position and the position of the second pointer, the position of the first pointer is moved until the distance between the position of the first pointer and the reading position is a second distance.
6. The method according to claim 1, characterized in that The determining of the management strategy of the pointer group based on the data access mode further includes: Determine the size of the free space in the data buffer; Determine the number of concurrent threads for sending request data blocks to the second electronic device based on the size of the free space, wherein each thread corresponds to a data block transmission request service; In response to the free space in the data buffer receiving the data block corresponding to the thread, the position of the second pointer is moved.
7. The method according to claim 6, characterized in that Determine the size of the free space in the data buffer, including: determining the size of the free space in the data buffer according to a third distance between the position of the second pointer and the position of the first pointer; If the free space size is greater than or equal to a second threshold, determining to start the concurrent thread requesting the data block from the second electronic device; If the free space is less than the second threshold or the target file has ended, the concurrent thread is not started.
8. The method according to claim 1, characterized in that: Determining a data access mode of a target application to a plurality of data blocks in the data buffer, and determining a management strategy of the pointer group based on the data access mode, including: If the data buffer does not include the data block that the target application needs to access, determining that the data access mode is a third mode; Determining that the management strategy is a third strategy corresponding to the third mode; Wherein, the third strategy includes: Acquire target data from the second electronic device, delete the cached data blocks in the data buffer and cache the acquired target data, wherein the target data at least includes: data blocks that the target application needs to access; The pointer group of the data buffer is re-established according to the cached target data.
9. The method according to claim 1, characterized in that: The method further comprises: Using multiple threads to concurrently send multiple network data requests to the second electronic device; A data block of a preset size sent by the second electronic device in response to each network data request is received.
10. A first electronic device, comprising: a data buffer capable of caching a plurality of data blocks of a preset size from a target file of a second electronic device; A first processing module is used to establish a pointer group of the data buffer, wherein the pointer group includes a first pointer and a second pointer, the first pointer indicates the head position of the multiple data blocks stored in the data buffer; the second pointer indicates the tail position of the multiple data blocks stored in the data buffer; A second processing module is used to determine the data access mode of the target application to the multiple data blocks in the data buffer; The third processing module is used to determine a management strategy for the pointer group based on the data access mode, where the management strategy includes whether to move the first pointer to release the position of the data block accessed by the target application.
Citation Information
Patent Citations
SSD data storage node management method and device, computer device and storage medium
CN110674053A
Efficient data collection and storage method, system and equipment and storage medium
CN114661624A
Strategies for reading information from a mass storage medium using a cache memory
US20050223165A1
Adaptive Buffering of Database Server Response Data
US20090265398A1
Mechanism for a Lockless Ring Buffer in Overwrite Mode
US20100312985A1
