A main memory data management method and apparatus

Abstract

The embodiment of the present application provides a kind of main memory data management method and device, it is related to data processing technical field, above-mentioned method includes: receiving data to be stored;Judge whether the first free space in the second storage space in main memory after being located in write position is sufficient to store data to be stored, second storage space is located in the fixed first storage space in main memory, the size of second storage space is less than the storage space size of main memory, store data to the first free space;If insufficient, in the case where the second free space is sufficient to store data to be stored, from the first starting position of second free space, data to be stored is stored to second free space, and the second free space is: the free storage space before being located in reading position in second storage space.The application of the embodiment of the present application provides scheme and can reduce the number of data handling, improve the utilization of data processor.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a main memory data management method and apparatus. Background Technology

[0002] During operation, the data that a data processor needs to process may be stored in main memory. In this case, the data processor needs to access main memory, read the data from main memory, and write it into a high-speed cache with faster access speed. The data processor can then read the data from the cache for processing. In related technologies, when storing data in main memory, the data is stored at the end of the already stored data to ensure the continuity of the stored data. When reading data from main memory, the data is read continuously starting from the beginning of the already stored data.

[0003] However, if the free storage space in main memory after the end of the stored data is insufficient to accommodate the data to be stored, it is difficult to store the data to be stored in main memory using the above storage method. Therefore, the stored data in main memory can be moved to the beginning of the main memory storage space to expand the free storage space after the end of the stored data.

[0004] However, during data transfer, main memory becomes locked. When main memory is locked, the data processor has difficulty accessing the data stored in main memory, making data management of main memory difficult. Using related technologies for main memory data management involves frequent data transfer of already stored data, which affects the normal operation of the data processor and results in low data processor utilization. Summary of the Invention

[0005] The purpose of this invention is to provide a main memory data management method and apparatus to reduce the number of data transfer operations and improve the utilization rate of the data processor. The specific technical solution is as follows:

[0006] In a first aspect, embodiments of the present invention provide a main memory data management method, the method comprising:

[0007] Receive data to be stored;

[0008] Determine whether the first free space in the second storage space of the main memory after the write position is sufficient to store the data to be stored, wherein the second storage space is located in the fixed first storage space in the main memory, the size of the second storage space is smaller than the size of the first storage space, and the write position is: the position where the data tail of the most recently written data is located;

[0009] If sufficient, starting from the write position, the data to be stored is stored in the first free space;

[0010] If the second free space is insufficient, and if the second free space is sufficient to store the data to be stored, the data to be stored is stored in the second free space starting from the first starting position of the second free space. The second free space is the free storage space within the second storage space located before the read position, and the read position is the position where the data header of the data with the longest storage duration is located.

[0011] In one embodiment of the present invention, before storing the data to be stored into the second free space, starting from the beginning position of the second free space, when the second free space is sufficient to store the data to be stored, the method further includes:

[0012] Update the second starting position of the second storage space to the first position, wherein the first position is: the position of the second data volume before the second starting position before the update, and the second data volume is: the amount of data that the first free space can accommodate;

[0013] Update the termination position of the second storage space to the second position, wherein the second position is the position of the second data volume before the termination position before the update.

[0014] In one embodiment of the present invention, the method further includes:

[0015] Upon receiving a data extraction request from a data processing service, target data is read and sent to the data processing service so that the data processing service can process the target data. The target data is data located in the second storage space after the read position.

[0016] Receive processing status information from the data processing service;

[0017] If the processing status information indicates that the data processing was successful, the reading position is updated to the third position, wherein the third position is: the position where the data tail of the data successfully processed by the data processing service was located before reading;

[0018] If the processing status information indicates that the data processing failed, then if there is data in the second storage space that is located before the read position, the target data is moved to the position before the second starting position, with the second starting position of the second storage space as the tail, and both the second starting position and the read position are updated to the position where the target data header is located after the movement.

[0019] In one embodiment of the present invention, after updating both the second starting position and the reading position to the position where the target data header is located after transportation, the method further includes:

[0020] The termination position of the second storage space is updated to the fourth position, wherein the fourth position is the position of the first data volume before the termination position before the update, and the first data volume is the data volume of the target data.

[0021] In one embodiment of the present invention, before moving the target data to the second starting position, the process includes:

[0022] The target data is moved to the second starting position using either the main memory copy (memcpy) or main memory move (memmove) method.

[0023] or

[0024] The data to be stored is audio data, image data, or video data.

[0025] In one embodiment of the present invention, the method further includes:

[0026] When the read position is located before the write position, it is determined whether the size of the third storage space is less than the size of the fourth storage space, wherein the third storage space is: the storage space in the second storage space located before the read position, and the fourth storage space is: the storage space in the first storage space located after the write position;

[0027] If so, the second starting position of the second storage space is updated to the read position, and the ending position of the second storage space is updated based on the second starting position of the updated second storage space according to the size of the second storage space before the update.

[0028] In one embodiment of the present invention, the method further includes:

[0029] If the first free space is sufficient to store the data to be stored, the data receiving service that sent the data to be stored sends back first status information, wherein the first status information indicates that the data to be stored was successfully stored.

[0030] and / or

[0031] If the first free space is insufficient to store the data to be stored, a second status information is sent to the data receiving service, wherein the second status information indicates that the data storage of the data to be stored has failed.

[0032] In a second aspect, embodiments of the present invention provide a main memory data management device, the device comprising:

[0033] The data receiving module is used to receive data to be stored.

[0034] The space judgment module is used to determine whether the first free space in the second storage space in the main memory after the write position is sufficient to store the data to be stored. The second storage space is located in the fixed first storage space in the main memory. The size of the second storage space is smaller than the size of the first storage space. The write position is the position where the last data written is located.

[0035] The first storage module is configured to, when the space determination module determines that the first free space is sufficient to store the data to be stored, store the data to be stored in the first free space starting from the write position.

[0036] The second storage module is used to store the data to be stored in the second free space, starting from the first starting position of the second free space, when the space judgment module determines that the first free space is insufficient to store the data to be stored, and when the second free space is sufficient to store the data to be stored. The second free space is: the free storage space within the second storage space located before the read position, and the read position is: the position where the data header of the data with the longest storage duration is located.

[0037] In one embodiment of the present invention, the apparatus further includes:

[0038] The first position update module is used to update the second starting position of the second storage space to the first position before storing the data to be stored in the second free space, wherein the first position is: the position of the second amount of data before the second starting position before the update, and the second amount of data is: the amount of data that the first free space can accommodate;

[0039] The second position update module is used to update the termination position of the second storage space to a second position, wherein the second position is the position of the second data volume before the termination position before the update.

[0040] In one embodiment of the present invention, the second storage module is specifically used for:

[0041] If the space determination module determines that the first free space is insufficient to store the data to be stored, and if the second free space is sufficient to store the data to be stored, the target data is moved to the position before the second starting position by using the main memory copy memcpy method or the main memory move memmove method, starting from the first starting position of the second free space.

[0042] or

[0043] The data to be stored is audio data, image data, or video data.

[0044] In one embodiment of the present invention, the apparatus further includes:

[0045] The data reading module is used to read and send target data to the data processing service upon receiving a data extraction request from the data processing service, so that the data processing service can process the target data. The target data is data located after the reading position in the second storage space.

[0046] The information receiving module is used to receive processing status information fed back by the data processing service;

[0047] The third position update module is used to update the read position to the third position if the processing status information indicates that the data processing was successful. The third position is the position of the data tail of the data successfully processed by the data processing service before reading.

[0048] The fourth position update module is used to, if the processing status information indicates that the data processing has failed, and if there is data in the second storage space located before the read position, move the target data to before the second starting position, with the second starting position of the second storage space as the tail, and update both the second starting position and the read position to the position where the target data header is located after the move.

[0049] In one embodiment of the present invention, the second position update module is further configured to:

[0050] The termination position of the second storage space is updated to the fourth position, wherein the fourth position is the position of the first data volume before the termination position before the update, and the first data volume is the data volume of the target data.

[0051] In one embodiment of the present invention, the apparatus further includes:

[0052] A size determination module is used to determine whether the size of the third storage space is less than the size of the fourth storage space when the read position is located before the write position, wherein the third storage space is the storage space in the second storage space located before the read position, and the fourth storage space is the storage space in the first storage space located after the write position;

[0053] The data transfer module is used to update the second starting position of the second storage space to the read position if the size determination module determines that the size of the third storage space is less than the size of the fourth storage space, and to update the ending position of the second storage space based on the second starting position of the updated second storage space according to the size of the second storage space before the update.

[0054] In one embodiment of the present invention, the apparatus further includes:

[0055] The first information sending module is configured to, if the first free space is sufficient to store the data to be stored, send back first status information to the data receiving service that sends the data to be stored, wherein the first status information indicates that the data to be stored has been successfully stored.

[0056] and / or

[0057] The second information sending module is used to send a second status information to the data receiving service if the first free space is insufficient to store the data to be stored, wherein the second status information indicates that the data storage to be stored has failed.

[0058] Thirdly, an electronic device provided by an embodiment of the present invention includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0059] Memory, used to store computer programs;

[0060] When a processor executes a program stored in memory, it implements any of the steps described in the first aspect.

[0061] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the methods described in the first aspect.

[0062] Fifthly, embodiments of the present invention also provide a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the steps described in the first aspect.

[0063] Beneficial effects of the embodiments of the present invention:

[0064] As can be seen from the above, when managing main memory using the solution provided in this embodiment of the invention, if the first free space after the write position can store the data to be stored, the data to be stored can be directly stored in the first free space. Conversely, if the first free space is insufficient to store the data to be stored, it can be determined whether the second free space before the read position is sufficient to store the data to be stored. If it is sufficient, the data to be stored can be stored in the second free space before the read position. In related technologies, the data to be stored is stored at the end of the already stored data, that is, in the first free space after the already stored data. However, when storing data using this solution, not only can the data to be stored be written into the first free space located after the data already stored in main memory, but also, if the first free space is small, the data to be stored can be stored in the second free space before the read position, which is different from the first free space. In this process, the data transfer process in related technologies is not required. Therefore, when managing main memory data, this embodiment of the invention can make full use of the free storage space in the first storage space, thereby reducing the number of data transfers and improving the utilization rate of the data processor while still achieving data storage. Attached Figure Description

[0065] To more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0066] Figure 1A This is a schematic diagram of a first data storage structure provided in an embodiment of the present invention;

[0067] Figure 1B This is a schematic diagram of a second data storage structure provided in an embodiment of the present invention;

[0068] Figure 1C This is a schematic diagram of a third data storage structure provided in an embodiment of the present invention;

[0069] Figure 2A This is a schematic diagram of a first application scenario provided by an embodiment of the present invention;

[0070] Figure 2B This is a schematic diagram of a second application scenario provided by an embodiment of the present invention;

[0071] Figure 3 A flowchart illustrating the first main memory data management method provided in an embodiment of the present invention;

[0072] Figure 4AThis is a schematic diagram of the fourth data storage structure provided in an embodiment of the present invention;

[0073] Figure 4B This is a schematic diagram of the fifth data storage structure provided in an embodiment of the present invention;

[0074] Figure 4C This is a schematic diagram of the sixth data storage structure provided in an embodiment of the present invention;

[0075] Figure 5 A flowchart illustrating the second main memory data management method provided in this embodiment of the invention;

[0076] Figure 6A This is a schematic diagram of the seventh data storage structure provided in an embodiment of the present invention;

[0077] Figure 6B This is a schematic diagram of the eighth data storage structure provided in an embodiment of the present invention;

[0078] Figure 7 A flowchart illustrating the third main memory data management method provided in this embodiment of the invention;

[0079] Figure 8A A schematic diagram of the ninth data storage structure provided in this embodiment of the invention;

[0080] Figure 8B This is a schematic diagram of a multimedia data management process provided in an embodiment of the present invention;

[0081] Figure 9 A flowchart illustrating the fourth main memory data management method provided in this embodiment of the invention;

[0082] Figure 10 A flowchart illustrating the fifth main memory data management method provided in this embodiment of the invention;

[0083] Figure 11 This is a schematic diagram of the structure of a first type of main memory data management device provided in an embodiment of the present invention;

[0084] Figure 12 This is a schematic diagram of the structure of a second main memory data management device provided in an embodiment of the present invention;

[0085] Figure 13 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0086] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0087] Because data access using related technologies involves frequent data movement of stored data, the utilization rate of the data processor is low. To address this issue, embodiments of the present invention provide a main memory data management method and apparatus.

[0088] This invention provides a main memory data management method, the method comprising:

[0089] Receive data to be stored;

[0090] Determine whether the first free space in the second storage space of the main memory after the write position is sufficient to store the data to be stored, wherein the second storage space is located in the fixed first storage space in the main memory, the size of the second storage space is smaller than the size of the first storage space, and the write position is: the position where the data tail of the most recently written data is located;

[0091] If sufficient, starting from the aforementioned write position, store the aforementioned data to be stored in the aforementioned first free space;

[0092] If the second free space is insufficient, and if the second free space is sufficient to store the data to be stored, the data to be stored shall be stored in the second free space starting from the first starting position of the second free space. The second free space is the free storage space within the second storage space located before the read position, and the read position is the position where the data header of the data with the longest storage duration is located.

[0093] As can be seen from the above, when managing main memory using the solution provided in this embodiment of the invention, if the first free space after the write position can store the data to be stored, the data to be stored can be directly stored in the first free space. Conversely, if the first free space is insufficient to store the data to be stored, it can be determined whether the second free space before the read position is sufficient to store the data to be stored. If it is sufficient, the data to be stored can be stored in the second free space before the read position. In related technologies, the data to be stored is stored at the end of the already stored data, that is, in the first free space after the already stored data. However, when storing data using this solution, in addition to writing the data to be stored into the first free space located after the data already stored in main memory, if the first free space is small, the data to be stored can also be stored in the second free space before the read position, which is different from the first free space. In this process, the data transfer process in related technologies is not required. Therefore, when managing main memory data using this embodiment of the invention, the free storage space in the first storage space can be fully utilized, thereby reducing the number of data transfers and improving the utilization rate of the data processor while still achieving data storage.

[0094] First, the relevant technologies will be described through examples:

[0095] See Figure 1A This is a schematic diagram of the first data storage structure provided by an embodiment of the present invention. The horizontal lines in the figure represent the storage space of the main memory, and the rectangles 1-5 in the figure represent the data packets already stored in the main memory. When a new data packet 6 needs to be written to the main memory, the data packet 6 can be stored after the data packet 5 located at the end. When retrieving data from the main memory, the data will be retrieved starting from the beginning of the stored data, and the data contained in the data packet 1 located at the beginning will be retrieved first.

[0096] See Figure 1B This is a schematic diagram of the second data storage structure provided in an embodiment of the present invention, and... Figure 1A Compared to the data storage structure shown, Figure 1B The diagram shows the storage structure of the data packets in main memory after writing data packet 6 to main memory and reading data packet 1 from main memory.

[0097] In addition, with Figure 1B Taking the data storage structure shown as an example, the free storage space after data packet 6 is relatively small. If the amount of data to be written subsequently exceeds the free storage space after data packet 6, data can be moved from the already stored data packets 2 to 6. See also... Figure 1C This is a schematic diagram of the third data storage structure provided in an embodiment of the present invention. Figure 1C It shows the Figure 1B The data storage structure in main memory after data packets 2-6 are moved, as shown, is similar to... Figure 1B compared to, Figure 1C The free storage space located after the stored data, i.e. after data packet 6, is expanded.

[0098] As can be seen from the examples above, in related technologies, when the available storage space after the stored data is small, the data processor needs to move the stored data, resulting in a large number of data moves.

[0099] Furthermore, the main memory data management method provided by the embodiments of the present invention will be described in detail below through specific examples.

[0100] Specifically, in this embodiment of the invention, the executing entity is a data processor, such as a CPU or MCU (Microcontroller Unit). The data processor can control the storage of data in main memory and can also retrieve stored data from main memory.

[0101] See Figure 2A This is a schematic diagram of the first application scenario provided by an embodiment of the present invention.

[0102] In this embodiment of the invention, the data processor is a CPU, and the application scenario includes CPU cache (CPURegister), cache memory, and main memory. The CPU cache is a data storage unit within the CPU. The data access speed of the CPU cache is typically less than 1ns, while the data access speed of main memory is typically around 65ns, a significant difference. Furthermore, the storage space of the CPU cache is typically 4k-64k, while the storage space of main memory is typically 4G-32G, also a significant difference. Direct data interaction between the two is difficult; therefore, a cache can be used to achieve data interaction between the CPU cache and main memory.

[0103] The storage space of a cache is often larger than that of the CPU cache, but smaller than that of main memory. For example, the storage space of the aforementioned cache is often between 2MB and 16MB.

[0104] Figure 2AThe arrows between CPU caches indicate that data can be exchanged between them, and the arrows between caches and main memory indicate that data can be exchanged between them. The CPU can control the retrieval of data from main memory and its storage in the cache, then retrieve data from the cache and its storage in the CPU cache, and then process the data in the CPU cache. Alternatively, the CPU can also control the retrieval of data from the CPU cache and its storage in the cache, then retrieve data from the cache and its storage in main memory. This invention can be applied to the process of storing data in or retrieving data from main memory.

[0105] Furthermore, the data managed in this embodiment of the invention can be of any type, such as audio data, image data, video data, and other multimedia data. Specifically, the multimedia data can be collected by a collection device, which can be installed on the target device where the execution subject of this invention is located, or it can be another device outside the target device. The collection device can be an audio collection device, an image collection device, a video collection device, or other multimedia collection devices. Additionally, the multimedia data can also be data obtained from the network or sent to the target device by other devices; this embodiment does not limit this.

[0106] See Figure 2B This is a schematic diagram of a second application scenario provided by an embodiment of the present invention, in which the data processor manages multimedia data.

[0107] The arrow between the acquisition device and the data receiving service indicates that after the acquisition device acquires multimedia data, it can send the data packet containing the multimedia data to the data receiving service in the target device. For example, the data packet can be sent to the data receiving service via the network. After receiving the data packet, the data receiving service can send a feedback signal to the acquisition device, indicating that the target device has received the data packet acquired by the acquisition device.

[0108] The arrow between the data receiving service and main memory indicates that the CPU controls the storage of data packets received by the data receiving service into main memory. The arrow between main memory and data processing service indicates that the CPU controls the reading of data from main memory and the sending of the read data to the data processing service. The solution provided by this invention is applied to both of the above processes.

[0109] Since the data processed in this embodiment is multimedia data, such data is often encapsulated and encoded. After receiving the multimedia data, the target device needs to decapsulate and decode the data before it can be used. For example, the encapsulation formats of the multimedia data include, but are not limited to, PS (PhotoShop), TS (Transport Stream), ISO, MP4, and AVI formats, and the encoding formats of the multimedia data include, but are not limited to, H.264, H.265, VP8, VP9, ​​Hik264, and Hik265 formats. Therefore, the aforementioned data processing service can be used to decapsulate and decode the received multimedia data.

[0110] The arrow between the data processing service and the business processing service indicates that the CPU can control the sending of data processed by the data processing service to the business processing service. The business processing service can then perform subsequent business processing based on the data processed by the data processing service. For example, the aforementioned business could be AI (Artificial Intelligence) services, video playback services, etc. Specifically, the data sent by the data processing service to the business processing service could be decapsulated and decoded multimedia data, etc.

[0111] Specifically, the data receiving service, data processing service, and business processing service shown in the diagram are all virtual services within the target device.

[0112] The specific processing flow of this invention embodiment is described below, see [link to relevant documentation]. Figure 3 The above is a flowchart illustrating the first main memory data management method provided in the embodiment of the present invention. The method includes the following steps S301-S304.

[0113] S301: Receive data to be stored.

[0114] The aforementioned data to be stored may be data sent from other devices to the target device where the data processor of the execution subject of this embodiment is located, and to be processed by the data processor.

[0115] S302: Determine whether the first free space in the second storage space of the main memory after the write position is sufficient to store the above-mentioned data to be stored.

[0116] The second storage space is located within the first storage space fixed in the main memory. The size of the second storage space is smaller than the size of the main memory. The write location is the position where the last data written is located.

[0117] Specifically, the first storage space can be located at any fixed position in main memory. The start and end positions of the first storage space are fixed. When this embodiment of the invention begins execution, the second storage space can be located in the middle of the first storage space, meaning that there are other storage spaces before the second storage space in the first storage space and other storage spaces after the second storage space in the first storage space. Alternatively, the second storage space can be located at the end of the first storage space, meaning that there are other storage spaces before the second storage space in the first storage space and no other storage spaces after the second storage space in the first storage space.

[0118] Furthermore, the aforementioned write position is the location of the last data entry in the most recent write operation; that is, the write position will be gradually updated as data is written. Specifically, when no data has been written to the second storage space, the initial value of the write position can be any position within the second storage space. Specifically, it can be the second starting position of the second storage space, maximizing the size of the first free space after the write position in the initial state. In this case, the first free space represents the entire second storage space.

[0119] In addition, the above-mentioned write location can be represented in the form of an address in main memory, and the above-mentioned second storage space can also be represented in the form of the address of the second starting position and the address of the second ending position of the second storage space.

[0120] Furthermore, the aforementioned data to be stored can be represented in the form of data packets.

[0121] In one embodiment of the present invention, if the first free space is sufficient to store the data to be stored, the data to be stored can be directly stored in the first free space, and then step S303 can be executed.

[0122] Conversely, if the first free space is insufficient to store the data to be stored, then the data to be stored cannot be stored in the first free space, and step S304 can be executed.

[0123] S303: Starting from the above-mentioned write position, store the above-mentioned data to be stored in the above-mentioned first free space.

[0124] Specifically, the principle of physical continuation of storage can be adopted. The data header of the data to be stored is stored in the above-mentioned write position, and each byte is stored in the first free space from front to back according to the order of the bytes in the data to be stored. That is, the order of the bytes in the first free space after storage is the same as the order of the bytes in the data to be stored.

[0125] In addition, after the data to be stored is written into the first free space, the above-mentioned write position will be updated to the position of the last data written, which is the storage position of the last data of the data to be stored. Then, when new data to be stored is written next time, the new data to be stored will continue to be stored from the new write position, that is, the storage position of the last data of the data to be stored this time, so that there will be no gap between the two consecutive data storages.

[0126] Specifically, when the write position is represented by an address, the amount of data to be stored can be added to the address representing the write position before the update to obtain the address representing the write position after the update.

[0127] See Figure 4A This is a schematic diagram of the fourth data storage structure provided in an embodiment of the present invention. The dashed area in the figure represents the storage space of the first storage space, and the solid area represents the second storage space. It can be seen that in this embodiment, the second storage space is located at the end of the first storage space.

[0128] The rectangles ac shown in the figure represent data packets ac that are sequentially written into the second storage space. The specific storage process is as follows: Initially, the second storage space is empty, and the write position is located at the second starting position of the second storage space. At this time, the first free space is the aforementioned second storage space. After data packet a is stored in the second storage space based on the write position, the write position is updated to the position of the end of data packet a, and the first free space is updated to the free storage space in the second storage space located after data packet a. Then, after data packet b is stored in the second storage space based on the updated write position, the write position is updated to the position of the end of data packet b, and the first free space is updated to the free storage space in the second storage space located after data packet b. Then, after data packet c is stored in the second storage space based on the updated write position, the write position is updated to the position of the end of data packet c, and the first free space is updated to the free storage space in the second storage space located after data packet c.

[0129] Figure 4A The solid arrow indicates the current write position, and the dashed arrow indicates the current read position. A detailed description of the read position can be found in step S304 below, and will not be elaborated here. If a new data packet d exists as data to be stored, step S302 can be used to determine whether the current first free space is sufficient to store data packet d. If so, step S303 can be executed to store data packet d after data packet c in the first free space; otherwise, step S304 can be executed.

[0130] As can be seen from the above process, as the data to be stored is written sequentially, the writing position gradually moves backward and the first free space gradually shrinks.

[0131] S304: If the second free space is sufficient to store the data to be stored, starting from the first starting position of the second free space, the data to be stored is stored in the second free space.

[0132] The second free space is the free storage space within the second storage space before the read position, and the read position is the location of the data header of the data with the longest storage duration.

[0133] In an optional embodiment of the present invention, the aforementioned read position is the starting position of the data read when reading data from the second storage space. The read position is also the position where the data header of the data with the longest storage duration is located, which is equivalent to always prioritizing reading the data with the longest storage duration from the second storage space. In this embodiment of the present invention, the process of reading data from the second storage space and the process of writing data to the second storage space can be executed synchronously. As described above... Figure 4A Taking the second storage space as an example, data from data packet 'a' will be read first, starting from the reading position indicated by the dashed arrow. After the data is read, it may no longer be stored in the second storage space; that is, the storage space closest to the beginning of the second storage space will be preferentially converted to free storage space. The specific reading process can be found below. Figure 7 Step S701 shown will not be described in detail here.

[0134] Specifically, the initial value of the read position can be the same as the initial value of the write position.

[0135] After data is read from the read position, data after the read position is no longer stored in the second storage space. The second storage space, which was previously used for data storage, becomes free again and can be used to store data. After retrieving data, the read position can be updated to the original storage position of the end of the read data. The specific method for updating the read position can be found in steps S703-S704 below, and will not be detailed here. After the update, the free storage space before the read position becomes larger, that is, the second free space becomes larger.

[0136] If the second free space is sufficient to store the data to be stored, the data to be stored can be stored in the second free space. Otherwise, if the second free space is also insufficient to accommodate the data to be stored, the second storage space is temporarily unable to store the data to be stored, and the process of this embodiment of the invention can be terminated.

[0137] Specifically, the method of storing the data to be stored in the second free space is similar to the method of storing the data to be stored in the first free space, except that the storage position of the beginning of the data to be stored is changed from the writing position to the first starting position of the second free space. This embodiment will not elaborate on this.

[0138] Furthermore, in the initial stage of this embodiment, the amount of data written to the second storage space is relatively small, and the size of the first free space after the write position is relatively large, capable of accommodating a large amount of data. Therefore, new data to be stored will be stored sequentially, meaning that the data newly written to the second storage space will be located after the data already stored in the second storage space. Since the write position is located at the end of the most recently written data, and the read position is located at the beginning of the data with the longest storage duration, the read position is located before the write position. Theoretically, all data originally located before the current read position has already been read, and there is no data in the data storage area before the read position. The first starting position of the second free space before the read position is the second starting position of the second storage space. Meanwhile, the storage space after the current write position has not yet stored any data; that is, the ending position of the first free space after the write position is the ending position of the second storage space.

[0139] After the data to be stored is stored in the second free space, the write position is updated to the position of the last data entry written to the second free space. Then, the first start position of the second free space changes to the updated write position, while the end position remains the read position. In other words, the second free space becomes the free storage space after the write position and before the read position; therefore, the second free space can also be referred to as the first free space. When storing data to be stored again, if the first free space is sufficient to store the new data, step S303 can be executed to write the data to be processed into the first free space.

[0140] Specifically, when the second starting position of the second storage space is represented by an address, the amount of data to be stored can be added to the address to obtain a new address, which serves as the address indicating the write position after the update.

[0141] See Figure 4B This is a schematic diagram of the fifth data storage structure provided in the embodiments of the present invention.

[0142] As mentioned above Figure 4B Compared to the illustrated embodiment, if the data contained in data packets a and b is read, the read position is updated to the original storage position of the data tail of data packet b. The second free space is the free storage space in the second storage space located before the read position, i.e., before data packet c. At this time, if data packet d to be stored in the second storage space is to be stored, and the first free space after data packet c is insufficient to store data packet d, but the second free space before data packet c is sufficient to store data packet d, then data packet d can be stored in the second free space.

[0143] See Figure 4C This is a schematic diagram of the sixth data storage structure provided in the embodiments of the present invention.

[0144] As mentioned above Figure 4B Compared to the embodiments shown, Figure 4C The data storage structure of the second storage space is shown after data packet d is stored in the second free space. After data packet d is stored in the second free space, the above-mentioned write position is updated to the position of the data tail of data packet d, and the above-mentioned first free space and second free space are both updated to the free storage space between data packet d and data packet c.

[0145] As can be seen from the above, when managing main memory using the solution provided in this embodiment of the invention, if the first free space after the write position can store the data to be stored, the data to be stored can be directly stored in the first free space. Conversely, if the first free space is insufficient to store the data to be stored, it can be determined whether the second free space before the read position is sufficient to store the data to be stored. If it is sufficient, the data to be stored can be stored in the second free space before the read position. In related technologies, the data to be stored is stored at the end of the already stored data, that is, in the first free space after the already stored data. However, when storing data using this solution, not only can the data to be stored be written into the first free space located after the data already stored in main memory, but also, if the first free space is small, the data to be stored can be stored in the second free space before the read position, which is different from the first free space. In this process, the data transfer process in related technologies is not required. Therefore, when managing main memory data, this embodiment of the invention can make full use of the free storage space in the first storage space, thereby reducing the number of data transfers and improving the utilization rate of the data processor while still achieving data storage.

[0146] In one embodiment of the present invention, in the foregoing Figure 3 The main memory data management method shown also includes the following steps A-B.

[0147] Step A: If the first free space is sufficient to store the data to be stored, the first status information is fed back to the data receiving service that sent the data to be stored.

[0148] The first status information mentioned above indicates that the data to be stored has been successfully stored.

[0149] Specifically, the aforementioned data receiving service receives and sends the data to be stored to the data processor, enabling the data processor to store the data in main memory. This data receiving service can receive the data from acquisition devices, networks, etc.

[0150] Once the data receiving service receives the first status information and determines that the data to be stored has been successfully stored, it can continue to send the next piece of data to be stored.

[0151] Step B: If the first free space is insufficient to store the data to be stored, send the second status information to the data receiving service.

[0152] The second status information mentioned above indicates that the data to be stored has failed.

[0153] If the data receiving service receives the second status information and determines that the data to be stored has failed to be stored, it can either pause sending the data to be stored or wait for a preset time before resending the data to be stored to the data processor.

[0154] As described in step S304 above, before storing data into the second free space, the end position of the first free space is the end position of the second storage space. That is, there is a continuous, unused free space before the end position of the second storage space. In order to make full use of the second storage space, the free space before the end position of the second storage space can be removed.

[0155] For this reason, see Figure 5 This is a flowchart illustrating the second main memory data management method provided in this embodiment of the invention, which is consistent with the aforementioned... Figure 3 Compared to the embodiment shown, the steps S305-S306 are included before step S304.

[0156] S305: Update the second starting position of the second storage space to the first position.

[0157] Wherein, the first position is: the position of the second data volume before the second starting position before the update, and the second data volume is: the amount of data that the first free space can accommodate.

[0158] S306: Update the termination position of the second storage space to the second position.

[0159] The second position mentioned above refers to the position of the second data volume before the termination position before the update.

[0160] Specifically, in this embodiment of the invention, steps S305-S306 are only executed when the first free space before the termination position of the second storage space is insufficient to store the data to be stored. Therefore, when executing S305, it can be determined that the data to be stored will not be stored in the first free space, and the first free space will not be used. The first free space can be removed, and the termination position of the second storage space can be updated to a second position. Since the second position is the position of a second amount of data before the termination position before the update, it is equivalent to moving the termination position of the second storage space forward by a second amount of data. The second amount of data is the amount of data that the first free space can accommodate before the termination position is updated, so it is equivalent to the first free space being removed after the update.

[0161] Furthermore, to ensure the size of the second storage space remains unchanged, the second starting position can be shifted forward by the same amount, thus updating the second starting position to the first position. Also, as described in step S304 above, before data is stored in the second free space, the first starting position of the second free space is the same as the second starting position of the second storage space. Therefore, shifting the second starting position of the second storage space forward is equivalent to shifting the first starting position of the second free space forward, thereby expanding the second free space and allowing it to store more data. After updating the second starting and ending positions of the second storage space, step S304 is executed again to write the data to be stored into the second free space.

[0162] See Figure 6A This is a schematic diagram of the seventh data storage structure provided in an embodiment of the present invention. Figure 6A The illustrated embodiment is performed after executing the aforementioned steps S305-S306, from the aforementioned Figure 4B This is based on the embodiment shown. (As described above) Figure 4B In comparison, the second start position and the end position were moved forward by the same amount. The first free space, which was originally after the write position, i.e. after data packet c, was removed, while the second free space, which was before the read position, was increased.

[0163] See Figure 6B This is a schematic diagram of the eighth data storage structure provided in this embodiment of the invention. Figure 6B The illustrated embodiment is in Figure 6A This is obtained by performing the aforementioned step S304 based on the second storage space shown. Figure 6A In contrast, data packet d is written into the second free space before the read position, and the write position is updated to the position where the data tail of data packet d is located.

[0164] Instead of executing steps S305-S306, the data packet d is directly written. Figure 4BThe aforementioned obtained in the second free space shown Figure 4C Compared to the data storage structure shown, the data obtained after executing steps S305-S306... Figure 6B In the second storage space shown, there is no unused free space after data packet c, and the second storage space is fully utilized.

[0165] As can be seen from the above, when the first free space is insufficient to store the data to be stored, the end position of the second storage space and the second start position can be moved forward by the same magnitude. Without changing the size of the second storage space, the unused first free space is removed and the second free space is increased, so that the second free space can store more data and make reasonable use of data storage resources.

[0166] See Figure 7 This is a flowchart illustrating a third main memory data management method provided in an embodiment of the present invention. The method includes the following steps S701-S704. It should be noted that, since the processes of writing and reading data in the main memory data management method can run independently and synchronously, the description of the data reading process... Figure 7 It does not include Figure 3 The steps shown are for writing data. But Figure 7 The data read in the illustrated embodiment is obtained through the aforementioned Figure 3 The illustrated embodiment is written to the second storage space.

[0167] S701: Upon receiving a data extraction request sent by a data processing service, read and send the target data to the data processing service so that the data processing service processes the target data.

[0168] The target data mentioned above refers to the data located after the read position within the second storage space.

[0169] In one embodiment of the present invention, all data located after the aforementioned reading position can be sent to the data processing service as target data, or data of a preset amount starting from the reading position can be sent to the data processing service as target data.

[0170] For example, see Figure 4A In the illustrated embodiment, the target data can be all data located after the read position, i.e., data packets a-c. Alternatively, it can be a portion of data with a predetermined amount of data after the read position, such as data packets a-b.

[0171] Furthermore, the aforementioned data processing services can often only process complete data, not just a portion of the data. Therefore, the minimum amount of data that a data processing service can successfully process at one time can be termed a data unit. For example, if the target data is matrix data, the data unit can be a matrix; if the target data is image data, the data unit can be an image; if the target data is video data, the data unit can be a video frame; and if the target data is audio data, the data unit can be an audio frame.

[0172] Specifically, as seen above, after each piece of data to be stored is sequentially stored in the second storage space, they are interconnected within the second storage space. This is equivalent to the data being concatenated within the second storage space; once data is stored in the second storage space, it is no longer strictly distinguished by the unit of data to be stored. Data that can form a complete data unit can come from the same piece of data to be stored, or it can come from different pieces of data to be stored. For example, see... Figure 4A The data storage structure shown allows a complete data unit to contain data that can come entirely from data packet a, or the first half can come from data packet a and the second half from data packet b.

[0173] If the target data contains data that cannot form a complete data unit, then even if the data processing service receives the target data, it cannot process that part of the data. To prevent the loss of this part of the data, it can continue to be stored in the second storage space. Data that has been successfully processed by the data processing service does not need to be stored in the second storage space and can be removed from the second storage space.

[0174] In one embodiment of the present invention, if the data processing service successfully processes the target data, the processing status information can indicate that the data processing service has successfully processed the data, and the processing status information can include information about the data successfully processed by the data processing service. Depending on the specific circumstances, the processed data can be the entire target data, that is, the data processing service completely processes the target data, or the processed data can be a part of the target data.

[0175] In addition, the target data mentioned above may not be able to form any complete data unit, and the data processing service will not be able to successfully complete any data processing. Therefore, the above processing status information can indicate that the data processing service has failed to process the data.

[0176] S702: Receive the processing status information fed back by the above data processing service.

[0177] S703: If the above processing status information indicates that the data processing was successful, then update the above read position to the third position.

[0178] The third position mentioned above refers to the location of the data tail of the data successfully processed by the aforementioned data processing service before reading.

[0179] In one embodiment of the present invention, data that has been successfully processed by the data processing service will not be processed again. Therefore, such data does not need to be stored in the second storage space. The successfully processed data can be removed from the second storage space, and the above-mentioned read position is updated to the third position where the data tail of the data successfully processed by the above-mentioned data processing service was located before reading. This allows the next time the target data is read based on the read position, the reading can start from the data head of the unprocessed data.

[0180] Specifically, the processing status information mentioned above may include the amount of processed data for successfully processed data. In this case, the third position is the position of the amount of processed data after the read position before the update. If the read position is represented in the form of an address, the amount of processed data can be added to the address indicating the read position to obtain the address indicating the third position, which is then used as the updated address indicating the read position.

[0181] As mentioned above Figure 4A Taking the data storage structure shown as an example, the target data located after the read position can be the aforementioned data packets a-c. If data packets a and b form a complete data unit, and data packet c is not a complete data unit, then data packets a and b can be successfully processed by the data processing service, while data packet c is not successfully processed. In this case, data packets a and b can be removed from the second storage space, and the read position is updated to the original position of the data tail of data packet b in the second storage space. Specifically, the data storage structure in the updated second storage space can be found in the aforementioned... Figure 4B .

[0182] S704: If the above processing status information indicates that the data processing failed, then if there is data in the second storage space that is located before the read position, the target data is moved to the position before the second starting position of the second storage space, and both the second starting position and the read position are updated to the position where the target data header is located after the move.

[0183] Specifically, if the above processing information indicates that the data processing failed, it means that the target data does not contain any complete data units. In this case, the target data still needs to be stored in the second storage space, and there is no other data after the target data in the second storage space.

[0184] In one scenario, if the second storage space currently only contains the target data, then it can wait for the data subsequently written into the second storage space to combine with the target data to form a complete data unit.

[0185] In another scenario, if the second storage space not only contains the target data, but also contains data located before the read position, it indicates that data was written to the second free space through the aforementioned step S304. When this data is written to the second free space, it is written starting from the first starting position of the second free space. Based on the content described in the aforementioned step S304, before writing data into the second free space, the first starting position of the second free space is the second starting position of the second storage space, which means that the data header of the data located before the read position is located at the second starting position.

[0186] Furthermore, the current write position is located at the end of the data stored before the read position. If there is new data to be stored and written to the second storage space, the data to be stored will be written to the free space before the read position starting from the current write position. That is, it will not be written after the target data located after the read position, and therefore will not be connected with the target data to form a complete data unit.

[0187] Since no new data will be stored after the target data to form a complete data unit with the target data, the target data can be moved to the position before the data before the read position, so that the target data is connected with the data before the read position. That is, with the second starting position of the second storage space as the tail, the target data is moved to the position before the second starting position, so that the target data and the data before the read position form a complete data unit, realizing the physical continuity of the stored data, that is, the data is stored continuously in the main memory.

[0188] Specifically, the target data can be moved to the second starting position using the following step C.

[0189] Step C: Use memcpy (main memory copy) or memmove (main memory move) to move the target data to the second starting position.

[0190] Furthermore, since the target data is moved outside the second storage space before being moved to the second starting position, the location of the second storage space can be adjusted. The second starting position of the second storage space can be updated to the location of the target data header after the move, thus ensuring that the target data remains contained within the second storage space. Moreover, because the location of the target data with the longest storage duration changes, the read position of the data header of the longest-stored data also changes along with the target data, updating to the location of the target data header after the move.

[0191] As mentioned above Figure 6BTaking the data storage structure shown as an example, the above data packet c is the target data located after the read position. There is a data packet d before the read position. The data header of data packet d is located at the second starting position. Therefore, data packet c can be moved to the position before the second starting position, and the second starting position and read position can be updated to the position where the data header of data packet c is located.

[0192] See Figure 8A This is a schematic diagram of the ninth data storage structure provided in an embodiment of the present invention. Figure 8A It shows in Figure 6B Based on the data storage structure shown, this is the data storage structure of the second storage space after data transfer of data packet c. (Similar to the aforementioned...) Figure 6B Compared to the embodiment shown, data packet c is currently located before data packet d, and the second start position and read position are the positions where the data header of data packet c is located.

[0193] As can be seen from the above, embodiments of the present invention can read target data from the second storage space, send the target data to a data processing service for processing, and then manage the second storage space based on the processing status information fed back by the data processing service. Since in embodiments of the present invention, when writing data, it is not strictly required that the data be written after the data already stored in the second storage space; that is, data written later that can form a complete data unit with the target data may not be located after the target data, but before the target data. Therefore, if the processing status information indicates processing failure and the target data cannot form a complete data unit, the target data can be moved to before the second preset position, that is, moved to before the data located before the reading position, so that the target data can be connected with the data located before the reading position, thereby potentially forming a complete data unit for subsequent normal data processing. Furthermore, the amount of data moved in this solution is less than the amount of data in a data unit; therefore, embodiments of the present invention can not only reduce the number of data moves but also reduce the amount of data moved.

[0194] In combination with the above Figure 7 The illustrated embodiment and steps A-B above are similar. When this embodiment of the invention is applied to multimedia data management, please refer to... Figure 8B This is a schematic diagram of a multimedia data management process provided in an embodiment of the present invention.

[0195] Specifically, the data receiving service sends data packets to the main memory. The main memory determines whether the first free space is sufficient to store the data packets. If it is sufficient, it sends a first status message back to the data receiving service; if it is insufficient, it sends a second status message back to the data receiving service. The data packets contain encapsulated and encoded multimedia data.

[0196] The main memory sends the encapsulated and encoded multimedia data to the data processing service. The data processing service determines whether the received multimedia data contains at least one complete frame of data. If so, it performs decapsulation and decoding processing on each frame of data and feeds back to the data processor information indicating the amount of data containing the decapsulated and decoded data, indicating successful processing. Otherwise, it feeds back to the data processor information indicating processing failure.

[0197] In addition, in related technologies, as mentioned above Figure 2A In the illustrated embodiment, the cache size differs significantly from the main memory size. When the data processor reads data from main memory, it can only read a small amount of data, within the cache's capacity, at a time. Furthermore, since the data processor often only determines the approximate location of the requested data in main memory, the small amount of data read based on this approximate location may not be the actual data requested by the CPU. This problem can be termed a cache miss. To obtain the data requested by the CPU, the CPU needs to read data from main memory multiple times, impacting CPU efficiency. However, in this embodiment, the read location is explicitly specified, allowing the CPU to read data from that location, significantly reducing the frequency of cache misses and thus improving CPU efficiency.

[0198] Furthermore, as can be seen from the above, in this embodiment of the invention, data transfer may be performed on target data that does not contain complete data units. Before the target data is transferred to the second storage space before the update, the maximum data volume of the target data is infinitely close to the data volume of a complete data unit. Therefore, the storage space in the first storage space located before the second storage space theoretically needs to be able to accommodate a complete data unit. For this purpose, the initial position of the second storage space can be set to the end of the main memory, thereby reserving a sufficiently large storage space before the second storage space. Alternatively, the size of the second storage space can be set based on the data volume of a complete data unit, thereby reserving a sufficiently large storage space before the second storage space. For example, the size of the second storage space can be half or less than the size of the first storage space.

[0199] Since the second starting position of the second storage space is updated in step S704, if the ending position of the second storage space is not updated accordingly, the size of the second storage space will change.

[0200] For this reason, see Figure 9 This is a flowchart illustrating the fourth main memory data management method provided in this embodiment of the invention, which is consistent with the aforementioned... Figure 7 Compared to the embodiment shown, the step S705 is included after step S704 described above.

[0201] S705: Update the termination position of the second storage space to the fourth position.

[0202] The fourth position mentioned above refers to the position of the first data volume before the aforementioned termination position before the update, and the first data volume refers to the data volume of the aforementioned target data.

[0203] In one embodiment of the present invention, since the target data is moved in step S704, the data tail of the target data is located at the second starting position before the update, while the updated second starting position is located at the position of the data head of the moved target data. Therefore, it is equivalent to the second starting position being moved forward by the first data amount of the target data. Accordingly, in order to keep the size of the second storage space unchanged after the second starting position is updated, the ending position of the second storage space can also be moved forward by the first data amount, that is, the ending position of the second storage space is updated to the fourth position.

[0204] As can be seen from the above, after the second starting position of the second storage space is moved forward, the ending position of the second storage space in this embodiment will be moved forward by the same amount, so that the size of the second storage space in this embodiment remains unchanged.

[0205] See Figure 10 This is a flowchart illustrating the fifth main memory data management method provided in this embodiment of the invention, which is consistent with the aforementioned... Figure 3 Compared to the embodiments shown, the above method further includes the following steps S307-S308.

[0206] From the previous text Figure 5 , Figure 7 , Figure 9 As can be seen from the embodiments shown, during the operation of the embodiments of the present invention, the second starting position of the second storage space may be moved forward, that is, the position of the second storage space may be moved forward. During the cyclic execution of the embodiments of the present invention, the second storage position may move forward and move to the front of the first storage space, making the size of the storage space before the second storage space smaller. In the above case, if the target data is moved, the storage space before the second storage space may be difficult to accommodate the target data. In order to solve the above problem, steps S307-S308 can be executed.

[0207] S307: When the above-mentioned read position is located before the write position, determine whether the size of the third storage space is less than the size of the fourth storage space.

[0208] The third storage space is the storage space located before the read position in the second storage space, and the fourth storage space is the storage space located after the write position in the first storage space.

[0209] Specifically, if the size of the third storage space is smaller than the size of the fourth storage space, it means that the data stored in the second storage space is earlier, so the position of the second storage space can be updated, and step S308 is executed; otherwise, no operation is required.

[0210] In one embodiment of the present invention, step S307 can be executed after the second starting position of the second storage space changes, or step S307 can be executed according to a preset period to determine whether the position of the second storage space needs to be updated.

[0211] S308: Update the second starting position of the second storage space to the read position, and update the ending position of the second storage space based on the second starting position of the updated second storage space according to the size of the second storage space before the update.

[0212] Specifically, the second starting position of the second storage space can be updated to the aforementioned read position, which is equivalent to shifting the second starting position of the second storage space backward, and updating the ending position of the second storage space based on the second starting position of the updated second storage space according to the size of the second storage space before the update, so that the size of the second storage space remains unchanged before and after the update.

[0213] As can be seen from the above, when the size of the third storage space is smaller than the size of the fourth storage space, in order to prevent the third storage space from being unable to store the target data, the position of the second storage space can be updated, and the second storage space can be moved backward without moving the data in the second storage space.

[0214] Corresponding to the above-described main memory data management method, this embodiment of the invention also provides a main memory data management device.

[0215] See Figure 11 This is a schematic diagram of the structure of a first type of main memory data management device provided in an embodiment of the present invention. The device includes:

[0216] Data receiving module 1101 is used to receive data to be stored;

[0217] The space judgment module 1102 is used to determine whether the first free space in the second storage space in the main memory after the write position is sufficient to store the data to be stored. The second storage space is located in the fixed first storage space in the main memory. The size of the second storage space is smaller than the size of the first storage space. The write position is the position where the data tail of the most recently written data is located.

[0218] The first storage module 1103 is used to store the data to be stored into the first free space, starting from the write position, when the space determination module determines that the first free space is sufficient to store the data to be stored.

[0219] The second storage module 1104 is used to store the data to be stored in the second free space, starting from the first starting position of the second free space, when the space judgment module determines that the first free space is insufficient to store the data to be stored, and when the second free space is sufficient to store the data to be stored. The second free space is: the free storage space in the second storage space located before the read position, and the read position is: the position where the data header of the data with the longest storage duration is located.

[0220] As can be seen from the above, when managing main memory using the solution provided in this embodiment of the invention, if the first free space after the write position can store the data to be stored, the data to be stored can be directly stored in the first free space. Conversely, if the first free space is insufficient to store the data to be stored, it can be determined whether the second free space before the read position is sufficient to store the data to be stored. If it is sufficient, the data to be stored can be stored in the second free space before the read position. In related technologies, the data to be stored is stored at the end of the already stored data, that is, in the first free space after the already stored data. However, when storing data using this solution, not only can the data to be stored be written into the first free space located after the data already stored in main memory, but also, if the first free space is small, the data to be stored can be stored in the second free space before the read position, which is different from the first free space. In this process, the data transfer process in related technologies is not required. Therefore, when managing main memory data, this embodiment of the invention can make full use of the free storage space in the first storage space, thereby reducing the number of data transfers and improving the utilization rate of the data processor while still achieving data storage.

[0221] In one embodiment of the present invention, the apparatus further includes:

[0222] The first position update module is used to update the second starting position of the second storage space to the first position before storing the data to be stored in the second free space, wherein the first position is: the position of the second amount of data before the second starting position before the update, and the second amount of data is: the amount of data that the first free space can accommodate;

[0223] The second position update module is used to update the termination position of the second storage space to a second position, wherein the second position is the position of the second data volume before the termination position before the update.

[0224] As can be seen from the above, when the first free space is insufficient to store the data to be stored, the end position of the second storage space and the second start position can be moved forward by the same magnitude. Without changing the size of the second storage space, the unused first free space is removed and the second free space is increased, so that the second free space can store more data and make reasonable use of data storage resources.

[0225] In one embodiment of the present invention, the second storage module 1104 is specifically used for:

[0226] If the space determination module determines that the first free space is insufficient to store the data to be stored, and if the second free space is sufficient to store the data to be stored, the target data is moved to the position before the second starting position by using the main memory copy memcpy method or the main memory move memmove method, starting from the first starting position of the second free space.

[0227] or

[0228] The data to be stored is audio data, image data, or video data.

[0229] See Figure 12 This is a schematic diagram of the structure of a second main memory data management device provided in an embodiment of the present invention. The device includes:

[0230] The data reading module 1201 is used to read and send target data to the data processing service when a data extraction request is received from the data processing service, so that the data processing service can process the target data. The target data is data located after the reading position in the second storage space.

[0231] Information receiving module 1202 is used to receive processing status information fed back by the data processing service;

[0232] The third position update module 1203 is used to update the read position to the third position if the processing status information indicates that the data processing was successful. The third position is the position of the data tail of the data successfully processed by the data processing service before reading.

[0233] The fourth position update module 1204 is used to, if the processing status information indicates that the data processing has failed, and if there is data in the second storage space that is located before the read position, move the target data to before the second starting position, with the second starting position of the second storage space as the tail, and update both the second starting position and the read position to the position where the target data header is located after the move.

[0234] As can be seen from the above, embodiments of the present invention can read target data from the second storage space, send the target data to a data processing service for processing, and then manage the second storage space based on the processing status information fed back by the data processing service. Since in embodiments of the present invention, when writing data, it is not strictly required that the data be written after the data already stored in the second storage space; that is, data written later that can form a complete data unit with the target data may not be located after the target data, but before the target data. Therefore, if the processing status information indicates processing failure and the target data cannot form a complete data unit, the target data can be moved to before the second preset position, that is, moved to before the data located before the reading position, so that the target data can be connected with the data located before the reading position, thereby potentially forming a complete data unit for subsequent normal data processing. Furthermore, the amount of data moved in this solution is less than the amount of data in a data unit; therefore, embodiments of the present invention can not only reduce the number of data moves but also reduce the amount of data moved.

[0235] In one embodiment of the present invention, the fourth position update module 1204 is further configured to:

[0236] The termination position of the second storage space is updated to the fourth position, wherein the fourth position is the position of the first data volume before the termination position before the update, and the first data volume is the data volume of the target data.

[0237] As can be seen from the above, after the second starting position of the second storage space is moved forward, the ending position of the second storage space in this embodiment will be moved forward by the same amount, so that the size of the second storage space in this embodiment remains unchanged.

[0238] In one embodiment of the present invention, the apparatus further includes:

[0239] A size determination module is used to determine whether the size of the third storage space is less than the size of the fourth storage space when the read position is located before the write position, wherein the third storage space is the storage space in the second storage space located before the read position, and the fourth storage space is the storage space in the first storage space located after the write position;

[0240] The data transfer module is used to update the second starting position of the second storage space to the read position if the size determination module determines that the size of the third storage space is less than the size of the fourth storage space, and to update the ending position of the second storage space based on the second starting position of the updated second storage space according to the size of the second storage space before the update.

[0241] As can be seen from the above, when the size of the third storage space is smaller than the size of the fourth storage space, in order to prevent the third storage space from being unable to store the target data, the position of the second storage space can be updated, and the second storage space can be moved backward without moving the data in the second storage space.

[0242] In one embodiment of the present invention, the apparatus further includes:

[0243] The first information sending module is configured to, if the first free space is sufficient to store the data to be stored, send back first status information to the data receiving service that sends the data to be stored, wherein the first status information indicates that the data to be stored has been successfully stored.

[0244] and / or

[0245] The second information sending module is used to send a second status information to the data receiving service if the first free space is insufficient to store the data to be stored, wherein the second status information indicates that the data storage to be stored has failed.

[0246] This invention also provides an electronic device, such as... Figure 13 As shown, it includes a processor 1301, a communication interface 1302, a memory 1303, and a communication bus 1304. The processor 1301, the communication interface 1302, and the memory 1303 communicate with each other through the communication bus 1304.

[0247] Memory 1303 is used to store computer programs;

[0248] When the processor 1301 executes the program stored in the memory 1303, it implements any of the steps described in the above main memory data management method.

[0249] When managing main memory data using the electronic device provided in this embodiment of the invention, if the first free space after the write position can store the data to be stored, the data to be stored can be directly stored in the first free space. Conversely, if the first free space is insufficient to store the data to be stored, it can be determined whether the second free space before the read position is sufficient to store the data to be stored. If it is sufficient, the data to be stored can be stored in the second free space before the read position. In related technologies, the data to be stored is stored at the end of the already stored data, that is, in the first free space after the already stored data. However, when storing data using this solution, not only can the data to be stored be written into the first free space after the data already stored in the main memory, but also, if the first free space is small, the data to be stored can be stored in the second free space before the read position, which is different from the first free space. In this process, the data transfer process in related technologies is not required. Therefore, when managing main memory data, this embodiment of the invention can make full use of the free storage space in the first storage space, thereby reducing the number of data transfers and improving the utilization rate of the data processor while still achieving data storage.

[0250] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0251] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0252] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0253] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0254] In another embodiment of the present invention, a computer-readable storage medium is also provided, wherein a computer program is stored therein, and when the computer program is executed by a processor, it implements the steps of any of the main memory data management methods described above.

[0255] Specifically, the aforementioned computer-readable storage medium is the main memory in this embodiment of the invention. After receiving the data to be stored, the data processor can temporarily store the data to be stored in the cache inside the data processor, and then transfer the data in the cache inside the data processor to the computer-readable storage medium, i.e., the main memory. When it is necessary to read the data, the data processor can also extract the stored data from the computer-readable storage medium, i.e., the main memory, and temporarily store it in the cache inside the data processor.

[0256] The data access speed of the cache inside the data processor is often less than 1ns, while the data access speed of the main memory is often around 65ns. The access speeds of the two are quite different. In addition, the storage space of the cache inside the data processor is often 4k-64k, while the storage space of the main memory is often 4G-32G. The storage space sizes of the two are also quite different, making it difficult for them to interact directly. Therefore, data interaction between the cache inside the data processor and the main memory can be achieved by using an external cache outside the data processor.

[0257] The storage space of the external cache of the data processor is often larger than that of the internal cache of the data processor, but smaller than that of the main memory. For example, the storage space of the aforementioned cache is often between 2MB and 16MB.

[0258] Specifically, after acquiring the data to be stored, the data processor can store the data in the storage unit inside the data processor, namely the cache inside the data processor. Then, the data in the cache inside the data processor can be transferred and stored in the cache outside the data processor. After that, the cache outside the data processor can be controlled to transfer the stored data to the computer-readable storage medium, namely the main memory.

[0259] When a data processor reads stored data from a computer-readable storage medium, i.e., main memory, the data processor can first control the computer-readable storage medium to store the read data in a cache outside the data processor, and then control the cache outside the data processor to store the stored data in a cache inside the data processor.

[0260] When performing main memory data management using a computer program stored in a computer-readable storage medium for an agent, as provided in this embodiment of the invention, if the first free space after the write position can store the data to be stored, the data to be stored can be directly stored in the first free space. Conversely, if the first free space is insufficient to store the data to be stored, it can be determined whether the second free space before the read position is sufficient to store the data to be stored. If it is sufficient, the data to be stored can be stored in the second free space before the read position. In related technologies, the data to be stored is stored at the end of the already stored data, that is, in the first free space after the already stored data. However, when storing data using this solution, not only can the data to be stored be written into the first free space after the data already stored in the main memory, but also, if the first free space is small, the data to be stored can be stored in the second free space before the read position, which is different from the first free space. This process does not require the data transfer process in related technologies. Therefore, when managing main memory data, this embodiment of the invention can make full use of the free storage space in the first storage space, thereby reducing the number of data transfers and improving the utilization rate of the data processor while still achieving data storage.

[0261] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform the steps of any of the main memory data management methods described above.

[0262] When performing main memory data management using the computer program provided in this embodiment of the invention, if the first free space after the write position can store the data to be stored, the data to be stored can be directly stored in the first free space. Conversely, if the first free space is insufficient to store the data to be stored, it can be determined whether the second free space before the read position is sufficient to store the data to be stored. If it is sufficient, the data to be stored can be stored in the second free space before the read position. In related technologies, the data to be stored is stored at the end of the already stored data, that is, in the first free space after the already stored data. However, when storing data using this solution, not only can the data to be stored be written into the first free space after the data already stored in the main memory, but if the first free space is small, the data to be stored can also be stored in the second free space before the read position, which is different from the first free space. In this process, the data transfer process in related technologies is not required. Therefore, when managing main memory data, this embodiment of the invention can make full use of the free storage space in the first storage space, thereby reducing the number of data transfers and improving the utilization rate of the data processor while still achieving data storage.

[0263] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0264] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0265] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, for devices, electronic devices, computer-readable storage media, and computer program products, since they are basically similar to the method embodiments, the descriptions are relatively simple, and relevant parts can be referred to the descriptions of the method embodiments.

[0266] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A main memory data management method characterized by comprising: The method comprises: receiving data to be stored; determining whether a first free space after a write position in a second storage space in the main memory is sufficient to store the data to be stored, wherein the second storage space is in a fixed first storage space in the main memory, the size of the second storage space is smaller than the size of the first storage space, and the write position is the position of the data tail of the last time data is written; if it is sufficient, storing the data to be stored to the first free space from the write position; if it is not sufficient, in the case that a second free space is sufficient to store the data to be stored, storing the data to be stored to the second free space from a first start position of the second free space, wherein the second free space is a free storage space before a read position in the second storage space, the read position is the position of the data head of the data with the longest storage time, after reading data from the read position, the data after the read position is no longer stored in the second storage space, and the second storage space storing the read data becomes a free storage space; if the second free space is not sufficient to accommodate the data to be stored, ending the process.

2. The method of claim 1, wherein, Before storing the data to be stored to the second free space from the start position of the second free space in the case that the second free space is sufficient to store the data to be stored, the method further comprises: updating a second start position of the second storage space to a first position, wherein the first position is a position before a second data amount before the update of the second start position, and the second data amount is the amount of data that the first free space can accommodate; updating an end position of the second storage space to a second position, wherein the second position is a position before a second data amount before the update of the end position.

3. The method of claim 1, wherein, The method further comprises: in the case that a data extraction request sent by a data processing service is received, reading and sending target data to the data processing service to make the data processing service process the target data, wherein the target data is data after the read position in the second storage space; receiving processing condition information fed back by the data processing service; if the processing condition information indicates that the data is processed successfully, updating the read position to a third position, wherein the third position is the position of the data tail of the data successfully processed by the data processing service before reading; if the processing condition information indicates that the data is not processed successfully, in the case that there is data before the read position in the second storage space, carrying the target data to before the second start position of the second storage space with the second start position as the tail, and updating the second start position and the read position to the position of the data head of the target data after the carrying.

4. The method of claim 3, wherein, After updating the second start position and the read position to the position of the data head of the target data after the carrying, the method further comprises: updating a termination position of the second storage space as a fourth position, wherein the fourth position is a position located a first data amount before the termination position before the updating, and the first data amount is a data amount of the target data.

5. The method of claim 3, wherein, Before the carrying of the target data to the second start position, the method comprises: carrying the target data to the second start position by using a memcpy mode or a memmove mode; or the to-be-stored data is audio data, image data or video data.

6. The method according to any one of claims 1-5, characterized in that, The method further comprises: in a case where the read position is located in front of the write position, determining whether a third storage space is smaller than a fourth storage space, wherein the third storage space is a storage space located before the read position in the second storage space, and the fourth storage space is a storage space located after the write position in the first storage space; if yes, updating a second start position of the second storage space as the read position, and updating a termination position of the second storage space based on the second start position of the second storage space after the updating according to a size of the second storage space before the updating.

7. The method according to any one of claims 1-5, characterized in that, The method further comprises: if the first free space is sufficient to store the to-be-stored data, feeding back first state information to a data receiving service that sends the to-be-stored data, wherein the first state information indicates that the to-be-stored data is stored successfully; and / or if the first free space is insufficient to store the to-be-stored data, sending second state information to the data receiving service, wherein the second state information indicates that the to-be-stored data is stored unsuccessfully.

8. A main data management device, characterized by comprising: The apparatus comprises: a data receiving module configured to receive to-be-stored data; a space determining module configured to determine whether a first free space located after a write position in a second storage space in a main memory is sufficient to store the to-be-stored data, wherein the second storage space is located in a fixed first storage space in the main memory, a size of the second storage space is smaller than a size of the first storage space, and the write position is a position where a data tail of the most recently written data is located; a first storing module configured to, in a case where the space determining module determines that the first free space is sufficient to store the to-be-stored data, store the to-be-stored data to the first free space starting from the write position. A second storage module is configured to, when the space judging module determines that the first free space is insufficient to store the to-be-stored data, store the to-be-stored data into a second free space from a first start position of the second free space, when the second free space is sufficient to store the to-be-stored data, wherein the second free space is a free storage space in the second storage space before a read position, the read position is a position of a data head of data with the longest storage time, after reading data from the read position, data after the read position is no longer stored in the second storage space, the second storage space storing the read data becomes a free storage space, and when the second storage module determines that the second free space is insufficient to store the to-be-stored data, the process ends.

9. An electronic device, comprising: The device comprises a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; The memory is configured to store a computer program. The processor is configured to execute the program stored in the memory to implement the method steps in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the method steps in any one of claims 1-7.

Images