A method, device and electronic device for sharing cache
By creating shared memory in audio and video data processing and using producers and consumers to process data, the problem that fixed-sized memory blocks are difficult to meet dynamic changes is solved, memory utilization and data insertion speed are improved, and delay is reduced. .
Patent Information
- Application Number
- CN202110442251.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-04-23
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2041-04-23
AI Technical Summary
In the prior art, fixed-sized memory blocks are difficult to meet the dynamically changing audio and video data processing requirements, resulting in low memory utilization and large data processing delays.
By creating shared memory and establishing its virtual memory address mapping relationship with the target process, producers and consumers insert and process data to be consumed in shared memory respectively, and dynamically allocate memory through sequential insertion and sequential overlay, supporting multiple production and multi-consumer, improving memory utilization and data insertion speed.
It improves memory utilization, reduces data processing delay, supports multiple producers and multiple consumers, flexibly uses shared memory, and adapts to audio and video data processing needs.
Smart Images

Figure CN113176942B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer technology, and in particular to a method, device and electronic device for sharing cache. Background Art
[0002] With the advancement of technology, the performance of electronic equipment and other hardware has been continuously improved. In terms of video data processing, as the amount of video data increases, the video resolution is also constantly improving, and the functions of video display are becoming more complex. As a result, the amount of audio and video data that electronic equipment needs to process is increasing, which makes the memory capacity requirements of electronic equipment higher and higher.
[0003] In order to alleviate the demand for memory capacity of electronic devices, a method of using a shared cache is to divide a memory pool into blocks of fixed sizes, and then use the divided blocks to process audio and video data.
[0004] However, in actual applications, audio and video data can change dynamically, and fixed-size memory blocks can hardly meet the needs of audio and video data processing, resulting in low memory utilization. Real-time real-time division of the memory pool according to the needs of the audio and video data to be processed is time-consuming and will cause a large data processing delay. Summary of the invention
[0005] The purpose of the embodiments of the present invention is to provide a method, device and electronic device for sharing cache to improve memory utilization and reduce data processing delay. The specific technical solution is as follows:
[0006] In a first aspect, an embodiment of the present invention provides a method for sharing a cache, the method comprising:
[0007] Create and initialize shared memory;
[0008] Establishing a mapping relationship between the address of the shared memory and the virtual memory address of the target process;
[0009] Creating a producer according to a create producer instruction, wherein the producer is used to insert data to be consumed into the shared memory;
[0010] Create a consumer according to a create consumer instruction, wherein the consumer is used to process the data to be consumed inserted into the shared memory;
[0011] The producer inserts the data to be consumed into the shared memory by sequential insertion and sequential overwriting, and sends information about the data to be consumed to the consumer, wherein the sequential insertion and sequential overwriting represent repeatedly traversing the data area of the shared memory from beginning to end, so as to insert the data to be consumed into the shared memory by sequential insertion and sequential overwriting;
[0012] The consumer obtains the data to be consumed from the shared memory according to the information of the data to be consumed, and processes the data to be consumed.
[0013] Optionally, the shared memory includes: a shared memory header control information area and a data area, the data area includes a plurality of data blocks, and the producer inserts the to-be-consumed data into the shared memory in a sequential insertion and sequential overwriting manner, including:
[0014] The producer sequentially traverses the data area of the shared memory, and when the available space at the end of the data area is insufficient to insert the data to be consumed, or retraverses the data area, and when the available space before the occupied data block is insufficient to insert the data to be consumed, the data block is skipped, and target data blocks capable of inserting the data to be consumed are sequentially obtained from the data area;
[0015] The data to be consumed is written into the target data block.
[0016] Optionally, the shared memory includes: a shared memory header control information area and a data area, the data area includes a plurality of data blocks, and the data block includes a data header and data; the step of the producer inserting the data to be consumed into the shared memory in a sequential insertion and sequential overwriting manner includes:
[0017] Obtaining the data usage offset position in the shared memory;
[0018] Traversing the data area based on the data using the offset position to determine the acquired memory size;
[0019] Determine whether the acquired memory size is smaller than the sum of the target memory size and a data header size, and whether the data area has been traversed;
[0020] If yes, return to the step of traversing the data area based on the data usage offset position to determine the acquired memory size;
[0021] If not, determining the location of the target memory according to the acquired memory size and the relationship between the target memory size and the sum of a data header size, wherein the location of the target memory is used to indicate the address of the data block into which the data to be consumed is inserted;
[0022] Based on the location of the target memory, the to-be-consumed data is inserted into the acquired memory.
[0023] Optionally, the step of traversing the data area based on the data usage offset position to determine the acquired memory size includes:
[0024] Determining whether the data usage offset position points to the end of the data area;
[0025] When the data usage offset position points to the tail end of the data area, the data usage offset position is pointed to the head of the data area, the acquired memory size is cleared, the number of times the shared memory is completely covered is increased by 1, and the position of the target memory is pointed to the head of the data area;
[0026] Determine whether the data block corresponding to the head of the data area pointed to by the data usage offset position is occupied;
[0027] If yes, clear the acquired memory size to zero, and point the location of the target memory to the data header of the next data block in the data area;
[0028] If not, the acquired memory size is updated to be the sum of the historically acquired memory size and the size of the unoccupied data block;
[0029] The data is pointed to the next data block in the data area using an offset position.
[0030] Optionally, the step of determining the location of the target memory according to the acquired memory size and the relationship between the target memory size and the sum of a data header size includes:
[0031] Determine whether the acquired memory size is smaller than the sum of the target memory size and a data header size;
[0032] If yes, return an error;
[0033] If not, subtract the size of a data header from the acquired memory size to obtain the insertable data memory size, and then determine whether the insertable data memory size is larger than the target memory size;
[0034] If the insertable data memory size is larger than the target memory size, determining whether the insertable data memory size is smaller than the sum of the target memory size and a data header size;
[0035] If the insertable data memory size is not greater than the target memory size, determining the location of the target memory, and determining the acquired memory size as the actual available memory size corresponding to the location of the target memory;
[0036] If the size of the insertable data memory is smaller than the sum of the target memory size and a data header size, determining whether the data header of the next data block of the data area pointed to by the position of the target memory is not the end and is not occupied;
[0037] If the insertable data memory size is not less than the sum of the target memory size and a data header size, the portion of the insertable data memory size that is larger than the target memory size is treated as a separate data block, and the location of the target memory is determined, and the target memory size is determined as the actual available memory size corresponding to the location of the target memory;
[0038] If the data header of the next data block of the data area pointed to by the position of the target memory is not the end and is not occupied, then the part of the insertable data memory whose size is larger than the target memory size is merged into the next data block, and the position of the target memory is determined, and the target memory size is determined as the actual available memory size corresponding to the position of the target memory;
[0039] If the data header of the next data block in the data area pointed to by the position of the target memory is the end and / or is occupied, the part of the insertable data memory size that is larger than the target memory size is placed in the acquired memory, and the position of the target memory is determined, and the acquired memory size is determined as the actual available memory size corresponding to the position of the target memory.
[0040] Optionally, when creating a producer according to a create producer instruction, a consumer subscription bitmap is obtained, where the consumer subscription bitmap is used to represent a consumer bitmap subscribed to the producer; and the step of the producer sending information about data to be consumed to the consumer includes:
[0041] The producer sends information about the data to be consumed to the consumer according to the consumer subscription bitmap;
[0042] Determine whether the subscribed consumer in the consumer subscription bitmap needs to be awakened;
[0043] If yes, wake up the subscribed consumer;
[0044] If not, determine whether there is any consumption data that has not been inserted;
[0045] In the case that there is still data to be consumed that has not been inserted, the process returns to the step of inserting the data to be consumed into the shared memory by the producer in a sequential insertion and sequential overwriting manner.
[0046] Optionally, the method further comprises:
[0047] When the insertion of the data to be consumed is completed, the producer is deregistered, the producer information corresponding to the producer is cleared, and the position relationship of the producer in the consumer usage bitmap corresponding to the consumer subscribing to the producer is updated; the consumer usage bitmap is used to represent the bitmap of the producer information currently being used by the consumer;
[0048] Determine whether the consumer usage bitmap is zero;
[0049] When the consumer usage bitmap is zero, determining whether the subscribed consumer needs to be awakened;
[0050] When the subscribed consumer needs to be awakened, the subscribed consumer is awakened.
[0051] Optionally, the step of the consumer obtaining the data to be consumed from the shared memory according to the information of the data to be consumed, and processing the data to be consumed includes:
[0052] The consumer processes information of the data to be consumed according to the type of the data to be consumed;
[0053] Determine whether the producer in the subscription producer bitmap has been created and is in use, wherein the subscription producer bitmap is used to represent the bitmap of the consumer subscription producer;
[0054] If the producer in the subscription producer bitmap has been created and is in use, determine whether there is information of unread data to be consumed in the message loop queue of the message control information, and the message control information is used to manage the information of the data to be consumed by the consumer;
[0055] When there is information of unread data to be consumed in the message loop queue, checking whether the shared memory is overwritten;
[0056] When there is no information of unread data to be consumed in the message loop queue, waiting to be awakened;
[0057] In the case where the shared memory is not overwritten, a consumer occupation bitmap in a data header of a data block to be consumed is set; the data header of the data block to be consumed is determined according to information of the data to be consumed;
[0058] The consumer obtains the data to be consumed from the shared memory according to the information of the data to be consumed, processes the data to be consumed, and releases the data to be consumed when the processing is completed;
[0059] If the producer in the subscription producer bitmap is not created and in use, or the data to be consumed is released, then determine whether to continue to obtain the data to be consumed from the producer;
[0060] When no longer obtaining data to be consumed from the producer, the consumer is deregistered, the consumer subscription bitmap in the producer is cleared, and the message loop queue in the message control information is cleared;
[0061] While continuing to obtain the data to be consumed from the producer, the step of returning to execute the consumer processing the information of the data to be consumed according to the type of the data to be consumed obtained.
[0062] In a second aspect, an embodiment of the present invention provides a data reading device, the device comprising:
[0063] The first creation module is used to create and initialize the shared memory;
[0064] An address mapping module, used to establish a mapping relationship between the address of the shared memory and the virtual memory address of the target process;
[0065] A second creation module, used for creating a producer according to a producer creation instruction, wherein the producer is used for inserting data to be consumed into the shared memory;
[0066] A third creation module, used to create a consumer according to a create consumer instruction, wherein the consumer is used to process the to-be-consumed data inserted into the shared memory;
[0067] A data insertion module, used for the producer to insert the data to be consumed into the shared memory by sequential insertion and sequential overwriting, and to send information of the data to be consumed to the consumer, wherein the sequential insertion and sequential overwriting represent repeatedly traversing the data area of the shared memory from beginning to end, so as to insert the data to be consumed into the shared memory by sequential insertion and sequential overwriting;
[0068] The data processing module is used for the consumer to obtain the data to be consumed from the shared memory according to the information of the data to be consumed, and to process the data to be consumed.
[0069] In a third aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;
[0070] Memory, used to store computer programs;
[0071] The processor is used to implement the steps of a shared cache method described in the first aspect when executing a program stored in the memory.
[0072] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of a shared cache method described in the first aspect are implemented.
[0073] Beneficial effects of the embodiments of the present invention:
[0074] A method, device and electronic device for sharing cache provided by an embodiment of the present invention creates a shared memory and maps the address of the shared memory to the virtual memory address of the target process, so that the shared memory can be used in multiple processes and multiple threads, and has a wide range of applications. The producer inserts the data to be consumed in the shared memory by sequential insertion and sequential overwriting, and can dynamically allocate memory according to the size of the inserted data to be consumed, without the need to re-partition the memory, thereby improving the utilization rate of the memory. The use of shared memory can reduce the number of repeated data copies, thereby increasing the data insertion speed and reducing the data processing delay. Moreover, sequential insertion and sequential overwriting can eliminate the need for synchronization when processing audio and video data, create producers and consumers, support multiple producers and multiple consumers, and make the use of shared memory more flexible. Of course, implementing any product or method of the present invention does not necessarily require all of the advantages described above to be achieved at the same time. BRIEF DESCRIPTION OF THE DRAWINGS
[0075] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.
[0076] Figure 1 A schematic diagram of a flow chart of a method for sharing cache provided by an embodiment of the present invention;
[0077] Figure 2 A schematic diagram of shared memory allocation provided by an embodiment of the present invention;
[0078] Figure 3 Another shared memory allocation schematic diagram provided for an embodiment of the present invention;
[0079] Figure 4 A schematic diagram of a shared memory header control information structure provided by an embodiment of the present invention;
[0080] Figure 5 A schematic diagram of a data header information structure provided by an embodiment of the present invention;
[0081] Figure 6 A schematic diagram of a shared memory data area provided by an embodiment of the present invention;
[0082] Figure 7 A schematic diagram of an implementation method of inserting data into a shared memory provided by an embodiment of the present invention;
[0083] Figure 8 A schematic diagram of an implementation method of determining the size of an acquired memory provided by an embodiment of the present invention;
[0084] Fig. 9 A schematic diagram of an implementation method of determining a target memory location provided by an embodiment of the present invention;
[0085] Fig.10 A schematic diagram of an implementation method of a producer sending information to a consumer provided by an embodiment of the present invention;
[0086] Fig.11 A schematic diagram of an implementation method of data processing by a consumer provided in an embodiment of the present invention;
[0087] Fig.12 A schematic diagram of the structure of a shared cache device provided by an embodiment of the present invention;
[0088] Fig.13 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0089] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field based on this application belong to the scope of protection of the present invention.
[0090] In order to solve the problems of low memory utilization and long data processing delay in the prior art, the embodiments of the present invention provide a method, device and electronic device for sharing cache.
[0091] A method for sharing a cache provided by an embodiment of the present invention may include:
[0092] Create and initialize a shared memory; establish a mapping relationship between the address of the shared memory and the virtual memory address of the target process; create a producer according to a producer creation instruction, wherein the producer is used to insert the data to be consumed into the shared memory; create a consumer according to a consumer creation instruction, wherein the consumer is used to process the data to be consumed inserted into the shared memory; the producer inserts the data to be consumed into the shared memory in a sequential insertion and sequential overwriting manner, and sends information about the data to be consumed to the consumer, wherein the sequential insertion and sequential overwriting represent repeated traversal of the data area of the shared memory from beginning to end, so as to insert the data to be consumed into the shared memory in a sequential insertion and sequential overwriting manner; the consumer obtains the data to be consumed from the shared memory according to the information about the data to be consumed, and processes the data to be consumed.
[0093] A method for sharing cache provided by an embodiment of the present invention creates a shared memory and maps the address of the shared memory to the virtual memory address of the target process, so that the shared memory can be used in multiple processes and multiple threads, and has a wide range of applications. The producer inserts the data to be consumed in the shared memory by sequential insertion and sequential overwriting, and can dynamically allocate memory according to the size of the inserted data to be consumed, without the need to re-partition the memory, thereby improving the utilization rate of the memory. The use of shared memory can reduce the number of repeated data copies, thereby increasing the data insertion speed and reducing the data processing delay. Moreover, sequential insertion and sequential overwriting can eliminate the need for synchronization when processing audio and video data, and the creation of producers and consumers can support multiple producers and multiple consumers, making the use of shared memory more flexible.
[0094] The following is a detailed description of a shared cache method provided by an embodiment of the present invention:
[0095] like Figure 1 As shown, an embodiment of the present invention provides a method for sharing a cache, which may include the following steps:
[0096] S101, create and initialize shared memory.
[0097] In the embodiment of the present invention, a shared memory is used to store and process data. When using shared memory, the shared memory can be created and initialized first. Preferably, the shared memory in the embodiment of the present invention may include: a shared memory header control information area and a data area, and the shared memory header control information area may include: shared memory information, producer information, consumer information and message control information, the data area may include multiple data blocks, and each data block may include a data header and data. Among them, the message control information is used to manage the information of the consumer's to-be-processed data to be consumed. Exemplarily, the shared memory created in the embodiment of the present invention is allocated as follows Figure 2 and Figure 3 As shown, ShmInfo represents the shared memory header control information.
[0098] Preferably, in the embodiment of the present invention, the shared memory header control information structure of the created shared memory can be designed as follows: Figure 4 As shown, the design of the data header information structure can be as follows Figure 5As shown. When creating and initializing shared memory, the information of each area in the shared memory can be initialized. Among them, ShmInfo.mark (shared memory mark) can be fixed to 0x4D445342, XXXX, XXXX represent field identifiers, the length of a single stream extension information is a reserved field, and ShmInfo.coverCnt (number of covers) can be used to determine whether the data frame is covered. ShmInfo.curCnsmrId (currently used consumer information position) means that the position is increased by 1 for each registered consumer, and is used for cyclic judgment. bitmap represents bitmap, DataStrmInfo represents producer information, CnsmrInfo represents consumer information, MsgCtrl represents message control information, DataStrmInfo.dataCnt, DataStrmInfo.dataUseCnt and DataStrmInfo.newTm are used for debugging and printing. CnsmrInfo.usedStrmBitmap (the bitmap currently in use in subscription) is used to cancel the data stream and clear the bitmap. If all subscribed data streams are cancelled, the error of no stream in use is returned when getting data to avoid continuous blocking. MsgInfo.offset is used for comparison to confirm that it has not been overwritten during use.
[0099] S102, establishing a mapping relationship between the address of the shared memory and the virtual memory address of the target process.
[0100] After creating and initializing the shared memory, a mapping relationship between the address of the shared memory and the virtual memory address of the target process can be established, that is, the address of the shared memory is mapped to the virtual memory address of the target process, or the address of the shared memory is mapped to a part of the virtual memory address of the target process. The shared memory in the embodiment of the present invention can support multi-process and multi-threaded access.
[0101] S103, creating a producer according to the create producer instruction.
[0102] The producer is used to insert the data to be consumed into the shared memory. Preferably, when creating a producer, the producer information corresponding to the producer can also be initialized. For specific initialization parameters, see Figure 4 The parameters in the structure are used to obtain the consumer subscription bitmap, which is used to represent the consumer bitmap that subscribes to the producer.
[0103] S104, creating a consumer according to the create consumer instruction.
[0104] The consumer is used to process the data to be consumed that is inserted into the shared memory. Preferably, when creating a consumer, the consumer information can also be initialized. For specific initialization parameters, see Figure 4 The parameters in the structure are used to obtain the subscription producer bitmap, which is used to represent the bitmap of consumer subscription producers. Message control information can also be generated for each consumer.
[0105] S105, the producer inserts the data to be consumed into the shared memory by sequential insertion and sequential overwriting, and sends the information of the data to be consumed to the consumer.
[0106] Among them, sequential insertion and sequential overwriting mean repeatedly traversing the data area of the shared memory from beginning to end, so as to insert the data to be consumed into the shared memory in a sequential insertion and sequential overwriting manner.
[0107] As an optional implementation of the embodiment of the present invention, the producer may insert the to-be-consumed data into the shared memory by using the sequential insertion and sequential overwriting methods, which may include:
[0108] The producer sequentially traverses the data area of the shared memory. If the available space at the end of the data area is insufficient to insert the data to be consumed, or re-traverses the data area. If the available space before the occupied data block is insufficient to insert the data to be consumed, the data block is skipped and the target data block capable of inserting the data to be consumed is sequentially obtained from the data area.
[0109] Write the data to be consumed into the target data block.
[0110] Exemplarily, a shared memory data area provided by an embodiment of the present invention is as follows: Figure 6 As shown, the producer can traverse the shared memory data area from beginning to end in sequence, and the available space at the end of the data area (such as Figure 6 The available space after the data block 7 shown in FIG. 7 is not enough to insert the data to be consumed, or the data area is traversed again, and the available space before the occupied data block (such as Figure 6 When the available space before the data block 8 (after the data block 8 shown in the figure, and before the data block 2 occupied by the data block 8) is not enough to insert the data to be consumed, the data block is skipped, and the target data block capable of inserting the data to be consumed is sequentially obtained from the entire data area of the shared memory, and then the data to be consumed is written into the target data block. The sequential insertion, sequential use and sequential overwriting of data are realized, wherein the skipped data block is wasted in this use, but can be used again in the subsequent data insertion process, so as to improve the utilization rate of the shared memory.
[0111] After the producer writes the data to be consumed into the target data block, it can send information about the data to be consumed to the consumer, wherein the information about the data to be consumed can be as follows: Figure 4The data header offset information and the data area overwrite times information in the structure shown enable the consumer to obtain the data to be consumed from the shared memory according to the information of the data to be consumed and process the data to be consumed.
[0112] S106, the consumer obtains the data to be consumed from the shared memory according to the information of the data to be consumed, and processes the data to be consumed.
[0113] The consumer may obtain the data to be consumed from the shared memory based on the information of the data to be consumed sent by the producer, specifically the data header offset information, and further process the data to be consumed.
[0114] As an optional implementation of the embodiment of the present invention, after step S105 or S106, the following may also be included:
[0115] Cancel the mapping relationship between the shared memory address and the virtual memory address of the target process.
[0116] After the data is inserted and processed, the mapping relationship between the shared memory address and the virtual memory address of the target process can be cancelled to release the shared memory occupied by the target process. Furthermore, the shared memory can also be destroyed.
[0117] A method for sharing cache provided by an embodiment of the present invention creates a shared memory and maps the address of the shared memory to the virtual memory address of the target process, so that the shared memory can be used in multiple processes and multiple threads, and has a wide range of applications. The producer inserts the data to be consumed in the shared memory by sequential insertion and sequential overwriting, and can dynamically allocate memory according to the size of the inserted data to be consumed, without the need to re-partition the memory, thereby improving the utilization rate of the memory. The use of shared memory can reduce the number of repeated data copies, thereby increasing the data insertion speed and reducing the data processing delay. In addition, sequential insertion and sequential overwriting can eliminate the need for synchronization when processing audio and video data, and the creation of producers and consumers can support multiple producers and multiple consumers, making the use of shared memory more flexible.
[0118] As an optional implementation of the embodiment of the present invention, Figure 7 As shown, Figure 7 A schematic diagram of an implementation method of inserting data into a shared memory provided in an embodiment of the present invention can be used to implement the producer inserting the to-be-consumed data into the shared memory using a sequential insertion and sequential overwriting method in the above step S105. The implementation method may include:
[0119] The shared memory in the embodiment of the present invention may include: a shared memory header control information area and a data area. The data area may include multiple data blocks, and each data block may include a data header and data.
[0120] S201, obtaining the data usage offset position in the shared memory.
[0121] When the created producer inserts the data to be consumed, it can obtain the data usage offset position in the shared memory (i.e. Figure 4 The offset position used by the data area in the shared memory (useOffset), the data use offset position can be a temporary pointer, used to represent the current use status of the data area in the shared memory, specifically, the data use offset position can point to the end of the last inserted data to be consumed, that is, the data block header of the first area to be released sequentially next time. The initial value of the data use offset position can be the offset position of the data write.
[0122] S202, traverse the data area based on the data usage offset position to determine the acquired memory size.
[0123] Based on the data usage offset position, the data area is sequentially traversed to determine the memory size obtained for inserting the data to be consumed. Specifically, the implementation method of traversing the data area based on the data usage offset position to determine the acquired memory size is described in detail below.
[0124] S203, determining whether the acquired memory size is smaller than the sum of the target memory size and a data header size, and whether the data area is traversed completely.
[0125] If the acquired memory size is smaller than the sum of the target memory size and a data header size, and the data area traversal is completed, it indicates that the memory size acquired after traversing the entire data area is not enough to insert the data to be consumed. At this time, the process returns to the step of traversing the data area based on the data usage offset position to determine the acquired memory size, and continues to traverse the data area to determine the acquired memory size. The target memory size indicates the memory size required to insert the data to be consumed.
[0126] S204: If not, determine the location of the target memory according to the acquired memory size and the relationship between the target memory size and the sum of a data header size.
[0127] If not, the location of the target memory is further determined based on the acquired memory size and the relationship between the target memory size and the sum of a data header size. The location of the target memory is used to indicate the address of the data block into which the data to be consumed is inserted.
[0128] S205, inserting the data to be consumed into the acquired memory based on the location of the target memory.
[0129] As an optional implementation of the embodiment of the present invention, Figure 8 As shown, Figure 8 A schematic diagram of an implementation method of determining the acquired memory size provided in an embodiment of the present invention can be used to implement the above step S202 of traversing the data area based on the data usage offset position to determine the acquired memory size. The implementation method may include:
[0130] S301, determining whether the data usage offset position points to the end of the data area.
[0131] S302, when the data usage offset position points to the end of the data area, the data usage offset position is pointed to the head of the data area, the acquired memory size is cleared, the number of times the shared memory is completely covered is increased by 1, and the position of the target memory is pointed to the head of the data area.
[0132] When the data usage offset position points to the end of the data area, the traversal starts again from the head of the entire data area, and then the data usage offset position is pointed to the head of the data area, and the acquired memory size is cleared to zero (that is, the acquired memory size is re-determined), the number of times the shared memory is completely covered is increased by 1, and the location of the target memory is pointed to the head of the data area.
[0133] When the data usage offset position has not yet pointed to the end of the data area, the operation of step S303 is performed.
[0134] S303, determining whether the data block corresponding to the head of the data area pointed to by the data usage offset position is occupied.
[0135] S304: If yes, clear the acquired memory size to zero, and point the location of the target memory to the data header of the next data block in the data area.
[0136] If the data block corresponding to the header of the data area pointed to by the data usage offset position is occupied, it means that the data block cannot be used to insert new data. At this time, it is necessary to redetermine the acquired memory size, clear the acquired memory size, and point the target memory position to the data header of the next data block in the data area.
[0137] S305: If not, update the acquired memory size to the sum of the historically acquired memory size and the size of the unoccupied data block.
[0138] If the data block pointed to by the data usage offset position is not occupied, it indicates that the data block can be used to insert new data. At this time, the acquired memory size is updated to be the sum of the historical acquired memory size and the size of the unoccupied data block.
[0139] S306, point the data usage offset position to the next data block in the data area.
[0140] Use the data offset position to point to the next data block in the data area and continue traversing the data area.
[0141] As an optional implementation of the embodiment of the present invention, Fig. 9 As shown, Fig. 9 A schematic diagram of an implementation method of determining the location of a target memory provided by an embodiment of the present invention can be used to implement the above step S204, and the location of the target memory is determined according to the relationship between the acquired memory size and the sum of the target memory size and a data header size. The implementation method may include:
[0142] S401, determining whether the acquired memory size is smaller than the sum of the target memory size and a data header size.
[0143] If the acquired memory size is smaller than the sum of the target memory size and a data header size, it indicates that no suitable memory has been acquired. In this case, an error is returned, indicating that an error has occurred in acquiring the target memory location, and the program exits.
[0144] S402: If not, subtract the size of a data header from the acquired memory size to obtain the insertable data memory size, and then determine whether the insertable data memory size is larger than the target memory size.
[0145] If the acquired memory size is not less than the sum of the target memory size and a data header size, the target memory position is the same as the data usage offset position. The acquired memory size is subtracted from the size of a data header to obtain the insertable data memory size, and then determine whether the insertable data memory size is larger than the target memory size.
[0146] S403: If the insertable data memory size is larger than the target memory size, determine whether the insertable data memory size is smaller than the sum of the target memory size and a data header size.
[0147] S404: If the insertable data memory size is not greater than the target memory size, determine the location of the target memory, and determine the acquired memory size as the actual available memory size corresponding to the location of the target memory.
[0148] S405, if the size of the insertable data memory is smaller than the sum of the target memory size and a data header size, determine whether the data header of the next data block in the data area pointed to by the position of the target memory is not the end and is not occupied.
[0149] S406, if the insertable data memory size is not less than the sum of the target memory size and a data header size, the part of the insertable data memory size that is larger than the target memory size is treated as a separate data block, and the location of the target memory is determined, and the target memory size is determined as the actual available memory size corresponding to the location of the target memory.
[0150] If the insertable data memory size is not less than the sum of the target memory size and a data header size, then the excess memory is processed as follows: the portion of the insertable data memory size that is larger than the target memory size is used as a data header of a data block.
[0151] S407, if the data header of the next data block in the data area pointed to by the target memory position is not the end and is not occupied, the part of the insertable data memory whose size is larger than the target memory size is merged into the next data block, and the target memory position is determined, and the target memory size is determined as the actual available memory size corresponding to the target memory position.
[0152] If the data header of the next data block in the data area pointed to by the target memory location is not the end and is not occupied, at this time, the processing method for the obtained excess memory is: the part of the insertable data memory whose size is larger than the target memory size is merged into the next data block.
[0153] S408, if the data header of the next data block in the data area pointed to by the target memory position is at the end and / or is occupied, the part of the insertable data memory size that is larger than the target memory size is placed in the acquired memory, and the position of the target memory is determined, and the acquired memory size is determined as the actual available memory size corresponding to the position of the target memory.
[0154] If the data header of the next data block in the data area pointed to by the target memory location is at the end and / or is occupied, at this time, the processing method for the acquired excess memory is: the part of the insertable data memory whose size is larger than the target memory size is placed in the acquired memory.
[0155] As an optional implementation of the embodiment of the present invention, Fig.10 As shown, Fig.10 A schematic diagram of an implementation method of a producer sending information to a consumer provided in an embodiment of the present invention can be used to implement the above step S105, where the producer sends information about data to be consumed to the consumer. The implementation method may include:
[0156] S501, the producer sends information about the data to be consumed to the consumer according to the consumer subscription bitmap.
[0157] The consumer subscription bitmap represents the consumer bitmap that subscribes to the producer. Therefore, after the producer completes writing the data to be consumed, it can send information about the data to be consumed to the subscribed consumer based on the consumer subscription bitmap. The information that the producer sends to the consumer about the data to be consumed can include: data header offset information, data area coverage times information, etc.
[0158] S502: Determine whether a subscribed consumer in a consumer subscription bitmap needs to be awakened.
[0159] In actual applications, some consumers are in a state of waiting for data and need to be awakened when obtaining data, while some consumers can directly obtain data from producers without being awakened. The producer can determine whether the subscribed consumers in the consumer subscription bitmap need to be awakened, and wake up the subscribed consumers if necessary. Specifically, the subscribed consumers can be awakened by sending semaphores.
[0160] S503: If yes, wake up the subscribed consumer.
[0161] S504: If not, determine whether there is any consumption data that has not been inserted.
[0162] In the case that there is still data to be consumed that has not been inserted, the operation can be returned to step S105 in which the producer inserts the data to be consumed into the shared memory by sequential insertion and sequential overwriting.
[0163] S505, when the insertion of the data to be consumed is completed, the producer is deregistered, the producer information corresponding to the producer is cleared, and the setting relationship of the producer in the consumer usage bitmap corresponding to the consumer subscribing to the producer is updated.
[0164] When the insertion of the data to be consumed is completed, the producer can be deregistered and the position relationship of the producer in the consumer usage bitmap corresponding to the consumer subscribing to the producer can be updated, where the consumer usage bitmap is a bitmap used to represent the producer information currently being used by the consumer.
[0165] S506, determining whether the consumer usage bitmap is zero.
[0166] It can be understood that each consumer can subscribe to multiple producers. Therefore, the consumer usage bitmap can contain multiple producer settings. When the producer is deregistered, the producer setting relationship in the consumer usage bitmap is also cancelled.
[0167] S507: When the consumer usage bitmap is zero, determine whether the subscribed consumer needs to be awakened.
[0168] S508: When the subscribed consumer needs to be awakened, awaken the subscribed consumer.
[0169] As an optional implementation of the embodiment of the present invention, Fig.11 As shown, Fig.11 A schematic diagram of an implementation method of a consumer performing data processing provided in an embodiment of the present invention can be used to implement the above step S106. The consumer obtains the data to be consumed from the shared memory according to the information of the data to be consumed, and processes the data to be consumed. The implementation method may include:
[0170] S601, the consumer processes the information of the data to be consumed according to the type of the data to be consumed.
[0171] In actual applications, when consumers obtain data, they can choose the type of data to be obtained. For example, for video data, they can obtain the next frame of data (if the information of the data to be consumed also contains data that has not been obtained before, the oldest unread message is obtained), or obtain the current data (at this time, all unread messages in the information of the data to be consumed are cleared, and new messages are waited for), or obtain the oldest data (obtain the oldest information in the information of the data to be consumed), or obtain data in a non-blocking manner, or obtain data in a blocking manner, and then, according to the type of data to be consumed, the information of the data to be consumed is processed.
[0172] S602, determine whether the producer in the subscription producer bitmap has been created and is in use.
[0173] The subscription producer bitmap is used to represent the bitmap of the consumer subscribing to the producer.
[0174] S603: If the producer in the subscription producer bitmap has been created and is in use, determine whether there is information about unread data to be consumed in the message loop queue of the message control information.
[0175] When creating a consumer, message control information can be generated for each consumer, and the message control information is used to manage the information of the consumer's pending data to be processed. Figure 4 As shown, the message control information may include: whether to be woken up, message writing position information, message reading position information, message loop queue information and message lock. The message loop queue may store information about the data to be consumed sent by the producer.
[0176] When the producer in the subscription producer bitmap has been created and is in use, it indicates that there is a producer that has been created among the producers that the consumer has subscribed to. At this time, it is further determined whether there is information about unread data to be consumed in the message loop queue of the message control information.
[0177] If the producer in the subscription producer bitmap is not created and in use, step S608 is executed to determine whether to continue the operation of obtaining the to-be-consumed data from the producer.
[0178] S604: When there is unread information of data to be consumed in the message loop queue, check whether the shared memory is overwritten.
[0179] S605, when there is no unread information of data to be consumed in the message loop queue, wait to be awakened.
[0180] S606: When the shared memory is not overwritten, set a consumer occupation bitmap in the data header of the data block to be consumed.
[0181] The data header of the data block to be consumed is determined according to the information of the data to be consumed. When the shared memory is overwritten, the next message in the message loop queue is read.
[0182] S607, the consumer obtains the data to be consumed from the shared memory according to the information of the data to be consumed, processes the data to be consumed, and releases the data to be consumed when the processing is completed.
[0183] The information of the data to be consumed may include: data header offset information, data area coverage times information, etc. Then, the consumer can obtain the data to be consumed from the shared memory according to the data header offset information, process the data to be consumed, and release the data to be consumed when the processing is completed. In essence, what is released is: the consumer occupancy bitmap in the data header of the data block to be consumed.
[0184] S608, determining whether to continue obtaining the data to be consumed from the producer.
[0185] After releasing the data to be consumed, it is further determined whether to continue to obtain the data to be consumed from the producer.
[0186] S609, when the consumer does not continue to obtain the data to be consumed from the producer, the consumer is deregistered, the consumer subscription bitmap in the producer is cleared, and the message loop queue in the message control information is cleared.
[0187] While continuing to obtain the data to be consumed from the producer, the process returns to step S601, where the consumer processes the information of the data to be consumed according to the type of the data to be consumed.
[0188] For example, IPC (Internet Protocol Camera) recording is used as an example: IPC has main stream, sub-stream, third stream, pictures and audio, which can be used as producers. IPC supports preview, recording, screenshot and alarm uploading pictures, etc., which can be used as consumers. Taking the main stream recording as an example:
[0189] Step 1, you can initialize the shared memory and initialize the semaphore (the size of the shared memory can be determined according to actual conditions).
[0190] Step 2, mapping the shared memory (a mapping relationship between the address of the shared memory and the virtual memory address of the target process can be established).
[0191] Step 3, create a producer, create the main stream, sub stream, third stream, picture and audio as producers. Assuming that when creating a producer, the id = 1 from the main stream, and the others are incremented in sequence, then the audio id = 5; after creation, the shared memory will initialize the corresponding producer information structure strmInfo according to the id.
[0192] Step 4, create a main stream video consumer and subscribe to related data. The consumer needs to subscribe to the main stream and audio data, so the corresponding main stream and audio id bits on the subscription producer bitmap are set (that is, 1 and 5 are set). Create a consumer, the shared memory program will return an unused consumer id, assuming id = 20. And according to the subscription producer bitmap, set the consumer subscription bitmap in the subscribed producer stmInfo, corresponding to consumer id = 20, to complete the subscription. Initialize the corresponding consumer information and message control information, namely cnsmrInfo and msgCtrl.
[0193] Step 5, the producer inserts data. The process of inserting data into shared memory is described in the above embodiment. After the data is inserted, the shared memory program will send the information msgInfo of the data to be consumed to the consumer who subscribes to the data according to the corresponding consumer subscription bitmap. The message includes the data coverage times converCnt and the data header offset offset, and updates the setting relationship of the producer in the consumer usage bitmap corresponding to the consumer who subscribes to the producer.
[0194] Step 6: Start recording.
[0195] Step 7, get the main stream and audio data. According to the consumer information cnsmrInfo obtained during registration (creation), get the corresponding message control information msgCtrl, get the information msgInfo of the data to be consumed, and get the data according to the data header offset offset in msgInfo. In the process of getting data, add 1 to the number of useCnt in the data header DataHead, and the consumer bitmap bitmap will set the consumer id = 20 bit (to avoid data being overwritten).
[0196] Step 8, record the video.
[0197] Step 9: Release the data. When releasing the data, the useCnt in the DataHead of the corresponding data will be reduced by 1, and the consumer id in the bitmap will be cleared to 20 bits.
[0198] Step 10, repeat steps 5-9 until the recording stops.
[0199] Step 11, log out of the consumer.
[0200] Step 12, deregister the producer.
[0201] Step 13, cancel the shared memory mapping.
[0202] Step 14, destroy the shared memory.
[0203] Corresponding to the above method embodiments, the embodiments of the present invention also provide corresponding device embodiments.
[0204] like Fig.12 As shown, an embodiment of the present invention provides a shared cache device, which may include:
[0205] The first creation module 701 is used to create and initialize a shared memory.
[0206] The address mapping module 702 is used to establish a mapping relationship between the address of the shared memory and the virtual memory address of the target process.
[0207] The second creation module 703 is used to create a producer according to the instruction to create a producer, and the producer is used to insert the data to be consumed into the shared memory.
[0208] The third creation module 704 is used to create a consumer according to the instruction to create a consumer, and the consumer is used to process the data to be consumed inserted into the shared memory.
[0209] The data insertion module 705 is used for the producer to insert the data to be consumed in the shared memory using the sequential insertion and sequential overwriting methods, and send the information of the data to be consumed to the consumer. The sequential insertion and sequential overwriting mean that the data area of the shared memory is repeatedly traversed from the beginning to the end to insert the data to be consumed into the shared memory using the sequential insertion and sequential overwriting methods.
[0210] The data processing module 706 is used for the consumer to obtain the data to be consumed from the shared memory according to the information of the data to be consumed, and to process the data to be consumed.
[0211] A shared cache device provided by an embodiment of the present invention maps the address of a shared memory to the virtual memory address of a target process, so that the shared memory can be used in multiple processes and multiple threads, and has a wide range of applications. The producer inserts the data to be consumed in the shared memory by sequential insertion and sequential overwriting, and can dynamically allocate memory according to the size of the inserted data to be consumed, without the need to re-partition the memory, thereby improving the utilization of the memory. The use of shared memory can reduce the number of repeated data copies, thereby increasing the data insertion speed and reducing the data processing delay. Moreover, sequential insertion and sequential overwriting can eliminate the need for synchronization when processing audio and video data, create producers and consumers, support multiple producers and multiple consumers, and make the use of shared memory more flexible.
[0212] Optionally, the shared memory includes: a shared memory header control information area and a data area, the data area includes a plurality of data blocks, and the data insertion module 705 is specifically used for:
[0213] The producer traverses the data area of the shared memory sequentially. If the available space at the end of the data area is insufficient to insert the data to be consumed, or it traverses the data area again and the available space before the occupied data block is insufficient to insert the data to be consumed, the data block is skipped and the target data block into which the data to be consumed can be inserted is sequentially obtained from the data area.
[0214] Write the data to be consumed into the target data block.
[0215] Optionally, the shared memory includes: a shared memory header control information area and a data area, the data area includes a plurality of data blocks, and the data block includes a data header and data; the data insertion module 705 includes:
[0216] Get submodule, used to get the offset position of data in shared memory.
[0217] The first determination submodule is used to traverse the data area based on the data usage offset position to determine the acquired memory size.
[0218] The judgment submodule is used to judge whether the acquired memory size is smaller than the sum of the target memory size and a data header size, and whether the data area is traversed.
[0219] The second determination submodule is used to determine the location of the target memory based on the acquired memory size and the relationship between the target memory size and the sum of a data header size when the judgment result of the judgment submodule is no, and the location of the target memory is used to indicate the address of the data block into which the data to be consumed is inserted; when the judgment result of the judgment submodule is yes, trigger the first determination submodule to execute a traversal of the data area based on the data usage offset position to determine the acquired memory size.
[0220] The data insertion submodule is used to insert the data to be consumed into the acquired memory based on the location of the target memory.
[0221] Optionally, the first determining submodule is specifically configured to:
[0222] Determine whether the data usage offset position points to the end of the data area.
[0223] When the data usage offset position points to the end of the data area, the data usage offset position is pointed to the head of the data area, the acquired memory size is cleared, the number of times the shared memory is completely covered is increased by 1, and the position of the target memory is pointed to the head of the data area.
[0224] Determine whether the data block corresponding to the head of the data area pointed to by the data usage offset position is occupied.
[0225] If yes, the acquired memory size is cleared to zero, and the location of the target memory is pointed to the data header of the next data block in the data area.
[0226] If not, the acquired memory size is updated to be the sum of the historical acquired memory size and the size of the unoccupied data block.
[0227] Use the offset position of the data to point to the next data block in the data area.
[0228] Optionally, the second determining submodule is specifically configured to:
[0229] Determine whether the acquired memory size is smaller than the sum of the target memory size and a data header size.
[0230] If yes, then return an error.
[0231] If not, subtract the size of a data header from the acquired memory size to obtain the insertable data memory size, and then determine whether the insertable data memory size is larger than the target memory size.
[0232] If the insertable data memory size is larger than the target memory size, it is determined whether the insertable data memory size is smaller than the sum of the target memory size and a data header size.
[0233] If the insertable data memory size is not greater than the target memory size, the location of the target memory is determined, and the acquired memory size is determined as the actual available memory size corresponding to the location of the target memory.
[0234] If the size of the insertable data memory is smaller than the sum of the target memory size and a data header size, it is determined whether the data header of the next data block in the data area pointed to by the position of the target memory is not the end and is not occupied.
[0235] If the insertable data memory size is not less than the sum of the target memory size and a data header size, the portion of the insertable data memory size that is larger than the target memory size is treated as a separate data block, and the location of the target memory is determined, and the target memory size is determined as the actual available memory size corresponding to the location of the target memory.
[0236] If the data header of the next data block in the data area pointed to by the target memory position is not the end and is not occupied, the part of the insertable data memory whose size is larger than the target memory size will be merged into the next data block, and the target memory position will be determined, and the target memory size will be determined as the actual available memory size corresponding to the target memory position.
[0237] If the data header of the next data block in the data area pointed to by the target memory location is at the end and / or is occupied, the part of the insertable data memory size that is larger than the target memory size is placed in the acquired memory, and the target memory location is determined, and the acquired memory size is determined as the actual available memory size corresponding to the target memory location.
[0238] Optionally, when creating a producer according to the create producer instruction, a consumer subscription bitmap is obtained, and the consumer subscription bitmap is used to represent the consumer bitmap subscribed to the producer; the above-mentioned data insertion module 705 is specifically used to:
[0239] The producer sends information about the data to be consumed to the consumer based on the consumer subscription bitmap.
[0240] Determine whether the subscribed consumer in the consumer subscription bitmap needs to be awakened.
[0241] If yes, wake up the subscribed consumer.
[0242] If not, determine whether there is any data to be consumed that has not been inserted.
[0243] If there is still data to be consumed that has not been inserted, return to execute the step of inserting the data to be consumed into the shared memory by the producer using sequential insertion and sequential overwriting.
[0244] Optionally, the data insertion module 705 is further specifically used for:
[0245] When the insertion of the data to be consumed is completed, the producer is deregistered, the producer information corresponding to the producer is cleared, and the position relationship of the producer in the consumer usage bitmap corresponding to the consumer subscribing to the producer is updated; the consumer usage bitmap is used to represent the bitmap of producer information currently being used by the consumer;
[0246] Determine whether the consumer usage bitmap is zero.
[0247] When the consumer usage bitmap is zero, determine whether the subscribed consumer needs to be woken up.
[0248] Wake up the subscribed consumer if it needs to be woken up.
[0249] Optionally, the data processing module 706 is further specifically configured to:
[0250] The consumer processes the information of the data to be consumed according to the type of the data to be consumed.
[0251] Determine whether the producer in the subscription producer bitmap has been created and is in use. The subscription producer bitmap is used to represent the bitmap of the consumer subscription producer.
[0252] If the producer in the subscription producer bitmap has been created and is in use, it is determined whether there is any unread data to be consumed in the message loop queue of the message control information. The message control information is used to manage the information of the data to be consumed by the consumer.
[0253] When there is unread data to be consumed in the message loop queue, check whether the shared memory is overwritten.
[0254] When there is no unread information about data to be consumed in the message loop queue, wait to be awakened.
[0255] In the case where the shared memory is not overwritten, a consumer occupation bitmap in the data header of the data block to be consumed is set; the data header of the data block to be consumed is determined according to the information of the data to be consumed.
[0256] The consumer obtains the data to be consumed from the shared memory according to the information of the data to be consumed, processes the data to be consumed, and releases the data to be consumed when the processing is completed.
[0257] If the producer in the subscription producer bitmap is not created and in use, or the data to be consumed is released, it is determined whether to continue to obtain the data to be consumed from the producer.
[0258] When no longer obtaining data to be consumed from the producer, the consumer is deregistered, the consumer subscription bitmap in the producer is cleared, and the message loop queue in the message control information is cleared.
[0259] While continuing to obtain the data to be consumed from the producer, the step of returning to execute the consumer to process the information of the data to be consumed according to the type of the data to be consumed.
[0260] The embodiment of the present invention further provides an electronic device, such as Fig.13As shown, it includes a processor 801, a communication interface 802, a memory 803 and a communication bus 804, wherein the processor 801, the communication interface 802, and the memory 803 communicate with each other through the communication bus 804.
[0261] Memory 803, used for storing computer programs;
[0262] The processor 801 is used to implement the steps of any of the above-mentioned shared cache methods when executing the program stored in the memory 803 to achieve the same technical effect.
[0263] The communication bus mentioned in the above electronic device may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.
[0264] The communication interface is used for communication between the above electronic device and other devices.
[0265] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0266] The memory may include RAM (Random Access Memory) or NVM (Non-Volatile Memory), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.
[0267] The above-mentioned processor can be a general-purpose processor, including a CPU (Central Processing Unit), an NP (Network Processor), etc.; it can also be a DSP (Digital Signal Processing), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
[0268] In another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the above-mentioned shared cache methods are implemented.
[0269] In another embodiment of the present invention, a computer program product including instructions is provided. When the computer program product is executed on a computer, the computer executes the steps of any one of the methods for sharing cache in the above embodiments.
[0270] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, DSL (Digital Subscriber Line, digital subscriber line)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD (Digital Versatile Disc)), or a semiconductor medium (eg, an SSD (Solid State Disk)), etc.
[0271] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0272] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device / electronic device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
[0273] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.
Claims
1. A method for sharing cache, characterized in that: The method comprises: Create and initialize shared memory; Establishing a mapping relationship between the address of the shared memory and the virtual memory address of the target process; Creating a producer according to a create producer instruction, wherein the producer is used to insert data to be consumed into the shared memory; Create a consumer according to a create consumer instruction, wherein the consumer is used to process the data to be consumed inserted into the shared memory; The producer inserts the data to be consumed into the shared memory by sequential insertion and sequential overwriting, and sends information about the data to be consumed to the consumer, wherein the sequential insertion and sequential overwriting represent repeatedly traversing the data area of the shared memory from beginning to end, so as to insert the data to be consumed into the shared memory by sequential insertion and sequential overwriting; The consumer obtains the data to be consumed from the shared memory according to the information of the data to be consumed, and processes the data to be consumed; The shared memory includes: a shared memory header control information area and a data area, the data area includes a plurality of data blocks, and the producer inserts the data to be consumed into the shared memory in a sequential insertion and sequential overwriting manner, including: The producer sequentially traverses the data area of the shared memory, and when the available space at the end of the data area is insufficient to insert the data to be consumed, or retraverses the data area, and when the available space before the occupied data block is insufficient to insert the data to be consumed, the data block is skipped, and target data blocks capable of inserting the data to be consumed are sequentially obtained from the data area; The data to be consumed is written into the target data block.
2. The method according to claim 1, characterized in that The data block of the shared memory data area includes a data header and data; the step of the producer inserting the data to be consumed into the shared memory in a sequential insertion and sequential overwriting manner includes: Obtaining the data usage offset position in the shared memory; Traversing the data area based on the data using the offset position to determine the acquired memory size; Determine whether the acquired memory size is smaller than the sum of the target memory size and a data header size, and whether the data area has been traversed; If yes, return to the step of traversing the data area based on the data usage offset position to determine the acquired memory size; If not, determining the location of the target memory according to the acquired memory size and the relationship between the target memory size and the sum of a data header size, wherein the location of the target memory is used to indicate the address of the data block into which the data to be consumed is inserted; Based on the location of the target memory, the to-be-consumed data is inserted into the acquired memory.
3. The method according to claim 2, characterized in that The step of traversing the data area based on the data usage offset position to determine the acquired memory size includes: Determining whether the data usage offset position points to the end of the data area; When the data usage offset position points to the tail end of the data area, the data usage offset position is pointed to the head of the data area, the acquired memory size is cleared, the number of times the shared memory is completely covered is increased by 1, and the position of the target memory is pointed to the head of the data area; Determine whether the data block corresponding to the head of the data area pointed to by the data usage offset position is occupied; If yes, clear the acquired memory size to zero, and point the location of the target memory to the data header of the next data block in the data area; If not, the acquired memory size is updated to be the sum of the historical acquired memory size and the size of the unoccupied data block; The data is pointed to the next data block in the data area using an offset position.
4. The method according to claim 2, characterized in that: The step of determining the location of the target memory according to the acquired memory size and the relationship between the target memory size and the sum of a data header size comprises: Determine whether the acquired memory size is smaller than the sum of the target memory size and a data header size; If yes, return an error; If not, subtract the size of a data header from the acquired memory size to obtain the insertable data memory size, and then determine whether the insertable data memory size is larger than the target memory size; If the insertable data memory size is larger than the target memory size, determining whether the insertable data memory size is smaller than the sum of the target memory size and a data header size; If the insertable data memory size is not greater than the target memory size, determining the location of the target memory, and determining the acquired memory size as the actual available memory size corresponding to the location of the target memory; If the size of the insertable data memory is smaller than the sum of the target memory size and a data header size, determining whether the data header of the next data block of the data area pointed to by the position of the target memory is not the end and is not occupied; If the insertable data memory size is not less than the sum of the target memory size and a data header size, the portion of the insertable data memory size that is larger than the target memory size is treated as a separate data block, and the location of the target memory is determined, and the target memory size is determined as the actual available memory size corresponding to the location of the target memory; If the data header of the next data block of the data area pointed to by the position of the target memory is not the end and is not occupied, then the part of the insertable data memory whose size is larger than the target memory size is merged into the next data block, and the position of the target memory is determined, and the target memory size is determined as the actual available memory size corresponding to the position of the target memory; If the data header of the next data block in the data area pointed to by the position of the target memory is the end and / or is occupied, the part of the insertable data memory size that is larger than the target memory size is placed in the acquired memory, and the position of the target memory is determined, and the acquired memory size is determined as the actual available memory size corresponding to the position of the target memory.
5. The method according to claim 1, characterized in that When creating a producer according to the create producer instruction, a consumer subscription bitmap is obtained, where the consumer subscription bitmap is used to represent a consumer bitmap subscribed to the producer; The step of the producer sending information about the data to be consumed to the consumer comprises: The producer sends information about the data to be consumed to the consumer according to the consumer subscription bitmap; Determine whether the subscribed consumer in the consumer subscription bitmap needs to be awakened; If yes, wake up the subscribed consumer; If not, determine whether there is any consumption data that has not been inserted; In the case that there is still data to be consumed that has not been inserted, the process returns to the step of inserting the data to be consumed into the shared memory by the producer in a sequential insertion and sequential overwriting manner.
6. The method according to claim 5, characterized in that The method further comprises: When the insertion of the data to be consumed is completed, the producer is deregistered, the producer information corresponding to the producer is cleared, and the position relationship of the producer in the consumer usage bitmap corresponding to the consumer subscribing to the producer is updated; the consumer usage bitmap is used to represent the bitmap of the producer information currently being used by the consumer; Determine whether the consumer usage bitmap is zero; When the consumer usage bitmap is zero, determining whether the subscribed consumer needs to be awakened; When the subscribed consumer needs to be awakened, the subscribed consumer is awakened.
7. The method according to claim 1, characterized in that The step of the consumer obtaining the data to be consumed from the shared memory according to the information of the data to be consumed and processing the data to be consumed includes: The consumer processes information of the data to be consumed according to the type of the data to be consumed; Determine whether the producer in the subscription producer bitmap has been created and is in use, wherein the subscription producer bitmap is used to represent the bitmap of the consumer subscription producer; If the producer in the subscription producer bitmap has been created and is in use, determine whether there is information of unread data to be consumed in the message loop queue of the message control information, and the message control information is used to manage the information of the data to be consumed by the consumer; When there is information of unread data to be consumed in the message loop queue, checking whether the shared memory is overwritten; When there is no information of unread data to be consumed in the message loop queue, waiting to be awakened; In the case where the shared memory is not overwritten, a consumer occupation bitmap in a data header of a data block to be consumed is set; the data header of the data block to be consumed is determined according to information of the data to be consumed; The consumer obtains the data to be consumed from the shared memory according to the information of the data to be consumed, processes the data to be consumed, and releases the data to be consumed when the processing is completed; If the producer in the subscription producer bitmap is not created and in use, or the data to be consumed is released, then determine whether to continue to obtain the data to be consumed from the producer; When no longer obtaining data to be consumed from the producer, the consumer is deregistered, the consumer subscription bitmap in the producer is cleared, and the message loop queue in the message control information is cleared; While continuing to obtain the data to be consumed from the producer, the step of returning to execute the consumer processing the information of the data to be consumed according to the type of the data to be consumed obtained.
8. A shared cache device, characterized in that: The device comprises: The first creation module is used to create and initialize the shared memory; An address mapping module, used to establish a mapping relationship between the address of the shared memory and the virtual memory address of the target process; A second creation module, used for creating a producer according to a producer creation instruction, wherein the producer is used for inserting data to be consumed into the shared memory; A third creation module, used to create a consumer according to a create consumer instruction, wherein the consumer is used to process the to-be-consumed data inserted into the shared memory; A data insertion module, used for the producer to insert the data to be consumed into the shared memory by sequential insertion and sequential overwriting, and to send information of the data to be consumed to the consumer, wherein the sequential insertion and sequential overwriting represent repeatedly traversing the data area of the shared memory from beginning to end, so as to insert the data to be consumed into the shared memory by sequential insertion and sequential overwriting; A data processing module, used for the consumer to obtain the data to be consumed from the shared memory according to the information of the data to be consumed, and to process the data to be consumed; The shared memory includes: a shared memory header control information area and a data area, the data area includes a plurality of data blocks, and the data insertion module is specifically used for: The producer sequentially traverses the data area of the shared memory, and when the available space at the end of the data area is insufficient to insert the data to be consumed, or retraverses the data area, and when the available space before the occupied data block is insufficient to insert the data to be consumed, the data block is skipped, and target data blocks capable of inserting the data to be consumed are sequentially obtained from the data area; The data to be consumed is written into the target data block.
9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 7 when executing a program stored in a memory.
Citation Information
Patent Citations
Data communication method and device, computing equipment and storage medium
CN111651286A
Management of ownership control and data movement in shared-memory systems
US20100131720A1
