Methods, electronic devices, and storage media for reading variable-length data from a circular buffer

Abstract

This invention relates to the field of chip design technology, and in particular to a method, electronic device, and storage medium for reading variable-length data from a circular buffer. The method divides the circular buffer into multiple consecutive fixed-length sub-regions and one variable-length sub-region based on the total capacity of the circular buffer and the maximum allowed data length for each read. Each time data is read from the circular buffer, the target data length of the valid data in the sub-region where the read pointer is located is determined comprehensively based on the relationship between the read and write pointers, whether they are in the same sub-region, and whether the sub-region where the read pointer is located is a variable-length sub-region. Data of the target data length, starting from the position pointed to by the read pointer, is read from the sub-region where the read pointer is located, and the read pointer is updated according to the target data length. This solves the problems of latency and consumption of additional hardware resources.

Description

Technical Field

[0001] This invention relates to the field of chip design technology, and in particular to a method, electronic device, and storage medium for reading variable-length data from a circular buffer. Background Technology

[0002] In a hardware-accelerated scenario: the software driver writes data packets to a ring buffer, and the hardware reads data from the ring buffer. When reading data from the ring buffer, the hardware reads one pre-defined fixed-length data unit at a time, such as the length of one cash line. However, the current position of the data being written and the current position of the data being read in the ring buffer determine the length of the readable data. Generally, a write pointer indicates the starting position that can be written, and a read pointer indicates the starting position that can be retrieved. However, if the currently retrieved data is less than one pre-defined fixed-length data unit, there are currently two handling methods: First, the system needs to wait for the written data to reach one pre-defined fixed-length data unit before reading, which causes latency, and if the data stream is interrupted, the condition may never be met. Second, the hardware reads the available data and fills the remaining part with 0 or a specific value, combining the two to form one pre-defined fixed-length data unit, but this requires hardware processing of the filler data, consuming additional hardware resources.

[0003] If the total capacity of the circular buffer can be dynamically adjusted, and the currently retrieved data is less than a preset fixed length, using the first method described above may result in the condition that the preset fixed length of data can never be met due to the adjustable total capacity, rendering the first method unsuitable. Using the second method may also result in the inability to determine the amount of data to be filled due to the adjustable total capacity, rendering the second method unsuitable. Summary of the Invention

[0004] To address the aforementioned technical problems, the present invention employs the following technical solution: a method for reading variable-length data from a circular buffer, the method comprising the following steps: S100, obtain the total capacity of the circular buffer and the maximum data length allowed to be read each time; wherein, the total capacity supports dynamic adjustment.

[0005] S200, based on the total capacity and the maximum data length, the circular buffer is divided into multiple consecutive fixed-length sub-regions and one variable-length sub-region; wherein the length of the fixed-length sub-region is equal to the maximum data length; and the length of the variable-length sub-region is not greater than the maximum data length.

[0006] S300, each time data is read from the circular buffer, the read pointer and write pointer of the circular buffer are obtained, and the target data length of the effective data in the sub-region where the read pointer is located is comprehensively determined based on the size relationship between the read pointer and the write pointer, whether they are in the same sub-region, and whether the sub-region where the read pointer is located is a variable-length sub-region.

[0007] S400, read data from the sub-region where the read pointer is located, starting from the position pointed to by the read pointer and having a length equal to the target data length, and update the read pointer according to the target data length.

[0008] Furthermore, the present invention also provides a non-transitory computer-readable storage medium storing at least one instruction or at least one program segment, wherein the at least one instruction or the at least one program segment is loaded and executed by a processor to implement the above-described method.

[0009] In addition, the present invention provides an electronic device including a processor and the aforementioned non-transitory computer-readable storage medium.

[0010] The present invention has at least the following beneficial effects: In scenarios where the total capacity of the circular buffer can be dynamically adjusted and the currently retrieved data is less than a preset fixed length, this invention provides a method, electronic device, and storage medium for reading variable-length data from the circular buffer. By dividing the circular buffer into fixed-length and variable-length sub-regions and combining the relative positions of the read and write pointers, the target data length of the effective data in the sub-region where the read pointer is located is calculated. This eliminates the need to wait for the data to be written to the maximum allowed read length each time, and also eliminates the need to fill in preset special characters. Instead, it directly obtains the target data length and retrieves the effective data directly through the read pointer in conjunction with this target data length, thus solving the problems of latency and the consumption of additional hardware resources. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 A flowchart illustrating a method for reading variable-length data from a circular buffer, provided in an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating various types of relative positional relationships between the read / write pointer and each sub-region in the circular buffer, as provided in an embodiment of the present invention. Detailed Implementation

[0013] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0014] Unless otherwise defined, all technical and scientific terms used in the embodiments of this invention have the same meaning as commonly understood by those skilled in the art.

[0015] Please see Figure 1 It illustrates a method for reading variable-length data from a circular buffer, the method comprising the following steps: S100, obtain the total capacity of the circular buffer and the maximum data length allowed to be read each time; wherein, the total capacity supports dynamic adjustment.

[0016] A circular buffer is used to represent a data structure of a buffer that is contiguous at both ends. A circular buffer is configured with a read pointer and a write pointer. The read pointer points to the next readable location, and the write pointer points to the next writable location. Logically, a circular buffer is circular. Taking the read pointer as an example, when the read pointer reaches the end of the circular buffer, it restarts from the beginning, forming a loop. Similarly, when the write pointer reaches the end, it also restarts from the beginning, forming a loop. It should be noted that the read and write pointers are stored in preset memory locations accessible by both software and hardware.

[0017] The total capacity of the circular buffer refers to the amount of data that the entire circular buffer can store. The total capacity of the circular buffer can be dynamically adjusted.

[0018] The maximum data length allowed to be read each time refers to the upper limit of the data that can be read in one atomic operation.

[0019] In one implementation, the maximum data length is equal to the data length of one cache line.

[0020] S200, based on the total capacity and the maximum data length, the circular buffer is divided into multiple consecutive fixed-length sub-regions and one variable-length sub-region; wherein the length of the fixed-length sub-region is equal to the maximum data length; and the length of the variable-length sub-region is not greater than the maximum data length.

[0021] In one implementation, the total number of sub-regions, Num_sub, satisfies: Num_sub = roundup(Size_ring / rlen_max), where roundup() is the round-up function, Size_ring is the total capacity of the circular buffer, and rlen_max is the maximum data length allowed to be read each time. Specifically, the length of variable-length sub-regions is equal to Size_ring - rlen_max × (Num_sub - 1), and the length of fixed-length sub-regions is equal to rlen_max. See also... Figure 2 The circular buffer is divided into a total of Num_sub sub-regions.

[0022] In one implementation, the variable-length sub-region is the last sub-region located at the tail of the annular buffer zone.

[0023] Dividing the circular buffer according to the maximum allowed data length simplifies boundary handling. For example, when the read and write pointers span multiple sub-regions, the entire sub-regions in between can be read as a whole without complex processing logic. The sub-regions where the read and write pointers are located also have their own defined head and tail boundaries, so the boundary handling logic is simpler than not dividing the buffer at all.

[0024] S300, each time data is read from the circular buffer, the read pointer and write pointer of the circular buffer are obtained, and the target data length of the effective data in the sub-region where the read pointer is located is comprehensively determined based on the size relationship between the read pointer and the write pointer, whether they are in the same sub-region, and whether the sub-region where the read pointer is located is a variable-length sub-region.

[0025] It should be noted that since the read pointer is updated after each data retrieval, the size relationship between the updated read pointer and the write pointer, whether they are in the same sub-region, and whether they are in a variable-length sub-region all depend on the length of the data read before the update. Therefore, each time data is read from the circular buffer, it is necessary to re-determine the relationship between the read and write pointers and their sub-regions, and then redetermine the target data length of the valid data in the sub-region where the read pointer is located.

[0026] The amount of valid data that can be read in the circular buffer is determined by the offset between the write pointer and the read pointer, which is the positive distance between the position of the write pointer and the position of the read pointer.

[0027] When the read pointer and write pointer are equal in size, it indicates that the circular buffer is empty, no valid data has been written, or all valid data written has been read. When the write pointer is greater than the read pointer, it indicates that there is valid readable data in the buffer. When the read pointer is greater than the write pointer, it indicates that the write operation corresponding to the write pointer has circled the tail of the circular buffer once, while the read pointer has not yet reached the tail of the circular buffer. The globally readable valid data between the read pointer and the write pointer spans the tail of the circular buffer. The length of the globally readable valid data between the read pointer and the write pointer consists of two consecutive segments: the first segment is the distance from the position pointed to by the read pointer to the tail of the circular buffer, and the second segment is the distance from the head of the circular buffer to the position pointed to by the write pointer.

[0028] Please refer to the following again. Figure 2 There are two main categories of situations where the write pointer and the read pointer are located in the same subregion: The first category is when the write pointer is located at a position greater than the read pointer, within the same subregion. For example, Figure 2 The first and second cases are mentioned. The second type occurs when the read pointer is at a position greater than the write pointer, and the regions are within the same sub-region. For example, Figure 2 The fourth and fifth cases are as follows. In the first case, the length of globally readable valid data is less than the length of the sub-region, meaning the length of valid data is equal to the distance between the write pointer and the read pointer. In the second case, the length of globally readable valid data spans the tail of the circular buffer, meaning it is equal to the sum of the distance between the read pointer and the tail of the circular buffer and the distance between the head of the circular buffer and the write pointer; or in other words, the remaining length of valid data that can be written is the distance between the read pointer and the write pointer.

[0029] Specifically, when the subregion where the read pointer is located is a variable-length subregion, since the size of the variable-length subregion is different from that of the fixed-length subregion, the length of the globally readable valid data is equal to the distance between the tail of the variable-length subregion and the read pointer when the read pointer and write pointer are in different subregions and the read pointer is in a variable-length subregion.

[0030] The target data length is not equal to the length of the globally readable valid data in the circular buffer. The length of the globally readable valid data is the sum of the lengths of all readable valid data in the circular buffer. The target data length, however, is equal to the length of the valid data in the sub-region where the read pointer is located. It should be noted that since the maximum data length retrieved each time is the data in a complete sub-region, when retrieving an entire sub-region, if the read pointer is at the beginning of the current sub-region and the write pointer is in another sub-region, the length of the valid data is the length of the current sub-region and does not need to be calculated. However, when retrieving an entire sub-region, if the read pointer is not at the beginning of the current sub-region, the length of the valid data is less than the length of the current sub-region.

[0031] In one implementation, step S300, which comprehensively determines the length of the target data to be read, includes: S310, when the write pointer is greater than the read pointer, and the write pointer and the read pointer do not belong to the same sub-region, the target data length is the distance between the tail of the sub-region where the read pointer is located and the read pointer itself. Figure 2 The third case, Figure 2 In the context of case 3, `len` refers to the length of the target data within the first sub-region. It's important to note that a write pointer greater than a read pointer indicates that globally readable valid data has not yet crossed the tail of the circular buffer.

[0032] In one implementation, step S300, which comprehensively determines the length of the target data to be read, further includes: S320, where, when the write pointer is greater than the read pointer, and the write pointer and the read pointer are within the same sub-region, the target data length is the distance between the write pointer and the read pointer. Figure 2 The first and second cases are related to this. It's important to note that if the write pointer is greater than the read pointer, it means that the globally readable valid data has not crossed the tail of the circular buffer. When the read and write pointers are within the same sub-region, it means that the globally readable valid data is less than the length of the current sub-region. In this case, there are two possibilities: First, the write and read pointers are in a variable-length sub-region. Second, the write and read pointers are in a fixed-length sub-region. The difference between these two cases lies in the length of the variable-length and fixed-length sub-regions. However, in both cases, the method for calculating the target data length is the same: the distance between the write pointer and the read pointer.

[0033] In one implementation, step S300, which comprehensively determines the length of the target data to be read, further includes: S330, when the write pointer is less than the read pointer, the target data length is the distance between the tail of the sub-region where the read pointer is located and the read pointer. It should be noted that a write pointer less than the read pointer indicates that the write pointer has already circled the tail of the circular buffer once, while the read pointer has not yet reached the tail. Specifically, when the sub-region is a fixed-length sub-region, the target data length is calculated using the distance between the tail of the fixed-length sub-region and the read pointer; when the sub-region is a variable-length sub-region, the target data length is calculated using the distance between the tail of the variable-length sub-region and the read pointer.

[0034] It should be noted that, as Figure 2 Cases 4-7 in the above. When the write pointer is less than the read pointer, and the indefinite-length sub-region is the last sub-region at the end of the circular buffer, there are four cases: such as... Figure 2 Case 4: The read pointer and write pointer reside in the same fixed-length sub-region. In case 4, the length of globally readable valid data includes: the target data length *len* of the current fixed-length sub-region, the length of valid data from the next sub-region to the tail of the circular buffer, and the length of valid data from the head of the circular buffer to the write pointer. Case 5: The read pointer and write pointer reside in the same variable-length sub-region. In case 5, the length of globally readable valid data includes: the target data length *len* of the variable-length sub-region, and the length of valid data from the head of the circular buffer to the write pointer. Case 6: The read pointer and write pointer reside in different fixed-length sub-regions. In case 6, the length of globally readable valid data includes: the target data length *len* of the current fixed-length sub-region, the length of valid data from the next sub-region to the tail of the circular buffer, and the length of valid data from the head of the circular buffer to the write pointer. Case 7: The read pointer and write pointer reside in different sub-regions, and the read pointer resides in a variable-length sub-region. In the seventh case, the length of the globally readable valid data includes: the target data length *len* within the variable-length sub-region, and the length of the valid data from the head of the circular buffer to the write pointer. The composition of the globally readable valid data length differs in these four cases, but the calculation method for the target data length is the same: the distance between the tail of the sub-region where the read pointer is located and the read pointer itself.

[0035] S400, read data from the sub-region where the read pointer is located, starting from the position pointed to by the read pointer and having a length equal to the target data length, and update the read pointer according to the target data length.

[0036] In one implementation, updating the read pointer includes: shifting the read pointer forward according to the target data length to point to the next readable position. It should be noted that the relative positional relationship between the read and write pointers and their positional relationship with sub-regions changes after the update; therefore, it is necessary to determine and calculate the corresponding target data length again according to S300.

[0037] In one implementation, S300 further includes: S340, when initiating multiple read requests, assigning a unique sequence identifier to each read request and generating a corresponding target data length; the sequence identifier and the target data length together constitute the user information of the read request, and are sent to the lower-level storage or response unit along with the request; when read response data is returned out of order, each response carries the user information of its original read request, and the response data and its user information are temporarily stored in the local storage unit; the system maintains a desired output sequence identifier, the initial value of which is the sequence identifier of the first read request; when the read response data corresponding to the desired output sequence identifier is ready, valid data is retrieved according to the read pointer and its target data length; the desired output sequence identifier is incremented, the read pointer is updated, and the next read request is processed, ensuring that the read response data is delivered in the order in which the requests were initiated. It should be noted that when obtaining valid data, the read pointer is obtained by accessing a preset memory location of the read pointer, and the updated read pointer is placed in the preset memory location to overwrite the previous read pointer.

[0038] It should be noted that when the write pointer is greater than the read pointer and both are in the same sub-region, only one read request can be initiated. When the write pointer is greater than the read pointer and both are in different regions, or when the read pointer is greater than the write pointer, the prefetch unit can initiate multiple read requests. Taking the example of the write pointer being greater than the read pointer and both being in different sub-regions, when the write pointer is in the 5th sub-region and the read pointer is in the 2nd sub-region, the globally readable valid data includes the target data in the 2nd sub-region, the 3rd sub-region, the 4th sub-region, and the 5th sub-region. The target data length of the valid data in the two sub-regions can be obtained through S300. The length of the valid data in the 3rd and 4th sub-regions is the fixed length of the region, and the length of the valid data in the 5th region is the distance between the beginning and end of the 5th region and the write pointer.

[0039] In one implementation, the local storage unit may be on-chip RAM or a register set (flop).

[0040] In summary, in scenarios where the total capacity of the circular buffer can be dynamically adjusted and the currently retrieved data is less than a preset fixed length, this invention provides a method for reading variable-length data from the circular buffer. This method divides the circular buffer into fixed-length and variable-length sub-regions and calculates the target data length of the effective data in the sub-region where the read pointer is located by combining the relative positions of the read and write pointers. It does not require waiting for the data to be written to the maximum allowed read length each time, nor does it require filling in preset special characters. Instead, it directly obtains the target data length and retrieves the effective data directly through the read pointer in conjunction with this target data length, thus solving the problems of latency and additional hardware resource consumption.

[0041] Embodiments of the present invention also provide a non-transitory computer-readable storage medium that can be disposed in an electronic device to store at least one instruction or at least one program related to implementing a method in the method embodiments, wherein the at least one instruction or the at least one program is loaded and executed by the processor to implement the method provided in the above embodiments.

[0042] Embodiments of the present invention also provide an electronic device, including a processor and the aforementioned non-transitory computer-readable storage medium.

[0043] Embodiments of the present invention also provide a computer program product including program code, which, when the program product is run on an electronic device, causes the electronic device to perform the steps of the methods described above in various exemplary embodiments of the present invention.

[0044] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0045] While specific embodiments of the invention have been described in detail by way of example, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of the invention. Those skilled in the art should also understand that various modifications can be made to the embodiments without departing from the scope and spirit of the invention. The scope of this invention is defined by the appended claims.

Claims

1. A method for reading variable-length data from a circular buffer, characterized in that, The method includes the following steps: S100, obtain the total capacity of the circular buffer and the maximum data length allowed to be read each time; wherein, the total capacity supports dynamic adjustment; S200, based on the total capacity and the maximum data length, the circular buffer is divided into multiple consecutive fixed-length sub-regions and one variable-length sub-region; wherein the length of the fixed-length sub-region is equal to the maximum data length; and the length of the variable-length sub-region is not greater than the maximum data length. S300, each time data is read from the circular buffer, the read pointer and write pointer of the circular buffer are obtained, and the target data length of the effective data in the sub-region where the read pointer is located is comprehensively determined based on the size relationship between the read pointer and the write pointer, whether they are in the same sub-region, and whether the sub-region where the read pointer is located is a variable-length sub-region. S400, read data from the sub-region where the read pointer is located, starting from the position pointed to by the read pointer and having a length equal to the target data length, and update the read pointer according to the target data length; In S300, the steps for comprehensively determining the length of the target data to be read include: S310, when the write pointer is greater than the read pointer, and the write pointer and the read pointer do not belong to the same sub-region, the target data length is the distance between the tail of the sub-region where the read pointer is located and the read pointer; S320, when the write pointer is greater than the read pointer, and the write pointer and the read pointer are in the same sub-region, the target data length is the distance between the write pointer and the read pointer; or, S330, when the write pointer is less than the read pointer, the target data length is the distance between the tail of the sub-region where the read pointer is located and the read pointer.

2. The method according to claim 1, characterized in that, The S300 also includes: S340, when initiating multiple read requests, a unique sequence identifier is assigned to each read request, and a corresponding target data length is generated; the sequence identifier and the target data length together constitute the user information of the read request, and are sent to the lower-level storage or response unit along with the request; When read response data is returned out of order, each response carries the user information of its original read request. The response data and user information are temporarily stored in the local storage unit. The system maintains an expected output order identifier, and the initial value of the expected output order identifier is the order identifier of the first read request. When the read response data corresponding to the expected output sequence identifier is ready, the valid data is retrieved according to the read pointer and the target data length it carries; the expected output sequence identifier is incremented, the read pointer is updated, and the next read request is processed to ensure that the read response data is delivered in an orderly manner according to the order in which the request was initiated.

3. The method according to claim 1, characterized in that, In S100, the maximum data length is equal to the data length of one cache line.

4. The method according to claim 1, characterized in that, In S200, the variable-length sub-region is the last sub-region located at the tail of the annular buffer zone.

5. The method according to claim 1, characterized in that, In S400, the step of updating the read pointer includes: the read pointer is shifted forward according to the target data length to point to the next readable position.

6. A non-transitory computer-readable storage medium, wherein the storage medium stores at least one instruction or at least one program segment, characterized in that, The at least one instruction or the at least one program segment is loaded and executed by the processor to implement the method as described in any one of claims 1-5.

7. An electronic device, characterized in that, Includes a processor and the non-transitory computer-readable storage medium as described in claim 6.

Images