Audio and video data frame buffer management method and system

Abstract

The invention provides an audio and video data frame buffer management method and system. According to the method, the set buffer upper limit is not broken through on the premise of ensuring the streaming media playing quality, and the memory overhead can be effectively reduced. In addition, the method does not need a link to specially download the I frame, but quickly deals with the seek operation by establishing an I frame queue and a common frame queue which are associated with each other in the data frame segment of the buffer area.

Description

Technical Field

[0001] The embodiment of the present invention relates to a streaming media playback technology, and in particular to an audio and video data frame buffer management technology of a streaming media player. Background Art

[0002] As streaming media technology becomes more and more popular, many different types of terminal devices working in different environments are required to support streaming media playback. For example, in-vehicle terminal devices have strict requirements on performance, security, and resource consumption.

[0003] The buffer management of the streaming media player plays a vital role in the playback experience, and on the other hand, it also affects the player's memory consumption.

[0004] In practice, some streaming media players use multi-threaded collaboration to implement buffer management. One thread monitors the current system available resource information in real time, and another thread manages the buffer and adjusts the buffer size in real time based on the data provided by the system monitoring thread. Although this multi-threaded synchronous buffer management method avoids system memory shortage caused by streaming media playback, it cannot control the peak value of the player buffer data storage below a certain upper limit.

[0005] In addition, in order to cope with the seek operation, some streaming media players, in addition to downloading the streaming media segment files to be played, also create separate instructions and links to download the I frames of the streaming media to be played from the server. This method of downloading and buffering I frame data and ordinary streaming media data separately increases the complexity of the system. Summary of the invention

[0006] This Summary is provided to introduce some concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0007] According to one aspect of the present invention, a method for adding a data frame to a buffer is provided, comprising: in response to receiving a new data frame, determining whether the amount of data stored in the buffer plus the amount of data of the new data frame exceeds a soft limit defined for the buffer, wherein the soft limit is dynamically determined during operation; in response to determining that the amount of data stored in the buffer plus the amount of data of the new data frame does not exceed the soft limit, adding the new data frame to the end of an active data frame segment in the buffer if the buffer is not in a search state; or in response to determining that the amount of data stored in the buffer plus the amount of data of the new data frame does not exceed the soft limit, adding the new data frame to the end of a search data frame segment in the buffer if the buffer is in a search state, wherein the search data frame segment was previously established when the buffer entered the search state.

[0008] According to another aspect of the present invention, a method for reading data frames from a buffer is provided, comprising: determining whether the buffer is in a search state; in response to determining that the buffer is not in the search state, returning a first unread data frame in an active data frame segment in the buffer; determining whether the amount of stored data in the buffer exceeds a soft limit defined for the buffer, wherein the soft limit is dynamically determined during operation; and in response to determining that the amount of stored data in the buffer exceeds the soft limit, deleting the first unread data frame from the active data frame segment.

[0009] According to another aspect of the present invention, a method for responding to a search request is provided, comprising: in response to the search request, determining whether an active data frame segment in a buffer includes a search target frame indicated by the search request; in response to determining that the active data frame segment does not include the search target frame, responding to the search request based on whether an inactive data frame segment in the buffer includes the search target frame; and in response to determining that the active data frame segment includes the search target frame, responding to the search request by comparing the search target frame with a current data frame in the active data frame segment.

[0010] According to another aspect of the present invention, a system for audio and video data frame buffer management is provided, comprising: a download module, which is configured to download or stop downloading audio data frames or video data frames from a server; and a buffer data access control module, which is configured to execute the method described above.

[0011] According to yet another aspect of the present invention, a vehicle is provided, comprising: the system for managing audio and video data frame buffer as described above; and an audio and / or video playback device.

[0012] These and other features and advantages will become apparent by reading the following detailed description and by reference to the associated drawings.It is to be understood that the foregoing general description and the following detailed description are illustrative only and are not restrictive of the aspects of what is claimed. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to understand the manner in which the above features of the present invention are used in detail, the above briefly summarized contents can be described in more detail with reference to various embodiments, some of which are shown in the accompanying drawings. However, it should be noted that the accompanying drawings only show some typical aspects of the present invention and should not be considered to limit its scope, because the description may allow for other equally effective aspects.

[0014] Figure 1 An example structure of an audio and video data frame buffer according to an embodiment of the present invention is shown;

[0015] Figure 2 shows a diagram of an audio data frame segment 201 according to an embodiment of the present invention;

[0016] Figure 3 A diagram showing a video data frame segment 301 according to one embodiment of the present invention;

[0017] Figure 4-1 to Figure 4-4 A flow chart of a method for adding a data frame to a buffer according to an embodiment of the present invention is shown;

[0018] Figure 5-1 to Figure 5-2 A flow chart of a method for reading a data frame from a buffer according to an embodiment of the present invention is shown;

[0019] Figure 6-1 to Figure 6-3 A flowchart of a method for responding to a search request according to an embodiment of the present invention is described;

[0020] Figure 7 A block diagram of a system 700 for managing audio and video data frame buffers according to various embodiments of the present invention is shown; and

[0021] Figure 8 A block diagram of an exemplary computing device according to one embodiment of the present invention is shown. DETAILED DESCRIPTION

[0022] The present invention will be described in detail below in conjunction with the accompanying drawings, and the features of the present invention will be further revealed in the following specific description.

[0023] The following detailed description refers to the accompanying drawings showing exemplary embodiments of the present invention. However, the scope of the present invention is not limited to these embodiments, but is defined by the appended claims. Therefore, embodiments other than those shown in the drawings, such as modified versions of the illustrated embodiments, are still encompassed by the present invention.

[0024] References in this specification to "one embodiment," "an embodiment," "an example embodiment," etc., mean that the embodiment may include a particular feature, structure, or characteristic, but not every embodiment necessarily includes the particular feature, structure, or characteristic. Furthermore, these phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it should be understood that the particular feature, structure, or characteristic can be implemented in conjunction with other embodiments within the knowledge of a person skilled in the relevant art, whether or not explicitly described.

[0025] Unless otherwise stated, the term "A or B" used throughout this specification refers to "A and B" and "A or B" but does not mean that A and B are exclusive.

[0026] The present invention provides a buffer management technology for audio and video data frames of a streaming media player, which can effectively reduce memory overhead without exceeding a set buffer upper limit while ensuring streaming media playback quality. In addition, the present invention does not require a link to download I frames specifically, but instead quickly responds to seek operations by establishing an interrelated I frame queue and a common frame queue in a data frame segment in a buffer.

[0027] Related terms:

[0028] Seek: In this article, it refers to the operation of jumping to the audio or video being played (for example, moving forward or backward to a certain time point of the audio or video) when the user is listening to or watching the audio or video.

[0029] Key frame (I frame): Frame is the basic unit of video images, and video files are composed of multiple continuous frames. MPEG encoding divides pictures (i.e. frames) into three types: I, P, and B. I is an internal coding frame, P is a forward prediction frame, and B is a bidirectional interpolation frame. Simply put, an I frame is a complete picture, while P frames and B frames record the changes relative to the I frame. Without I frames, P frames and B frames cannot be decoded. Therefore, key frames are also called I frames. They are important frames in inter-frame compression coding. When decoding, only the data of I frames can be used to reconstruct the complete image. I frames do not need to refer to other pictures to generate. I frames can be used as reference points for seeking.

[0030] GOP (group of pictures): A GOP is a group of continuous pictures with the same characteristics. A GOP starts with a key frame I frame, followed by a group of B frames and P frames. When an image is very different from the previous image and cannot be generated by referring to the previous frame, the previous GOP group of pictures ends and the next GOP group of pictures begins.

[0031] Figure 1 An example structure of an audio and video data frame buffer (hereinafter referred to as buffer) according to an embodiment of the present invention is shown. In the context of the present invention, the data in the buffer is obtained from an upstream device or module (eg, a server storing audio and video, etc.).

[0032] like Figure 1 As shown, 101 is an audio and video data frame buffer, which contains multiple data frame fragments 102. The audio and video buffers are separate, that is, a buffer contains either only audio data frame fragments or only video data frame fragments.

[0033] Figure 2 A diagram of an audio data frame segment 201 according to an embodiment of the present invention is shown. The audio data frame segment 201 includes an audio data frame list 202, in which a number of time-continuous audio data frames can be stored. Specifically, the audio data frame list 202 includes both audio data frames 204 that have been read and audio data frames 203 that have not been read. 205 represents the first audio data frame that has not been read, and 206 represents the audio data frame that will be added to the end of the audio data frame list 202.

[0034] Figure 3 1 shows a diagram of a video data frame segment 301 according to an embodiment of the present invention. The video data frame segment 301 includes a video data frame list 302 and a key data frame (I frame) list 308. The video data frame list 302 includes both video data frames 305 that have been read, and ordinary video data frames 304 and I frames 303 that have not been read. 306 represents the first video data frame that has not been read, and 307 represents the I frame of the GOP to which the first video data frame 306 that has not been read belongs. Each member 309 of the key data frame (I frame) list 308 represents the position of the I frame in 302, and 310 represents the video data frame that will be added to the end of the video data frame list 302.

[0035] return Figure 1Each buffer has a hardLimit parameter and a softLimit parameter to limit the amount of data in the buffer, where hardLimit>softLimit. The amount of data stored in the buffer generally does not exceed the softLimit. If it exceeds the softLimit, it will stop receiving new data and notify the download module to stop downloading. In addition, at any time, the amount of data in the buffer must not exceed the hardLimit.

[0036] Preferably, hardLimit is a fixed value (e.g., 50MB) determined based on the actual situation of system resources and in combination with experimental results. In addition, softLimit can be dynamically adjusted during operation. For example, in the case of poor network speed (e.g., less than 500KB / s), set it to a larger value to ensure smooth playback of streaming media for a relatively long period of time; in the case of good network speed (e.g., greater than 1MB / s), set it to a smaller value to reduce memory waste. Preferably, softLimit can generally be set to hardLimit minus 1 / 4 of the size of the streaming media segment file. That is, softLimit = hardLimit-size of the streaming media segment file / 4. Of course, other methods can also be used to determine softLimit, and softLimits of other sizes are also within the scope of the present invention.

[0037] According to one embodiment of the present invention, the buffer can be divided into three states. The first state is the seeking state. When the user initiates a seeking operation, if there is no search target frame in the buffer, and the search target frame needs to be downloaded from the server, the buffer will enter the seeking state. The second state is the underflow state, that is, due to reasons such as network freezes, physical failure of the device, etc., all the data frames in the current buffer have been played, and the buffer is in a state of waiting for new data frames from the server, which is called the underflow state. The third state is the normal state, that is, a state other than the above seeking state and the underflow state, such as a state in which the data frames can be played normally and smoothly, so that the upper module (for example, the player) can read the data frames smoothly from the buffer, which is a normal state.

[0038] Furthermore, in the context of the present invention, "non-seeking state" may refer to the buffer being in an underflow state or a normal state.

[0039] According to an embodiment of the present invention, the data frame segments in the buffer can be divided into three categories: active data frame segments, inactive data frame segments and search data frame segments. Among them, the active data frame segments buffer the data frames played from the beginning of the audio and video or played since the last search state was exited, and the data frames of a certain predetermined buffering time length after the current timestamp. The player reads the data frames from the active data frame segments for playback in the non-search state.

[0040] Inactive data frame segments buffer historical data frame segments so that when these historical data frames are played back later, there is no need to download them from the server. Each buffer has only one active data frame segment at a time, and the other data frame segments are inactive.

[0041] As described above, when the user initiates a seek operation, the buffer will be in a seeking state. If there is no buffered seek target frame in both the active data frame segment and the inactive data frame segment, a seek data frame segment will be created in the buffer, and all subsequent new data frames will be added to the seek data frame segment. At this time, there is no active data frame segment in the buffer.

[0042] In the context of the present invention, according to different implementations, "search target frame" may refer to the target frame corresponding to the search operation. For example, the search target frame may refer to the data frame corresponding to the time when the user drags the progress bar (i.e., the search target frame is one data frame in this case) or may refer to the data frame corresponding to the time when the user drags the progress bar and the data frames within a certain predetermined search time length thereafter (i.e., the search target frame is a group of continuous data frames in this case). When a data frame with a timestamp greater than the timestamp of the search target frame is received from the server, the search data frame segment may be set as an active data frame segment, at which point the buffer exits the search state.

[0043] In the context of the present invention, a "current data frame" may refer to the first unread data frame in the active data frame segment, ie, the data frame to be read by the next read operation.

[0044] refer to Figure 1 , 101 is an audio and video data frame buffer, which contains multiple data frame segments 102, and the buffer is not in a seeking state (seekSegment (search data frame segment) = null (empty)); 101' represents a buffer in a seeking state (activeSegment (active data frame segment) = null (empty), seekSegment (search data frame segment) is not empty).

[0045] For example, assuming that a video is currently being played and the video currently being played to the timestamp of 3:00, and the predetermined buffering time length is 2 minutes, the active data frame segment can buffer the video data frames from the start of the video (0:00) to the 2 minutes after 3:00 (i.e., to 5:00) to facilitate smooth viewing by the user.

[0046] Assume scenario one, when the user initiates a search operation, for example, when the user pulls the progress bar back from the current 3:00 to 4:00, the active data frame segment is searched first, because the video data frame at 4:00 has been buffered in the active data frame segment at this time, so the video data frame is read directly from the active data frame segment.

[0047] Assume scenario 2, when the user initiates a search operation, for example, when the user pulls the progress bar forward from the current 3:00 to 6:00, the active data frame segment is first searched, because the video data frame at 6:00 is not buffered in the active data frame segment at this time, so the search is further performed from the inactive data frame segment. If the video data frame at 6:00 does not exist in the inactive data frame segment, the buffer enters the search state, converts the active data frame segment into an inactive data frame segment (that is, the inactive data frame segment includes the video data frames between 0:00 and 5:00), and creates a search data frame segment, so that the video data frames starting from 6:00 downloaded from the server are buffered into the search data frame segment, and the video data frames are read from the search data frame segment for playback. After the search target frame is downloaded from the server, the buffer can exit the search state, and the search data frame segment can be converted into an active data frame segment, that is, subsequent data frames can be added to the active data frame segment and data frames can be read from it.

[0048] As can be seen from the above, each time the user performs a search operation, if there is no search target frame in the current active and inactive data frame segments, the current active data frame segment can be converted into an inactive data frame segment, and a search data frame segment can be created. Moreover, after downloading the search target frame, the search data frame segment can be converted into an active data frame segment. In this way, each buffer has only one active data frame segment at a time, and the other data frame segments are inactive.

[0049] Figure 4-1 to Figure 4-4 The flowchart of the method for adding data frames to a buffer according to an embodiment of the present invention is shown. The method can be executed by a module for controlling data access to the buffer.

[0050] First, refer to Figure 4-1 At 401, a new data frame is received. The new data frame may be an audio data frame or a video data frame.

[0051] At 402, it is determined whether the amount of data already stored in the buffer plus the new data frame will exceed the softLimit. If not, proceed to 403 to add a new data frame, and if so, proceed to 406 to delete the data frame from the buffer.

[0052] At 403, it is checked whether the buffer is currently in the search state. According to one embodiment of the present invention, it can be determined whether the current buffer is in the search state by checking whether there is a search data frame fragment in the buffer. If the buffer is currently in the search state, it proceeds to 404; if not, it proceeds to 405.

[0053] At 404, the new data frame is added to the end of the search data frame segment. Figure 4-2 To further describe the process of adding new data frames to the buffer in the search state.

[0054] At 405, the new data frame is added to the end of the active data frame segment. Figure 4-3 To further describe the process of adding a new data frame to the buffer in the non-seeking state.

[0055] At 406, old data frames are deleted from the buffer. Specifically, if there are inactive data frame segments in the buffer, the inactive data frame segments are deleted until the cumulative amount of released data exceeds the size of the new data frame. If there are no inactive data frame segments in the buffer, the read data frames are deleted from the active data frame segments when the buffer is in a non-searching state until the cumulative amount of released data exceeds the size of the new data frame, or when the buffer is in a searching state, it is further determined whether the size of the data frames stored in the buffer plus the size of the new data frame exceeds the hardLimit. The following is combined with Figure 4-4 To further describe the process of deleting old data frames.

[0056] At 407, it is determined whether the old data frame is deleted successfully. If yes, proceed to 402 to execute method 400 for the next new data frame, and if no, proceed to 408. According to one embodiment of the present invention, if the size of the data frame stored in the buffer plus the size of the new data frame exceeds hardLimit, it can be determined that the old data frame has failed to be deleted.

[0057] At 408, downloading of the new data frame is stopped.

[0058] Figure 4-2 A flowchart is shown for further describing the above step 404, ie, adding a new data frame to the buffer in the search state, according to an embodiment of the present invention.

[0059] At 441, the new data frame is added to the end of the search data frame segment.

[0060] At 442, determine whether the new data frame is a video I frame. If yes, proceed to 443, otherwise proceed to 444.

[0061] At 443 , the position information of the new data frame is updated into the key data frame list for searching the data frame segment.

[0062] At 444, determine whether the timestamp of the new time frame is greater than the timestamp of the search target frame. If yes, proceed to 445, if not, the adding is complete.

[0063] At 445, the search data frame segment is set as the active data frame segment.

[0064] At 446, the buffer state is set to a normal state.

[0065] Figure 4-3 FIG. 4 is a flowchart further describing step 405 , ie, adding a new data frame to the buffer in a non-seeking state, according to an embodiment of the present invention.

[0066] At 451, the new data frame is added to the end of the active data frame segment.

[0067] At 452, determine whether the new data frame is a video I frame. If yes, proceed to 453, otherwise proceed to 454.

[0068] At 453 , the location information of the new data frame is updated to the key data frame list of the active data frame segment.

[0069] At 454, it is determined whether the buffer is in an underflow state. If so, proceed to 455, and if not, the addition is complete.

[0070] At 455, the buffer is set to a normal state. According to one embodiment of the present invention, setting the buffer to a normal state may indicate that the buffer can buffer data frames normally, thereby notifying an upper layer module (eg, a player) that data frames can be read from the buffer.

[0071] Figure 4-4 A flowchart is shown to further describe the above step 406, ie, deleting old data frames, according to an embodiment of the present invention.

[0072] At 461, inactive data frame segments are deleted. According to one embodiment of the present invention, when there are more than one inactive data frame segments in the buffer, multiple inactive data frame segments may be deleted. For example, a predetermined number / predetermined ratio of inactive data frame segments may be deleted, all inactive data frame segments may be deleted, or the number of inactive data frame segments to be deleted may be dynamically determined according to the size of the new data frame to be received.

[0073] At 462 , it is determined whether the amount of data stored in the buffer after deleting the inactive data frame segment plus the new data frame exceeds the soft limit. If yes, proceed to 463 , if not, proceed to 467 .

[0074] At 463 , determine whether the buffer is in a seek state, if so, proceed to 465 , if not, proceed to 464 .

[0075] At 464, the read data frames are deleted from the active data frame segment. According to one embodiment of the present invention, when there are more than one read data frames in the active data frame segment, multiple read data frames can be deleted. For example, a predetermined number / predetermined ratio of read data frames can be deleted, all read data frames can be deleted, or the number of read data frames to be deleted can be dynamically determined according to the size of the new data frame to be received.

[0076] At 465, determine whether the amount of data stored in the buffer plus the new data frame exceeds the hard limit. If not, proceed to 467, and if yes, proceed to 466.

[0077] At 466, the old data frame fails to be deleted. According to an embodiment of the present invention, in the case where the old data frame fails to be deleted, the downloading of the new data frame may be stopped.

[0078] At 467, the old data frame is deleted successfully. According to an embodiment of the present invention, when the old data frame is deleted successfully, an instruction to receive the next data frame may be issued, so as to receive a subsequent new data frame.

[0079] Figure 5-1 to Figure 5-2 The flowchart of a method for reading a data frame from a buffer according to an embodiment of the present invention is shown. The method can be executed by a module for controlling data access to the buffer.

[0080] First, see Figure 5-1 , generally speaking, Figure 5-1 Describes how to determine whether the buffer is in the search state. If it is in the search state, it directly returns an empty data frame. If it is not in the search state, it reads the data frame from the active data frame fragment in the buffer. The details are as follows:

[0081] At 501, a request for reading a data frame is received. According to an embodiment of the present invention, the request may be issued by an upper module of the buffer (eg, a player).

[0082] At 502, determine whether the buffer is in a seek state. If yes, proceed to 503, if not, proceed to 504.

[0083] At 503, a null pointer is returned and the process ends. According to one embodiment of the present invention, if the buffer is in a search state, it is very likely that the search target frame has not been downloaded from the server, so a null pointer may be returned.

[0084] At 504, determine whether there are any unread data frames in the active data frame segment. If yes, proceed to 505, if not, proceed to 503 (ie, return a null pointer).

[0085] At 505 , unread data frames in the active data frame segment are returned.

[0086] For further information, see Figure 5-2 , which further describes the above step 505. Generally speaking, step 505 includes determining whether there are any data frames in the active data frame segment of the buffer that have not been read; if there are any data frames in the active data frame segment that have not been read, take the data frame with the earliest timestamp as the return result, and if the amount of data stored in the current buffer exceeds the softLimit, the data frame to be returned will be deleted from the active data frame segment, and if the deleted frame is a video I frame, the frame will also be deleted from the key frame list. Furthermore, if all data frames in the active data frame segment have been read, the buffer is set to an underflow state, waiting for new data frames to enter, and an empty data frame is returned. The details are as follows:

[0087] At 551, the first unread data frame in the active data frame segment is returned. According to an embodiment of the present invention, the first unread data frame is the unread data frame with the earliest timestamp.

[0088] At 552, it is determined whether the storage capacity of the buffer exceeds the softLimit. If yes, the process proceeds to 553, and if no, the process ends.

[0089] At 553 , the first unread data frame in the active data frame segment is deleted from the active data frame segment.

[0090] At 554, determine whether the first unread data frame to be deleted is a video I frame. If yes, then enter 555, if no, then process ends.

[0091] At 555 , the position information of the first unread data frame to be deleted is deleted from the key frame list of the active data frame segment.

[0092] Figure 6-1 to Figure 6-3 A flow chart of a method for responding to a search request according to an embodiment of the present invention is described. The method can be executed by a module for controlling buffer data access.

[0093] At 601, a search request is received. According to one embodiment of the present invention, the search request may be issued by the player, for example, by a user manipulating a progress bar or other controls of the player. The search request may include a timestamp of a search target frame required by the search.

[0094] At 602, it is determined whether the active data frame segment includes a search target frame. If not, proceed to 603, and if not, proceed to 604.

[0095] At 603, the search request is responded to based on whether the search target frame is included in the inactive data frame segment. Figure 6-2 The operation of step 603 is further described.

[0096] At 604, the search request is responded to by comparing the search target frame with the current frame in the active data frame segment. Figure 6-3 The operation of step 604 is further described.

[0097] Figure 6-2 The sub-process of step 603 is shown. That is, the sub-process for responding to the search request when the search target frame is not included in the active data frame segment. Generally speaking, if the search target frame is not found in the active data frame segment of the buffer, other inactive data frame segments are searched. Further, if the search target frame is found in a certain inactive data frame segment, the inactive data frame segment is set as the active data frame segment (that is, the data frame will be read from the active data frame segment next), and the I frame of the GOP where the search target frame is located is returned as a response to the search request, and the next read operation will return the next frame of the found I frame.

[0098] Furthermore, if the search target frame is not found in other inactive data frame fragments, the buffer is set to search mode, and a new data frame fragment is created in the buffer, that is, the search data frame fragment, and subsequent new data frames will be added to the newly created search data frame fragment, notifying the download module to download new data frames according to the timestamp of the search target frame.

[0099] The details are as follows:

[0100] At 631 , it is determined whether one or more inactive data frame segments in the buffer include a search target frame. If yes, the process proceeds to 632 , and if not, the process proceeds to 634 .

[0101] At 632, the inactive data frame segment where the search target frame is located is set as the active data frame segment, and the I frame of the GOP where the search target frame is located is set as the current data frame of the active data frame segment (ie, the next data frame to be read).

[0102] At 633, the current data frame of the active data segment is returned and the process ends.

[0103] At 634, a search data frame segment is created and the current active data frame segment is set to empty, whereby the buffer enters a search state.

[0104] At 635, the download module is notified to download the search target frame according to the timestamp of the search target frame. According to one embodiment of the present invention, after the search target frame is downloaded, it will be added to the newly created search data frame segment.

[0105] At 636, an empty data frame is returned.

[0106] Figure 6-3 The sub-flow of step 604 is shown. That is, the sub-flow of responding to the search request when the search target frame is included in the active data frame segment. Generally speaking, if the search target frame is found in the active data frame segment, it is determined whether the search target frame and the current data frame of the active data frame segment (that is, the next data frame to be read) are in the same GOP, that is, it is determined whether the two are between two adjacent elements in the key frame list.

[0107] If the search target frame and the current data frame are in the same GOP, and if the timestamp of the search target frame is smaller than the timestamp of the current data frame, the I frame of the GOP where the search target frame is located is returned as a response to the search request, and the next data frame of the found I frame in the active data frame segment is set as the current data frame; if the timestamp of the search target frame is greater than the timestamp of the current data frame, the current data frame is returned as a response to the search instruction.

[0108] If the search target frame and the current data frame are not in the same GOP, the I frame of the GOP where the search target frame is located is returned as a response to the search request, and the next frame of the found I frame is set as the current data frame in the active data frame segment.

[0109] The details are as follows:

[0110] At 641 , it is determined whether the search target frame in the active data frame segment and the current data frame of the active data frame segment are in the same GOP. If yes, the process proceeds to 643 , and if not, the process proceeds to 642 .

[0111] At 642, the I frame of the GOP to which the search target frame belongs is set as the current data frame of the active data frame segment.

[0112] At 643 , it is determined whether the timestamp of the search target frame is greater than the timestamp of the current data frame of the active data frame segment. If yes, the process proceeds to 644 , and if not, the process proceeds to 642 .

[0113] At 644, the current data frame of the active data frame segment is returned as a response to the seek request.

[0114] In summary, the above describes the composition structure of the audio / video data frame buffer area of ​​the present invention, the processing of adding data frames to the buffer area, the processing of reading data frames from the buffer area, and the response to the search operation. It can be seen that the method and system for managing the caching of audio and video data frames by the streaming media player provided by the present invention can effectively reduce memory overhead without exceeding the set cache limit while ensuring the streaming media playback quality as much as possible.

[0115] Figure 7 A block diagram of a system 700 for audio and video data frame buffer management according to various embodiments of the present invention is shown. The system 700 may include a download module 705 and a buffer data access control module 710. It is fully understood by those skilled in the art that the division of the above modules is only explained for the purpose of clarity. The functions of one or more of the above modules may be combined into a single module or split into more modules / submodules. Moreover, one or more of the above modules may be implemented in software, hardware, or a combination thereof. In addition, the data flow between the modules may be implemented in a manner known in the art, which is not discussed in the present invention.

[0116] According to one embodiment of the present invention, the download module 705 can be configured to download audio data frames or video data frames from a server (e.g., remote). In addition, the download module 705 can also be configured to start / stop downloading the data frames based on the instruction from the buffer data access control module 710.

[0117] According to one embodiment of the present invention, the buffer data access control module 710 may be configured to perform the following steps: Figures 4-1 to 4-3 The operations described for adding data frames to the buffer, such as Figure 5-1 to Figure 5-2 Describes the operations for reading data frames from a buffer and how Figures 6-1 to 6-3 Describes the operation used to respond to a crawl request.

[0118] According to one embodiment of the present invention, the system 700 may be implemented in a vehicle to be used in conjunction with an audio and / or video playback device installed in the vehicle (eg, an in-vehicle system).

[0119] Figure 8A block diagram of an exemplary computing device according to one embodiment of the present invention is shown, which is one example of a hardware device applicable to various aspects of the present invention.

[0120] refer to Figure 8 , a computing device 800 will now be described, which is an example of a hardware device applicable to various aspects of the present invention. The computing device 800 can be any machine that can be configured to perform processing and / or computing, and can be but is not limited to a workstation, a server, a desktop computer, a laptop computer, a tablet computer, a personal digital assistant, a smart phone, a vehicle-mounted computer, or any combination thereof. The various methods / apparatus / server / client devices described above can be implemented in whole or in part by the computing device 800 or a similar device or system.

[0121] The computing device 800 may include components that can be connected or communicated via one or more interfaces and a bus 802. For example, the computing device 800 may include a bus 802, one or more processors 804, one or more input devices 806, and one or more output devices 808. The one or more processors 804 may be any type of processor and may include, but are not limited to, one or more general-purpose processors and / or one or more special-purpose processors (e.g., a dedicated processing chip). The input device 806 may be any type of device capable of inputting information to the computing device and may include, but are not limited to, a mouse, a keyboard, a touch screen, a microphone, and / or a remote controller. The output device 808 may be any type of device capable of presenting information and may include, but are not limited to, a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The computing device 800 may also include or be connected to a non-transient storage device 810, which may be any storage device that is non-transient and capable of data storage, and may include, but is not limited to, a disk drive, an optical storage device, a solid-state memory, a floppy disk, a floppy disk, a hard disk, a tape or any other magnetic medium, an optical disk or any other optical medium, a ROM (read-only memory), a RAM (random access memory), a cache memory, and / or any memory chip or cassette, and / or any other medium from which a computer can read data, instructions, and / or code. The non-transient storage device 810 may be detachable from the interface. The non-transient storage device 810 may have data / instructions / code for implementing the above methods and steps. The computing device 800 may also include a communication device 812. The communication device 812 can be any type of device or system that can communicate with internal devices and / or with a network and can include but is not limited to a modem, a network card, an infrared communication device, a wireless communication device and / or a chipset, such as a Bluetooth device, an IEEE 1302.11 device, a WiFi device, a WiMax device, a cellular communication device and / or the like.

[0122] When the computing device 800 is used as an in-vehicle device, it can also be connected to external devices (e.g., a GPS receiver, sensors for sensing different environmental data (such as an accelerometer, a wheel speed sensor, a gyroscope, etc.)). In this way, the computing device 800 can, for example, receive positioning data and sensor data indicating the vehicle's form condition. When the computing device 800 is used as an in-vehicle device, it can also be connected to other devices (e.g., an engine system, wipers, an anti-lock brake system, a vehicle climate setting system, etc.) for controlling the travel and operation of the vehicle (e.g., vehicle climate settings). When the computing device 800 is used as an in-vehicle device, it can also access the Internet or access a prototyping tool through a VPN.

[0123] In addition, the non-transitory storage device 810 may have map information and software components so that the processor 804 can implement route guidance processing. In addition, the output device 806 may include a display for displaying a map, a positioning mark of the vehicle, and an image indicating the driving status of the vehicle. The output device 806 may also include a speaker or a headphone interface for audio guidance.

[0124] The bus 802 may include, but is not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus. In particular, for vehicle-mounted devices, the bus 802 may also include a Controller Area Network (CAN) bus or other structures designed for automotive applications.

[0125] The computing device 800 may also include a working memory 814 , which may be any type of working memory capable of storing instructions and / or data that facilitate the operation of the processor 804 and may include, but is not limited to, a random access memory and / or a read-only storage device.

[0126] Software components may be located in the working memory 814, including but not limited to an operating system 816, one or more application programs 818, drivers and / or other data and codes. Instructions for implementing the above methods and steps may be included in the one or more application programs 818, and the modules / units / components of the aforementioned various devices / servers / client devices may be implemented by the processor 804 reading and executing the instructions of the one or more application programs 818.

[0127] It should also be recognized that changes may be made according to specific needs. For example, custom hardware may also be used, and / or specific components may be implemented in hardware, software, firmware, middleware, microcode, hardware description language, or any combination thereof. In addition, connections to other computing devices, such as network input / output devices, etc. may be employed. For example, programming hardware (e.g., programmable logic circuits including field programmable gate arrays (FPGAs) and / or programmable logic arrays (PLAs)) with assembly language or hardware programming languages ​​(e.g., VERILOG, VHDL, C++) may be used to implement part or all of the disclosed methods and devices using logic and algorithms according to the present invention.

[0128] Although various aspects of the present invention have been described so far with reference to the accompanying drawings, the above-described methods, systems and devices are merely examples, and the scope of the present invention is not limited to these aspects, but is limited only by the appended claims and their equivalents. Various components may be omitted or replaced by equivalent components. In addition, the steps may be implemented in an order different from the order described in the present invention. In addition, various components may be combined in various ways. It is also important that, as technology develops, many of the components described may be replaced by equivalent components that appear later.

Claims

1. A method for adding a data frame to a buffer, comprising: In response to receiving a new data frame, determining whether the amount of data stored in the buffer plus the amount of data of the new data frame exceeds a soft limit defined for the buffer, wherein the soft limit is dynamically determined during operation; In response to determining that the amount of data stored in the buffer plus the amount of data of the new data frame does not exceed the soft limit, adding the new data frame to the end of the active data frame segment in the buffer when the buffer is not in a seek state; or In response to determining that the amount of data stored in the buffer plus the amount of data of the new data frame does not exceed the soft limit, the new data frame is added to the end of the search data frame segment in the buffer when the buffer is in the search state, wherein the search data frame segment was previously established when the buffer entered the search state.

2. The method according to claim 1, characterized in that Adding the new data frame to the end of the active data frame segment in the buffer further comprises: If the new data frame is a key frame, the position information of the new data frame is updated to the key data frame list of the active data frame segment.

3. The method according to claim 1, characterized in that Adding the new data frame to the end of the search data frame segment in the buffer further comprises: Determine whether the timestamp of the new data frame is greater than the timestamp of the search target frame; In response to determining that the timestamp of the new data frame is greater than the timestamp of the search target frame, the search data frame segment is set as an active data frame segment after the new data frame is added to the end of the search data frame segment in the buffer, and the buffer exits the search state.

4. The method of claim 1, further comprising: In response to determining that the amount of data stored in the buffer plus the amount of data of the new data frame exceeds the soft limit, deleting old data frames in the buffer; as well as In response to determining that deleting the old data frame in the buffer is unsuccessful, downloading of the new data frame is stopped.

5. The method according to claim 4, characterized in that Deleting the old data frame in the buffer further comprises: Deleting inactive data frame fragments in the buffer; In response to determining that the amount of data stored in the buffer after deleting the inactive data frame segment plus the amount of data of the new data frame does not exceed the soft limit, deleting is stopped.

6. The method of claim 5, further comprising: In response to determining that the amount of data stored in the buffer after deleting the inactive data frame segment plus the amount of data of the new data frame exceeds the soft limit, deleting the read data frame from the active data frame segment when the buffer is not in the seek state; Determining whether the amount of data stored in the buffer plus the amount of data of the new data frame exceeds a hard limit of the buffer, the hard limit being greater than the soft limit; In response to determining that the amount of data stored in the buffer plus the amount of data of the new data frame exceeds the hard limit of the buffer, downloading of the new data frame is stopped.

7. A method for reading a data frame from a buffer, comprising: determining whether the buffer is in a seek state; In response to determining that the buffer is not in the seek state, returning a first unread data frame in an active data frame segment in the buffer; Determining whether the amount of data stored in the buffer exceeds a soft limit defined for the buffer, wherein the soft limit is dynamically determined during operation; as well as In response to determining that the amount of data stored in the buffer exceeds the soft limit, the first unread data frame is deleted from the active data frame segment.

8. The method of claim 7, further comprising: In response to determining that the buffer is in the seek state, an empty data frame is returned.

9. The method of claim 7, further comprising: determining that the first unread data frame deleted from the active data frame segment is a key frame; as well as The position information of the first unread data frame is deleted from the key frame list of the active data frame segment.

10. A method for responding to a search request, comprising: In response to a search request, determining whether the active data frame segments in the buffer include a search target frame indicated by the search request; In response to determining that the active data frame segment does not include the search target frame, responding to the search request based on whether the inactive data frame segment in the buffer includes the search target frame; or In response to determining that the active data frame segment includes the search target frame, responding to the search request by comparing the search target frame with a current data frame in the active data frame segment.

11. The method according to claim 10, characterized in that Responding to the search request based on whether the search target frame is included in the inactive data frame segments in the buffer further includes: In response to the inactive data frame segment including the search target frame, setting the inactive data frame segment as an active data frame segment, and setting a key frame of a GOP where the search target frame is located as a current data frame of the active data frame segment; The current data frame is returned as a response to the seek request.

12. The method according to claim 10, characterized in that Responding to the search request based on whether the search target frame is included in the inactive data frame segments in the buffer further includes: In response to the inactive data frame segment not including the search target frame, creating a search data frame segment, and setting the active data frame segment to be empty, thereby indicating that a subsequent new data frame will be added to the search data frame segment; An empty data frame is returned as a response to the seek request.

13. The method according to claim 10, characterized in that Responding to the search request by comparing the search target frame with a current data frame in the active data frame segment further comprises: Determining whether the search target frame in the active data frame segment and the current data frame of the active data frame segment are in the same GOP; In response to determining that the search target frame in the active data frame segment and the current data frame of the active data frame segment are in the same GOP, and the timestamp of the search target frame is greater than the timestamp of the current data frame of the active data frame segment, returning the current data frame of the active data frame segment as a response to the search request; or In response to determining that the search target frame in the active data frame segment and the current data frame of the active data frame segment are in the same GOP, and the timestamp of the search target frame is less than the timestamp of the current data frame of the active data frame segment, setting the key frame of the GOP to which the search target frame belongs as the current data frame of the active data frame segment, and returning the current data frame of the active data frame segment as a response to the search request; or In response to determining that the search target frame in the active data frame segment and the current data frame of the active data frame segment are not in the same GOP, the key frame of the GOP to which the search target frame belongs is set as the current data frame of the active data frame segment, and the current data frame of the active data frame segment is returned as a response to the search request.

14. A system for managing audio and video data frame buffers, comprising: A download module, wherein the download module is configured to download or stop downloading audio data frames or video data frames from a server; as well as A buffer data access control module, wherein the buffer data access control module is configured to execute the method as claimed in any one of claims 1 to 13.

15. A vehicle comprising: The system for managing audio and video data frame buffers as claimed in claim 14; as well as Audio and / or video playback equipment.

Images