Audio and video playback method, device, system, electronic device and storage medium
By detecting the duration of cached data by live audio and video live broadcast and sending keyframes of real-time audio and video data, the bandwidth and cost increase in Internet of Things audio and video live broadcast is solved, and rapid playback and traffic optimization are achieved.
Patent Information
- Application Number
- CN202310096241.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-01-17
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2043-01-17
AI Technical Summary
In the live audio and video broadcast scenario of IoT, when there is a lot of cached data, sending audio and video data will increase the Internet bandwidth load and cost, affecting the user experience.
By detecting whether the cached data is greater than the set threshold, if it is greater than the threshold, a control signal is sent to the audio and video generation device, real-time audio and video data is obtained, and keyframes of real-time audio and video data are sent to reduce the traffic during a single playback.
While ensuring fast live broadcast playback, it reduces the instantaneous traffic during a single playback, and reduces the load and cost of Internet bandwidth.
Smart Images

Figure CN116055810B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of audio and video processing technology, and in particular to an audio and video playback method, device, system, electronic device and storage medium. Background Art
[0002] In recent years, with the development of the Internet, the cost of network bandwidth has gradually decreased. A large number of IoT audio and video devices are directly connected to cloud services through the Internet. More and more users can view the audio and video live broadcasts of video devices anytime and anywhere through terminal devices such as mobile phones and tablets. In the live broadcast scenario based on IoT audio and video devices, the first screen playback speed is the core indicator that affects the user experience. In order to increase the playback speed and quickly open the live video, some audio and video data are often cached first. When a new user watches the live broadcast, the cached data will be sent to the user first, and then the real-time live data will be sent. However, when there is a lot of cached data, each time you watch, the traffic will increase due to sending more cached data, which in turn increases the load and cost of the Internet bandwidth. Summary of the invention
[0003] The embodiments of the present application provide an audio and video playback method, device, system, electronic device and storage medium to solve the problems existing in the prior art.
[0004] In a first aspect, an embodiment of the present application provides an audio and video playback method, which is applied to a server, and the method includes:
[0005] In response to a playback request from an audio or video playback device, determining whether a first duration of the cached data is greater than a set duration threshold;
[0006] When the first duration is greater than the duration threshold, sending a first control signal to the audio and video generating device;
[0007] Acquire real-time audio and video data fed back by the audio and video generating device based on the first control signal;
[0008] Send real-time audio and video data to the audio and video playback device; the first frame of the real-time audio and video data is a key frame.
[0009] In a second aspect, an embodiment of the present application provides an audio and video playback method, which is applied to an audio and video generation device, and the method includes:
[0010] In response to a first control signal from the server, real-time audio and video data is generated; the first control signal is transmitted by the server to the audio and video generating device through the audio and video playing method provided by the first aspect of the embodiment of the present application;
[0011] Send real-time audio and video data to the server.
[0012] In a third aspect, an embodiment of the present application provides an audio and video playback method, which is applied to an audio and video playback device, and the method includes:
[0013] Send a play request to the server;
[0014] Receive real-time audio and video data fed back by the server in response to the playback request; the real-time audio and video data is transmitted by the server to the audio and video playback device through the audio and video playback method provided by the first aspect of the embodiment of the present application;
[0015] Play real-time audio and video data.
[0016] In a fourth aspect, an embodiment of the present application provides an audio and video playback device, which is applied to a server, and the device includes:
[0017] An analysis module, configured to determine, in response to a playback request from an audio or video playback device, whether a first duration of the cached data is greater than a set duration threshold;
[0018] A control module, configured to send a first control signal to the audio and video generating device when the first duration is greater than a duration threshold;
[0019] An acquisition module, used to acquire real-time audio and video data fed back by the audio and video generating device based on the first control signal;
[0020] The sending module is used to send real-time audio and video data to the audio and video playback device; the first frame of the real-time audio and video data is a key frame.
[0021] In a fifth aspect, an embodiment of the present application provides an audio and video playback device, which is applied to an audio and video generation device, and the device includes:
[0022] A generating module, configured to generate real-time audio and video data in response to a first control signal from a server; the first control signal is transmitted by the server to the audio and video generating device through the audio and video playing method provided by the first aspect of the embodiment of the present application;
[0023] The sending module is used to send real-time audio and video data to the server.
[0024] In a sixth aspect, an embodiment of the present application provides an audio and video playback device, which is applied to an audio and video playback device, and the device includes:
[0025] A request module, used to send a playback request to the server;
[0026] A receiving module, used to receive real-time audio and video data fed back by the server in response to a playback request; the real-time audio and video data is transmitted from the server to the audio and video playback device through the audio and video playback method provided by the first aspect of the embodiment of the present application;
[0027] The playback module is used to play real-time audio and video data.
[0028] In a seventh aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, and when the processor executes the computer program, the audio and video playback method provided in the first aspect of the embodiment of the present application is implemented.
[0029] In an eighth aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, and when the processor executes the computer program, the audio and video playback method provided in the second aspect of the embodiment of the present application is implemented.
[0030] In a ninth aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor implements the audio and video playback method provided in the third aspect of the embodiment of the present application when executing the computer program.
[0031] In a tenth aspect, an embodiment of the present application provides an audio and video playback system, including: a server, an audio and video generation device, and an audio and video playback device;
[0032] The server is the electronic device provided in the seventh aspect of the embodiment of the present application, the audio and video generating device is the electronic device provided in the eighth aspect of the embodiment of the present application, and the audio and video playing device is the electronic device provided in the ninth aspect of the embodiment of the present application;
[0033] The audio and video generating device communicates with the server via the Internet, and the server and the audio and video playing device communicate with each other via the Internet.
[0034] In the eleventh aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any one of the audio and video playback methods provided in the embodiments of the present application.
[0035] Compared with the prior art, this application has the following advantages:
[0036] According to the audio and video playback solution provided in the embodiment of the present application, the playable duration of the cached data, i.e., the first duration, can be used to reflect the size of the cached data. When the first duration is greater than the set duration threshold, it can be considered that the first duration is longer and the corresponding cached data has a larger data volume. At this time, the audio and video generation device is controlled by the first control signal to generate and feedback new audio and video data, i.e., the above-mentioned real-time audio and video data, so that the new audio and video data can be fed back to the audio and video playback device, so that the audio and video playback device can play the new audio and video playback device without playing the cached data. While ensuring the fast playback of the live broadcast, the instantaneous traffic during a single playback can be reduced to avoid excessive traffic consumption, thereby reducing the load and cost of the Internet bandwidth.
[0037] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0038] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments according to the present application and should not be regarded as limiting the scope of the present application.
[0039] Figure 1 A schematic diagram of an application scenario of the audio and video playback solution provided in an embodiment of the present application;
[0040] Figure 2 A flowchart of an audio and video playback method provided in an embodiment of the present application;
[0041] Figure 3 A flowchart of another audio and video playback method provided in an embodiment of the present application;
[0042] Figure 4 A flowchart of another audio and video playback method provided in an embodiment of the present application;
[0043] Figure 5 A schematic diagram of an interaction principle among a server, an audio and video generating device, and an audio and video playing device in an embodiment of the present application;
[0044] Figure 6 A schematic diagram of another interaction principle among a server, an audio and video generating device, and an audio and video playing device in an embodiment of the present application;
[0045] Figure 7 A schematic diagram of the structural framework of an audio and video playback device provided in an embodiment of the present application;
[0046] Figure 8 A schematic diagram of the structural framework of another audio and video playback device provided in an embodiment of the present application;
[0047] Fig. 9 A schematic diagram of the structural framework of another audio and video playback device provided in an embodiment of the present application;
[0048] Fig.10 A schematic diagram of data flow between various audio and video playback devices provided in an embodiment of the present application;
[0049] Fig.11A schematic diagram of the structural framework of an electronic device provided in an embodiment of the present application;
[0050] Fig.12 A schematic diagram of the structural framework of another electronic device provided in an embodiment of the present application; and
[0051] Fig.13 A schematic diagram of the structural framework of another electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0052] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the concept or scope of the present application. Therefore, the drawings and descriptions are considered to be exemplary in nature and not restrictive.
[0053] First, some technical terms involved in this application are introduced as follows:
[0054] Video coding: A method of converting raw video data into another video format file through compression technology. Currently, the most important codec standards for video streaming transmission include ITU's H.264 (a video compression coding standard), Motion-Join Photographic Experts Group's M-JPEG (Motion-Join Photographic Experts Group, motion still images or frame-by-frame compression technology) and ISO's Moving Pictures Experts Group's MPEG (Moving Pictures Experts Group, Moving Pictures Experts Group) series of standards.
[0055] H.264: H.264 / AVC (Advanced Video Coding) mainly supports 4:2:0 format continuous encoding or interlaced video encoding, and 4:2:2 and 4:4:4 can be used as reference information frame numbers. H.264 / AVC uses the classic motion compensation hybrid coding framework, with a total of three types of coded images: I frames (Intra coded frames, also known as key frames), P frames (Predicted frames, forward prediction coded frames) and B frames (Bi directional predicted frames, bidirectional prediction interpolation coded frames).
[0056] H.265 (a video compression coding standard): H.265 is a new video coding standard developed by ITU-T VCEG (Video Coding Experts Group) after H.264. The H.265 standard revolves around the existing video coding standard H.264, retains some of the original technologies, and improves some related technologies. The new technology uses advanced technology to improve the relationship between bit stream, encoding quality, delay and algorithm complexity to achieve optimal settings. Specific research contents include: improving compression efficiency, improving robustness and error recovery capabilities, reducing real-time delays, reducing channel acquisition time and random access delays, and reducing complexity. Compared with H.265, the data compression rate is increased by 40-50%.
[0057] I frame: I frame images use intra-frame coding, that is, only the spatial correlation within a single frame is used, but not the temporal correlation. I frame uses intra-frame compression and does not use motion compensation. Since I frame does not depend on other frames, it is a random access entry point and a reference frame for decoding. I frame is mainly used for receiver initialization and channel acquisition, as well as program switching and insertion. The compression multiple of I frame images is relatively low, and the data volume is larger than that of P frame. I frame images appear periodically in the image sequence, and the frequency of occurrence can be selected by the encoder.
[0058] P frame: P frame images use inter-frame coding, that is, they use both spatial and temporal correlations. P frame images only use forward temporal prediction, which can improve compression efficiency and image quality. P frame images can contain intra-frame coding parts, that is, each macroblock in a P frame can be forward predicted or intra-frame coded.
[0059] B frame: B frame images use bidirectional temporal prediction, which can greatly improve the compression ratio. It is worth noting that B frame images use future frames as references.
[0060] Frame rate: Frame rate is the frequency (rate) at which bitmap images in frames appear continuously on the display. Frame rate can also be called frame frequency and is expressed in Hertz (Hz). The frame rate per second indicates the number of times the graphics processor can update the field per second. A high frame rate can produce smoother and more realistic animations; the more frames per second (frame rate), the smoother the displayed action will be.
[0061] GOP (Group of Pictures): A GOP is a group of consecutive pictures. GOP is a set of pictures in a sequence to assist random access. The first picture of a GOP must be an I frame, so that the GOP can be decoded independently without reference to other pictures.
[0062] GOP unit: refers to a GOP that contains only one I frame.
[0063] Due to the characteristics of audio and video encoding, decoding is usually required during playback. The first frame decoding usually starts with the I frame. Therefore, when caching audio and video data, it is necessary to save the data volume of at least one GOP unit, and usually multiple GOP units. The data volume in multiple GOP units is large, and the instantaneous traffic will multiply several times during fast playback. In the IoT scenario, most live broadcasts are viewed for less than 20 seconds, and the playback devices belong to individuals, and the playback frequency is usually low. A large amount of directly cached data will lead to increased traffic each time the live broadcast is watched, thereby increasing the Internet bandwidth load and cost.
[0064] To facilitate understanding of the technical solutions of the embodiments of the present application, the following describes the related technologies of the embodiments of the present application. The following related technologies can be combined with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application.
[0065] Figure 1 The schematic diagram of an exemplary application scenario for implementing the method of the embodiment of the present application is shown in FIG. The technical solution provided in the embodiment of the present application can be applied to the following: Figure 1 The audio and video playback system shown in the figure may include an audio and video generation device 101, a server 102 and an audio and video playback device 103. The audio and video generation device 101 and the server 102 may communicate with each other through the Internet, and the audio and video playback device 103 and the server 102 may communicate with each other through the Internet. The audio and video generation device 101 may be used to collect audio and video data, encode the collected audio and video data, transmit the encoded audio and video data, receive and process the control signal sent by the server, etc. The audio and video playback device 103 may be used to send a playback request to the server, receive the audio and video data sent by the server, play the audio and video data, etc. The server 102 may be used to receive the audio and video data uploaded by the audio and video generation device, cache the audio and video data, send the audio and video data requested to be played to the audio and video playback device, send the control signal to the audio and video playback device, etc.
[0066] The audio and video generating device 101 may be a camera device or a mobile phone, computer or other terminal device with a camera function, and the audio and video playing device 103 may be a player or a mobile phone, computer or other terminal device with a playing function. The number of audio and video playing devices 103 may be multiple, and different audio and video playing devices 103 may be the same terminal device, such as all players, or different terminal devices, such as some audio and video playing devices 103 are players, and other audio and video playing devices 103 are mobile phones with a playing function.
[0067] Reference Figure 2, the embodiment of the present application provides an audio and video playback method 200, which can be applied to a server, and the method may include the following steps: S201, in response to a playback request from an audio and video playback device, determining whether the first duration of the cached data is greater than a set duration threshold; S202, when the first duration of the cached data is greater than the set duration threshold, sending a first control signal to the audio and video generation device; S203, obtaining real-time audio and video data fed back by the audio and video generation device based on the first control signal; S204, sending real-time audio and video data to the audio and video playback device. Among them, the first frame of the real-time audio and video data can be a key frame.
[0068] The above-mentioned play request may include a play request from one audio and video playback device, and may also include play requests from multiple audio and video playback devices. When the above-mentioned play request includes a play request from one audio and video playback device, the play request may be a play request corresponding to a single play; when the above-mentioned play request includes play requests from multiple audio and video playback devices, for each audio and video playback device, its play request may be a play request corresponding to a single play.
[0069] The cached data may be audio and video data pre-cached in the cache area, and the cached audio and video data may be audio and video data uploaded by the audio and video generating device, and the audio and video data may include at least one of audio data and video data. The cached data may include data of at least one GOP unit, and in one example, data of one GOP unit may be cached. The cache area may be a first-in-first-out data structure queue, so that the cache of the audio and video data uploaded by the audio and video generating device may be in a first-in-first-out form, for example, each time the data of a new GOP unit is received, the cached data of the old GOP unit is deleted, and the received data of the new GOP unit is cached in the cache area, that is, the data of the old GOP unit is replaced by the data of the new GOP unit in the cache area, so that the cache area can always cache the latest GOP unit data. In another example, if the first K frames (K can be any positive integer) of data received by the server are not key frames, the first K frames of data can be skipped, that is, the first K frames of data can be not cached until a key frame data is received, and the key frame data is cached. After storing the key frame data, the subsequently received non-key frames (P frames or B frames) can continue to be cached until the caching of all frame data in the current GOP unit (that is, the GOP unit to which the first received key frame belongs) is completed. When the next key frame is received, all cached audio and video data can be deleted, and caching can be started again from the next key frame.
[0070] The above-mentioned audio and video data may include at least one of audio data and video data. In one example, the cache area may include an audio cache area and a video cache area, which are used to cache audio data and video data respectively. In another example, the audio cache area and the video cache area may not be distinguished, and the audio data and video data may be cached in a unified cache area.
[0071] The above-mentioned first duration can be the duration that the cached data can be continuously played, for example, the total duration that each frame of data in the cached data can be played. Each frame of data in the cached data can carry timestamp information, and the first duration can be determined based on the timestamp information of each frame of data. The above-mentioned real-time audio and video data can include one frame of data or multiple frames of data. When including one frame of data, it can be key frame data. In this case, the first frame of the real-time audio and video data is the only frame, that is, the key frame. When including multiple frames of data, it can include data of one GOP unit, or it can include data of multiple GOP units. In this case, the first frame of the real-time audio and video data can be the first frame of the first GOP unit, that is, the key frame.
[0072] In the above-mentioned audio and video playback method 200, the playable duration of the cached data, i.e., the above-mentioned first duration, can be used to reflect the data volume of the cached audio and video data. When the first duration is greater than the set duration threshold, it can be considered that the first duration is longer and the corresponding cached data volume is larger. At this time, the audio and video generation device is controlled by the first control signal to generate and feedback new audio and video data, i.e., the above-mentioned real-time audio and video data, so that the new audio and video data can be fed back to the audio and video playback device, so that the audio and video playback device can play the new audio and video playback device without playing the cached data. While ensuring the fast playback of the live broadcast, the instantaneous traffic during a single playback can be reduced to avoid excessive traffic consumption, thereby reducing the load and cost of the Internet bandwidth.
[0073] In one implementation, the duration threshold may be a preset duration. The duration threshold may be a fixed duration threshold, and the specific value may be set according to actual needs, such as 2 seconds, 5 seconds or 10 seconds.
[0074] In another embodiment, the above-mentioned playback request may include information about the target playback duration, and the above-mentioned duration threshold may be the target playback duration. The target playback duration may reflect the playback requirements of the audio and video playback device, such as how long the data needs to be played in one playback. By using the target playback duration as the duration threshold, it is possible to determine the amount of cached data based on the playback requirements of the audio and video playback device. Relative to the fixed duration threshold, different audio and video playback devices may upload different playback requests, and different playback requests may include information about different target playback durations, so that the target playback duration may be flexible and adapted to the specific audio and video playback device. The amount of cached data determined based on the target playback duration and the playback strategy determined based on the amount of data may meet the playback requirements of different audio and video playback devices.
[0075] In one embodiment, the audio and video playback method 200 may further include: sending cached data to the audio and video playback device when the first duration is less than or equal to the duration threshold. Referring to the previous introduction, the first duration of the cached data can be used to characterize the size of the data volume of the cached data. When the first duration is less than or equal to the duration threshold, it can be considered that the first duration is small, and the corresponding data volume of the cached data is small. Sending the cached data to the audio and video playback device for playback will not cause excessive instantaneous traffic for a single playback. Therefore, in this case, the cached data is sent to the audio and video playback device first for fast playback to meet the speed requirements of live broadcast.
[0076] In one embodiment, in the above step S201, determining whether the first duration of the cached data is greater than the set duration threshold may include: determining the device type of the audio and video playback device, and when the device type is a low-frequency playback device, determining whether the first duration is greater than the duration threshold.
[0077] The specific method of determining the device type of the audio and video playback device may include: determining the device type of the audio and video playback device based on the receiving frequency of receiving the playback request of each audio and video playback device. In the interaction process between the server and each audio and video playback device, different audio and video playback devices may upload playback requests to the server in the same time period, so that the server receives multiple playback requests from different audio and video playback devices in the same time period, and then the receiving frequency of the server receiving the playback request in the time period, that is, the number of received playback requests per unit time, can be determined. Based on the receiving frequency, it can be determined whether the device type of each audio and video playback device connected to the server is a high-frequency playback device or a low-frequency playback device.
[0078] When determining the device type of the audio and video playback device based on the receiving frequency, the corresponding frequency threshold can be set according to actual needs, and the device type can be determined according to the size relationship between the receiving frequency and the frequency threshold. For example, a frequency threshold N can be set, and when the receiving frequency is greater than the frequency threshold N, it can be determined that the device types of the audio and video playback devices connected to the server are all high-frequency playback devices, and when the receiving frequency is less than or equal to the frequency threshold N, it can be determined that the device types of the audio and video playback devices connected to the server are low-frequency plug-in devices.
[0079] In one example, when there are a large number of audio and video playback devices connected to the current server, each audio and video playback device can send a playback request to the current server, so the current server receives a large number of playback requests, and thus the reception frequency is also high. At this time, it can be considered that each audio and video playback device connected to the current server is a high-frequency playback device. When there are a small number of audio and video playback devices connected to the current server, the current server receives a small number of playback requests, and thus the reception frequency is also low. At this time, it can be considered that each audio and video playback device connected to the current server is a low-frequency playback device.
[0080] In one example, after the device type is determined in the above manner, the device type information can be stored, and when the device type needs to be determined for the same audio and video playback device at a later time, it can be determined based on the stored device type information. In one example, the server can store the device identification of the connected audio and video playback device, and the device identification and device type of the same audio and video playback device can be stored accordingly, so that when the device type of the video playback device needs to be determined again, the corresponding device type can be quickly found based on the device identification.
[0081] In the case where the device type is a low-frequency playback device, due to the low playback frequency, even if the cached data needs to be abandoned and the regenerated real-time audio and video data needs to be played each time, the frequency of regenerating the real-time audio and video data will not be too high, and the data volume will not increase due to the high generation frequency. Therefore, the data volume of the cached data can be judged for the low-frequency playback device, and then the cached data can be avoided from being fed back to the audio and video playback device when the data volume of the cached data is large. In the case where the device type is a high-frequency playback device, due to the high playback frequency, if the cached data is replaced by the regenerated real-time audio and video data, the frequency of regenerating the real-time audio and video data will usually be high. The real-time audio and video data usually includes I frames, and the data volume of I frames is large. Therefore, replacing the cached data with the regenerated real-time audio and video data does not significantly reduce the data volume, but may increase the data volume. Therefore, the data volume of the cached data can be judged without judgment. Regardless of whether the data volume of the cached data is large or small, the cached data is sent to the audio and video playback device.
[0082] In the IoT scenario, most devices are low-frequency playback devices. The solution of the embodiment of the present application can be applied to most devices in the IoT scenario and has a good effect of reducing the amount of data sent. In one example, the encoding method of a low-frequency playback device is H265 encoding, the I frame interval is set to 10 seconds, the frame rate is 25fps (Frame PerSecond, the number of frames transmitted per second), the average size of the I frame is 100KB (kilobytes), and the average size of the P frame is 15KB. If it is played 100 times, the duration of each playback is 10 seconds, and the playback interval is 20 seconds. If the playback is played in the way of playing cached data, the average amount of data sent by the server to the audio and video playback device is about 5795KB. If the solution of the embodiment of the present application is used for playback, the average amount of data sent by the server to the audio and video playback device is about 3927KB, and the data volume can be reduced by 33.2%.
[0083] In one embodiment, the above-mentioned audio and video playback method 200 may also include: determining the device type of the audio and video playback device, and sending a second control signal to the audio and video generation device when the device type is a low-frequency playback device. The second control signal can be used to control the audio and video encoding device to perform a first encoding operation. The first encoding operation may include setting the key frame interval duration in the audio and video data generation process to a second duration, and the second duration may be greater than a preset interval threshold. Among them, the key frame interval duration may be the duration between the start times of two adjacent key frames. For example, for two adjacent key frames A and B, the duration between the start time of key frame A and the start time of key frame B is the key frame interval duration. The specific method for determining the device type of the audio and video playback device may refer to the previous introduction, which will not be repeated here.
[0084] The interval threshold can be set according to the overall situation of the playback frequency of each audio and video playback device in combination with actual needs. The second duration can be a duration that matches the receiving frequency of the server when receiving playback requests for low-frequency playback devices. The corresponding second duration can be determined based on the receiving frequency, or a larger value can be selected as the second duration in a value range greater than the interval threshold according to actual needs, such as 10 seconds or 15 seconds. The second control signal can carry information about the second duration.
[0085] The specific method of determining the device type of the audio and video playback device based on the receiving frequency can be referred to the previous introduction and will not be repeated here. When the audio and video playback device is a low-frequency playback device, the second control signal can be used to control the audio and video generation device to set the second duration in the encoding operation, that is, to set the key frame interval duration to a larger duration. This operation can reduce the generation frequency of key frames. The data volume of key frames is usually several times or even dozens of times larger than that of P frames or B frames. Reducing the generation frequency of key frames can avoid frequent generation of new key frames, thereby avoiding excessive overall data transmission.
[0086] In one implementation, the audio and video playback method 200 may further include: when the device type of the audio and video playback device is a high-frequency playback device, sending a third control signal to the audio and video generation device. The third control signal may be used to control the audio and video generation device to perform a second encoding operation, and the second encoding operation may include setting the key frame interval duration in the audio and video data generation process to a third duration, and the third duration may be less than or equal to the above interval threshold.
[0087] The third duration can be a duration that matches the receiving frequency of the server receiving the playback request for the high-frequency playback device. The corresponding third duration can be determined based on the receiving frequency, or a smaller value can be selected from a numerical range that is less than or equal to the interval threshold as the third duration according to actual needs. For example, it can be 2 seconds or 3 seconds. The third control signal can carry information about the third duration.
[0088] When the audio and video playback device is a high-frequency playback device, the third control signal can be used to control the audio and video generation device to set the third duration in the encoding operation, that is, to set the key frame interval duration to a smaller duration. This operation can increase the generation frequency of key frames to meet the needs of high-frequency playback.
[0089] When the device type of the audio and video playback device connected to the server changes dynamically, for example, the audio and video playback device connected in time period A is a low-frequency playback device, the audio and video playback device connected in time period B becomes a high-frequency device, and the audio and video playback device connected in time period C becomes a low-frequency device again, the control signal emitted by the server can change with the change of the device type, from the second control signal to the third control signal and then to the second control signal, thereby controlling the audio and video generation device to dynamically adjust the key frame interval duration.
[0090] In one embodiment, the above-mentioned audio and video playback method 200 may also include: updating the cached data to real-time audio and video data, the real-time audio and video data is updated in a first-in-first-out manner, the original cached data can be deleted, and the real-time audio and video data is stored in the cache area as new cached data. When a new audio and video playback device is connected to the server and sends a playback request, the new cached data can be sent to the new audio and video playback device to enable the newly connected audio and video playback device to achieve fast playback.
[0091] Based on the same technical concept, Figure 3 The present application also provides an audio and video playback method 300, which can be applied to Figure 1 The audio and video generating device shown in the figure may include the following steps: S301, generating real-time audio and video data in response to a first control signal from a server; S302, sending the real-time audio and video data to the server. The first control signal may be transmitted from the server to the audio and video generating device through any implementation of the audio and video playing method 200 described above.
[0092] Through the above-mentioned audio and video playback method 300, real-time audio and video data can be generated and fed back to the server when a first control signal sent by the server is received, so that the server can forward the real-time audio and video data to the audio and video playback device without sending cached data to the audio and video playback device. Therefore, when the amount of cached data is large, the amount of data for a single playback can be reduced by forwarding the real-time audio and video data.
[0093] In the above step S301, the specific method of generating real-time audio and video data can be: collecting real-time initial audio and video data, encoding the collected initial audio and video data, generating frame data such as I frames, P frames, B frames, etc., and using the encoded data as real-time audio and video data to be sent to the server. Collecting real-time initial audio and video data can include collecting at least one of real-time audio data and video data, for example, collecting audio signals in real time through a voice receiver in an audio and video generating device, and / or, shooting videos in real time through a camera in an audio and video generating device. In the above step S302, the operation of sending real-time audio and video data to the server can be achieved through the Internet.
[0094] In one embodiment, the above-mentioned audio and video playback method 300 may also include: in response to a second control signal from the server, performing a first encoding operation, the first encoding operation may include setting the key frame interval duration in the audio and video data generation process to a second duration, and the second duration may be greater than a preset interval threshold.
[0095] The second duration can be a duration that matches the receiving frequency of the server receiving the playback request for each low-frequency playback device. The corresponding second duration can be determined based on the receiving frequency, or a larger value (for example, greater than a set interval threshold, such as 5 seconds) can be set as the second duration according to actual needs, such as 10 seconds or 15 seconds. The second control signal can carry information about the second duration.
[0096] Under the control of the second control signal, the above-mentioned first encoding operation can set the key frame interval duration to a larger duration to reduce the generation frequency of key frames. The data volume of key frames is usually several times larger than the data volume of P frames or B frames. Reducing the generation frequency of key frames can avoid frequent generation of new key frames, thereby avoiding excessive overall data sending by the server.
[0097] In one embodiment, the above-mentioned audio and video playback method 300 may also include: performing a second encoding operation in response to a third control signal from the server, and the second encoding operation may include setting the key frame interval duration in the audio and video data generation process to a third duration, and the third duration may be less than or equal to the above-mentioned interval threshold.
[0098] The third duration can be a duration that matches the receiving frequency of the server receiving the playback request for each high-frequency playback device. The corresponding third duration can be determined based on the receiving frequency, or a smaller value (for example, less than a set interval threshold, such as 5 seconds) can be set as the third duration according to actual needs. For example, it can be 2 seconds or 3 seconds. The third control signal can carry information about the third duration.
[0099] Under the control of the third control signal, the second encoding operation can set the key frame interval duration to a shorter duration to increase the generation frequency of key frames, thereby meeting the demand for high-frequency playback.
[0100] When the control signal sent by the server changes dynamically, for example, changes from the second control signal to the third control signal and then to the second control signal, the audio and video generating device can dynamically adjust the key frame interval duration.
[0101] Based on the same technical concept, Figure 4 The present application also provides an audio and video playback method 400, which can be applied to Figure 1 Any audio and video playback device shown in the figure, the method may include the following steps: S401, sending a playback request to a server; S402, receiving real-time audio and video data fed back by the server in response to the playback request; S403, playing the audio and video data. The real-time audio and video data can be transmitted from the server to the audio and video playback device through any implementation of the audio and video playback method 400 described above.
[0102] Through the above-mentioned audio and video playback method 400, the audio and video playback device can obtain and play real-time audio and video data without having to play the cached data with a large amount of data in the server, thereby reducing the amount of data for a single playback, effectively reducing the traffic for a single playback, and reducing the load and cost of the Internet bandwidth.
[0103] In one implementation, when the server determines that the amount of cached data is small and feeds back the cached data, the audio and video playback device may receive and play the cached data.
[0104] The audio and video playback device receives real-time audio and video data or cached data, which can be achieved through the Internet. The specific way in which the audio and video playback device plays the real-time audio and video data or cached data may include: decoding the received real-time audio and video data or cached data, playing the decoded data, and rendering the decoded data before playing, and playing the decoded and rendered data.
[0105] Exemplarily, based on some implementations of the above-mentioned audio and video playback method 200, audio and video playback method 300 and audio and video playback method 400, Figure 5 The figure shows a schematic diagram of the interaction principle of a server, an audio and video generating device and an audio and video playing device. Figure 5 The interaction between the server, the audio and video generating device and the audio and video playing device may include the following steps S501 to S510:
[0106] S501, the audio and video playback device sends a playback request to the server;
[0107] S502, in response to the play request, the server determines whether the audio and video playback device is a low-frequency playback device, if so, executes step S503, if not, executes step S509;
[0108] S503, determining whether the first duration of the cached data is greater than a preset duration threshold, if so, executing step S504, if not, executing step S509;
[0109] S504, the server sends a first control signal to the audio and video generating device;
[0110] S505, in response to the first control signal, the audio and video generating device generates real-time audio and video data;
[0111] S506, the audio and video device feeds back real-time audio and video data to the server;
[0112] S507, the server forwards the real-time audio and video data to the audio and video playback device;
[0113] S508, the audio and video playback device plays the received real-time audio and video data, and then ends the current playback process;
[0114] S509, the server sends the cached data to the audio and video playback device;
[0115] S510: The audio and video playback device plays the received buffered data, and then ends the current playback process.
[0116] For example, Figure 6 Another schematic diagram of the interaction principle of a server, an audio and video generating device, and an audio and video playing device is shown based on some implementations of the above-mentioned audio and video playing method 200, audio and video playing method 300, and audio and video playing method 400. Figure 6 The interaction between the server, the audio and video generating device and the audio and video playing device may include the following steps S601 to S607:
[0117] S601, the audio and video playback device sends a playback request to the server;
[0118] S602, the server receives a play request and determines a frequency of receiving the play request;
[0119] S603, the server determines the device type of the audio and video playback device according to the receiving frequency, and if the device type is a low-frequency playback device, executes step S604, and if the device type is a high-frequency playback device, executes step S606;
[0120] S604, the server sends a second control signal to the audio and video generating device;
[0121] S605, the audio and video generating device performs a first encoding operation, and then ends the current process;
[0122] S606, the server sends a third control signal to the audio and video generating device;
[0123] S607: The audio and video generating device performs a second encoding operation, and then ends the current process.
[0124] Figure 5 The interactive steps shown and Figure 6 The interactive steps shown can be performed synchronously. Figure 5 The interactive steps shown and Figure 6 The specific principles of each interaction step shown can refer to the previous related introduction.
[0125] Based on the same technical concept, Figure 7 The embodiment of the present application also provides an audio and video playback device 700, which can be applied to a server. The device may include: an analysis module 701, a control module 702, an acquisition module 703 and a sending module 703.
[0126] The analysis module 701 is used to determine whether the first duration of the cached data is greater than the set duration threshold in response to a playback request from the audio and video playback device; the control module 702 is used to send a first control signal to the audio and video generation device when the first duration of the cached data is greater than the duration threshold; the acquisition module 703 is used to obtain the real-time audio and video data fed back by the audio and video generation device based on the first control signal; the sending module 704 is used to send the real-time audio and video data to the audio and video playback device. The first frame of the real-time audio and video data is a key frame.
[0127] In one implementation, the play request may include information about a target play duration, and the duration threshold may be the target play duration.
[0128] In one implementation, the sending module 704 may also be configured to: send the cached data to the audio and video playback device when the first duration of the cached data is less than or equal to a duration threshold.
[0129] In one implementation, the analysis module 701 may be specifically configured to: determine the device type of the audio and video playback device, and when the device type is a low-frequency playback device, determine whether the first duration is greater than a duration threshold.
[0130] In one implementation, the analysis module 701 may also be used to determine the device type of the audio and video playback device. The control module 702 may also be used to: when the device type is a low-frequency playback device, send a second control signal to the audio and video generation device. The second control signal is used to control the audio and video generation device to perform a first encoding operation, and the first encoding operation includes setting the key frame interval duration in the audio and video data generation process to a second duration, and the second duration may be greater than a preset interval threshold.
[0131] In one implementation, the control module 702 may also be used to: when the device type is a high-frequency playback device, send a third control signal to the audio and video generation device. The third control signal is used to control the audio and video generation device to perform a second encoding operation, and the second encoding operation includes setting the key frame interval duration in the audio and video data generation process to a third duration, and the third duration may be less than the interval threshold.
[0132] In one embodiment, the audio and video playback device 700 may further include: a cache module, which may be used to update the cache data into real-time audio data. The real-time audio and video data may be updated in a first-in-first-out manner.
[0133] Based on the same technical concept, Figure 8 The embodiment of the present application also provides an audio and video playback device 800, which can be applied to an audio and video generation device. The device may include: a generation module 801 and a sending module 802.
[0134] The generating module 801 can be used to receive a first control signal from a server and generate real-time audio and video data in response to the first control signal from the server. The sending module 802 can be used to send real-time audio and video data to the server. The first control signal can be transmitted from the server to the audio and video generating device through the above-mentioned audio and video playing method 200.
[0135] In one embodiment, the generation module 801 may also be used to receive at least one of a second control signal and a third control signal from a server. Upon receiving the second control signal from the server, the processing module 801 may perform a first encoding operation in response to the second control signal; upon receiving the third control signal from the server, the processing module 801 may perform a second encoding operation in response to the third control signal. The first encoding operation may include setting the key frame interval duration in the audio and video data generation process to a second duration, and the second duration may be greater than a preset interval threshold; the second encoding operation may include setting the key frame interval duration in the audio and video data generation process to a third duration, and the third duration may be less than a preset interval threshold.
[0136] In one embodiment, the above-mentioned generation module 801 may include a receiving unit, a collection unit and an encoding unit. The receiving unit can be used to receive control signals transmitted by the server, such as a first control signal, a second control signal and a third control signal. The collection unit can be used to collect real-time initial audio and video data. The encoding unit can be used to encode the collected initial audio and video data to generate frame data such as I frames, P frames, and B frames.
[0137] Based on the same technical concept, Fig. 9 The embodiment of the present application also provides an audio and video playback device 900, which can be applied to an audio and video playback device. The device may include: a request module 901, a receiving module 902 and a playback module 903.
[0138] The request module 901 can be used to send a play request to the server; the receiving module 902 can be used to receive the real-time audio and video data fed back by the server in response to the play request; the playing module 903 can be used to play the real-time audio and video data. The real-time audio and video data can be transmitted from the server to the audio and video playing device through the above-mentioned audio and video playing method 300.
[0139] In one implementation, the receiving module 902 may also receive cached data fed back by the server, and the playing module 903 may also play the cached data.
[0140] The playback module 903 may include a decoding unit, which may be used to decode the received real-time audio and video data or cached data, and the played data may be the decoded data. The playback module 903 may also include a rendering unit, which may be used to render the decoded data before playback, and the played data may be the decoded and rendered data.
[0141] Exemplarily, based on some implementations of the above-mentioned audio and video playback method 200, audio and video playback method 300 and audio and video playback method 400, Fig.10 A schematic diagram of data flow between the audio and video playback device 700, the audio and video playback device 800 and the audio and video playback device 900 is shown. Fig.10 A data flow direction between the audio and video playback device 700, the audio and video playback device 800 and the audio and video playback device 900 may be: the generation module 801 sends the generated audio and video data to the sending module 802, the sending module 802 sends the received audio and video data to the acquisition module 703, the acquisition module 703 sends the received audio and video data to the cache module 705, the cache module 705 stores the audio and video data in the cache area to form cache data, when playback is required, the cache module 705 sends the cache data to the sending module 704, the sending module 704 forwards the cache data to the receiving module 902, the receiving module 902 sends the received cache data to the playback module 903, and the playback module 903 plays the cache data.
[0142] The functions of each module in each device in the embodiments of the present application can be found in the corresponding description in the above method, and have corresponding beneficial effects, which will not be repeated here.
[0143] Based on the same technical concept, Fig.11 The embodiment of the present application further provides an electronic device that can be used as a server. The device may include: a memory 1101 and a processor 1102. The memory 1101 stores a computer program that can be run on the processor 1102. When the processor 1102 executes the computer program, the above-mentioned audio and video playback method 200 can be implemented. The number of the memory 1101 and the processor 1102 can be one or more.
[0144] The electronic device also includes:
[0145] The communication interface 1103 is used to communicate with external devices and perform data exchange transmission.
[0146] If the memory 1101, the processor 1102 and the communication interface 1103 are implemented independently, the memory 1101, the processor 1102 and the communication interface 1103 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.11 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0147] Optionally, in a specific implementation, if the memory 1101, the processor 1102 and the communication interface 1103 are integrated on a chip, the memory 1101, the processor 1102 and the communication interface 1103 can communicate with each other through an internal interface.
[0148] Based on the same technical concept, Fig.12 The embodiment of the present application further provides an electronic device that can be used as an audio and video generating device. The device may include: a memory 1201 and a processor 1202. The memory 1201 stores a computer program that can be run on the processor 1202. When the processor 1202 executes the computer program, the above-mentioned audio and video playing method 300 is implemented. The number of the memory 1201 and the processor 1202 may be one or more.
[0149] The electronic device also includes:
[0150] The communication interface 1203 is used to communicate with external devices and perform data exchange transmission.
[0151] If the memory 1201, the processor 1202 and the communication interface 1203 are implemented independently, the memory 1201, the processor 1202 and the communication interface 1203 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.12 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0152] Optionally, in a specific implementation, if the memory 1201, the processor 1202 and the communication interface 1203 are integrated on a chip, the memory 1201, the processor 1202 and the communication interface 1203 can communicate with each other through an internal interface.
[0153] Based on the same technical concept, Fig.13 The embodiment of the present application also provides an electronic device that can be used as an audio and video playback device. The device may include: a memory 1301 and a processor 1302. The memory 1301 stores a computer program that can be run on the processor 1302. When the processor 1302 executes the computer program, the method in the above embodiment is implemented. The number of the memory 1301 and the processor 1302 can be one or more.
[0154] The electronic device also includes:
[0155] The communication interface 1303 is used to communicate with external devices and perform data exchange transmission.
[0156] If the memory 1301, the processor 1302 and the communication interface 1303 are implemented independently, the memory 1301, the processor 1302 and the communication interface 1303 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.13 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0157] Optionally, in a specific implementation, if the memory 1301, the processor 1302 and the communication interface 1303 are integrated on a chip, the memory 1301, the processor 1302 and the communication interface 1303 can communicate with each other through an internal interface.
[0158] An embodiment of the present application provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements any one of the audio and video playback methods provided in the embodiments of the present application.
[0159] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in the memory from the memory, so that a communication device equipped with the chip executes any audio and video playback method provided in the embodiment of the present application.
[0160] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory. The input interface, the output interface, the processor and the memory are connected through an internal connection path. The processor is used to execute the code in the memory. When the code is executed, the processor is used to execute any audio and video playback method provided in the embodiment of the application.
[0161] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the Advanced RISC Machines (ARM) architecture.
[0162] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of exemplary but not limiting description, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM).
[0163] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.
[0164] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.
[0165] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
[0166] Any process or method described in the flow chart or otherwise described herein can be understood as a module, fragment or portion of a code representing one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.
[0167] The logic and / or steps described in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or used in combination with these instruction execution systems, devices or apparatuses.
[0168] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.
[0169] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.
[0170] The above is only an exemplary embodiment of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope recorded in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.
Claims
1. An audio and video playback method, characterized in that: Applied to a server, the method comprises: In response to a playback request from an audio or video playback device, determining whether a first duration of the cached data is greater than a set duration threshold, the first duration being a duration during which the cached data can be continuously played; When the first duration is greater than the duration threshold, sending a first control signal to the audio and video generating device; Acquire real-time audio and video data fed back by the audio and video generating device based on the first control signal; Sending the real-time audio and video data to the audio and video playback device; the first frame of the real-time audio and video data is a key frame; When the first duration is less than or equal to the duration threshold, the cached data is sent to the audio and video playback device.
2. The audio and video playback method according to claim 1, characterized in that: The playback request includes information about the target playback duration; The duration threshold is the target playback duration.
3. The audio and video playback method according to claim 1 or 2, characterized in that: The determining whether the first duration of the cached data is greater than a set duration threshold includes: Determine the device type of the audio and video playback device; When the device type is a low-frequency playback device, it is determined whether the first duration is greater than the duration threshold.
4. The audio and video playback method according to claim 1 or 2, characterized in that: Also includes: Determine the device type of the audio and video playback device; When the device type is a low-frequency playback device, sending a second control signal to the audio and video generation device; The second control signal is used to control the audio and video generating device to perform a first encoding operation; the first encoding operation includes setting the key frame interval duration in the audio and video data generation process to a second duration, and the second duration is greater than a preset interval threshold.
5. The audio and video playback method according to claim 4, characterized in that: Also includes: When the device type is a high-frequency playback device, sending a third control signal to the audio and video generation device; The third control signal is used to control the audio and video generating device to perform a second encoding operation; the second encoding operation includes setting the key frame interval duration in the audio and video data generation process to a third duration, and the third duration is less than the interval threshold.
6. The audio and video playing method according to claim 1 or 2, characterized in that: Also includes: Updating the cached data to the real-time audio and video data; The real-time audio and video data is updated in a first-in-first-out manner.
7. An audio and video playback method, characterized in that: Applied to an audio and video generating device, the method comprises: In response to a first control signal from a server, real-time audio and video data is generated; the first control signal is transmitted by the server to the audio and video generating device through the audio and video playing method according to any one of claims 1 to 6; Sending the real-time audio and video data to the server; When the first duration is less than or equal to the duration threshold, the cached data is sent to the server.
8. The audio and video playback method according to claim 7, characterized in that: Also includes: In response to a second control signal from the server, a first encoding operation is performed; the first encoding operation includes setting a key frame interval duration in the audio and video data generation process to a second duration, and the second duration is greater than a preset interval threshold.
9. The audio and video playing method according to claim 7 or 8, characterized in that: Also includes: In response to a third control signal from the server, a second encoding operation is performed; the second encoding operation includes setting the key frame interval duration in the audio and video data generation process to a third duration, and the third duration is less than a preset interval threshold.
10. An audio and video playback method, characterized in that: Applied to an audio and video playback device, the method comprises: Send a play request to the server; Receiving real-time audio and video data fed back by the server in response to the playback request; the real-time audio and video data is transmitted by the server to the audio and video playback device through the audio and video playback method according to any one of claims 1 to 6; Play the real-time audio and video data.
11. An audio and video playback device, characterized in that: Applied to a server, the device comprises: An analysis module, configured to determine, in response to a playback request from an audio or video playback device, whether a first duration of the cached data is greater than a set duration threshold, wherein the first duration is a duration during which the cached data can be continuously played; A control module, configured to send a first control signal to an audio and video generating device when the first duration is greater than the duration threshold; An acquisition module, used to acquire real-time audio and video data fed back by the audio and video generating device based on the first control signal; A sending module is used to send the real-time audio and video data to the audio and video playback device; the first frame of the real-time audio and video data is a key frame, and, when the first duration is less than or equal to the duration threshold, the cached data is sent to the audio and video playback device.
12. An audio and video playback device, characterized in that: Applied to audio and video generation equipment, the device comprises: A generating module, configured to generate real-time audio and video data in response to a first control signal from a server; the first control signal is transmitted by the server to the audio and video generating device through the audio and video playing method according to any one of claims 1 to 6; A sending module is used to send the real-time audio and video data to the server, and, when the first duration is less than or equal to the duration threshold, send cached data to the audio and video playback device.
13. An audio and video playback device, characterized in that: Applied to audio and video playback equipment, the device comprises: A request module, used to send a playback request to the server; A receiving module, used to receive real-time audio and video data fed back by the server in response to the playback request; the real-time audio and video data is transmitted by the server to the audio and video playback device through the audio and video playback method according to any one of claims 1 to 6; The playing module is used to play the real-time audio and video data.
14. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory, wherein the processor implements the audio and video playback method according to any one of claims 1 to 6 when executing the computer program.
15. An electronic device, characterized in that: include: A memory, a processor and a computer program stored in the memory, wherein the processor implements the audio and video playback method according to any one of claims 7 to 9 when executing the computer program.
16. An electronic device, characterized in that: include: A memory, a processor and a computer program stored in the memory, wherein the processor implements the audio and video playback method according to claim 10 when executing the computer program.
17. An audio and video playback system, characterized in that: include: Servers, audio and video generation equipment, and audio and video playback equipment; The server is the electronic device according to claim 14, the audio and video generating device is the electronic device according to claim 16, and the audio and video playing device is the electronic device according to claim 16; The audio and video generating device communicates with the server via the Internet, and the server communicates with the audio and video playing device via the Internet.
18. A computer-readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the audio and video playback method according to any one of claims 1 to 10 is implemented.
