Client coding and decoding method, device and equipment based on video playing system

Abstract

The embodiment of the invention provides a client coding and decoding method, device and equipment based on a video playing system, and is applied to the technical field of video processing. The method comprises the following steps: a client obtains a video stream of an original code, wherein the video stream is a video file which only generates one code rate in a whole video playing system; decoding the video stream to obtain a decoded video stream; according to a preset filter processing rule, carrying out filter processing on an image of the decoded video stream so as to carry out fuzzification processing on the video content; and displaying the processed video stream. In this way, transcoding and storage resources can be saved, transcoding and storage costs are reduced, and the playing success rate and the P2P sharing rate are also improved.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing technology, in particular to the field of video processing technology, and specifically to a client encoding and decoding method, apparatus, and device based on a video playback system. Background Art

[0002] Generally speaking, factors that determine video clarity include resolution, bitrate, frame rate, encoding format and compression algorithm, color depth, and bit depth. The video playback process on the client includes video file loading, protocol parsing, demultiplexing, decoding, audio and video synchronization (A / V Sync), rendering, and display.

[0003] The video playback system includes multiple steps, including production, uploading, transcoding, storage, and playback. To accommodate a variety of terminals, transcoding generates video content with multiple bitrates. Both central storage and CDN nodes must store the corresponding video files, placing a burden on transcoding and storage resources. Furthermore, transcoding delays the online delivery of video content, hindering its real-time performance. Furthermore, the multitude of video bitrates creates operational difficulties. When video files with certain bitrates are lost, video playback fails, impacting the user experience. Summary of the Invention

[0004] The present disclosure provides a client encoding and decoding method, apparatus, device, and storage medium based on a video playback system.

[0005] According to a first aspect of the present disclosure, a client encoding and decoding method based on a video playback system is provided, which is applied to a client. The method includes:

[0006] Obtaining an originally encoded video stream, wherein the video stream is a video file that generates only one bit rate in the entire video playback system;

[0007] Decoding the video stream to obtain a decoded video stream;

[0008] Perform filter processing on the decoded video stream image according to the preset filter processing rules;

[0009] Display the processed video stream.

[0010] According to the above aspects and any possible implementation, an implementation is further provided, wherein the filtering of the image of the decoded video stream according to a preset filter processing rule includes:

[0011] Determining filter parameters according to a preset display definition, wherein the filter parameters include the number of filter grids;

[0012] Dividing the decoded video stream image according to the number of filter grids, and determining pixels of each filter grid;

[0013] Apply filter processing to the pixels of each filter grid.

[0014] According to the above aspect and any possible implementation, an implementation is further provided, wherein the filtering of the pixels of each filter grid includes:

[0015] Based on the preset filter algorithm, the pixels of each filter grid are filtered.

[0016] According to the above aspects and any possible implementation, an implementation is further provided, wherein the preset filter algorithm includes a nearest neighbor interpolation algorithm, a bilinear interpolation algorithm, or a bicubic interpolation algorithm.

[0017] According to the above aspects and any possible implementation manner, further provided is an implementation manner, wherein the filter parameter further includes a frame extraction number; and before dividing the images of the decoded video stream according to the filter grid number, the method further includes:

[0018] According to the frame extraction number, frame extraction processing is performed on the images of the decoded video stream.

[0019] According to the above aspects and any possible implementation, an implementation is further provided, wherein the definition of the processed video stream increases as the number of filter grids increases until it is consistent with the definition of the decoded video stream.

[0020] As described above and any possible implementation, a further implementation is provided, in which the central storage and edge CDN nodes in the video playback system, and the client use the same video file.

[0021] According to a second aspect of the present disclosure, a client encoding and decoding device based on a video playback system is provided. The device includes:

[0022] An acquisition module is used to acquire an originally encoded video stream, wherein the video stream is a video file that generates only one bit rate in the entire video playback system;

[0023] A decoding module, configured to decode the video stream to obtain a decoded video stream;

[0024] The filter module is used to perform filter processing on the image of the decoded video stream according to preset filter processing rules;

[0025] The display module is used to display the processed video stream.

[0026] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above method when executing the program.

[0027] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method described above is implemented.

[0028] The embodiment of the present application provides a client encoding and decoding method, device, equipment and storage medium based on a video playback system, which is applied to the client and can obtain the original encoded video stream, which is a video file that only generates one bit rate in the entire video playback system; then decode the video stream to obtain a decoded video stream; then filter the image of the decoded video stream according to the preset filter processing rules to blur the video content; display the processed video stream; based on this, a new encoding and decoding scheme can be used to solve the redundancy problem of multiple links in the video playback system. Specifically, when the client plays the video content, no matter what bit rate is played, the content downloaded from the central storage or the CDN edge node is the same, and the adjustment of different bit rates of the client is achieved through the new encoding and decoding scheme. Since each video content corresponds to only one video file, transcoding and storage resources are saved, the cost of transcoding and storage is reduced, and the playback success rate and P2P sharing rate are also improved.

[0029] It should be understood that the contents described in the Summary of the Invention section are not intended to limit the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. The accompanying drawings are provided for a better understanding of the present disclosure and do not constitute a limitation of the present disclosure. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, among which:

[0031] Figure 1 A schematic diagram illustrating an exemplary operating environment in which embodiments of the present disclosure can be implemented is shown;

[0032] Figure 2 A flowchart of a client encoding and decoding method based on a video playback system according to an embodiment of the present disclosure is shown;

[0033] Figure 3 A schematic diagram of a filter principle according to an embodiment of the present disclosure is shown;

[0034] Figure 4 A block diagram of a client encoding and decoding device based on a video playback system according to an embodiment of the present disclosure is shown;

[0035] Figure 5 A block diagram of an exemplary electronic device capable of implementing embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0036] To make the purpose, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of the present disclosure.

[0037] In this document, the term "and / or" simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the related objects are in an "or" relationship.

[0038] In this disclosure, a new codec solution can be used to address redundancy issues in multiple links of the video playback system. Specifically, when a client plays video content, regardless of the bitrate, the content downloaded from the central storage or CDN edge node is the same. This new codec solution enables the client to adjust different bitrates. Because each video content corresponds to only one video file, transcoding and storage resources are saved, transcoding and storage costs are reduced, and playback success rate and P2P sharing rate are improved.

[0039] Figure 1 FIG. 1 is a schematic diagram of an exemplary operating environment 100 in which embodiments of the present disclosure can be implemented. The operating environment 100 includes a client 102 and a video playback system 104 .

[0040] In some embodiments, the video playback system 104 includes central storage and edge CDN nodes, which are used to store video files that only generate one bit rate in the entire video playback system, so as to abandon the previous method of transcoding video files with multiple bit rates by the video playback system, thereby saving transcoding resources.

[0041] In some embodiments, client 102 includes file loading parsing, demultiplexing, decoding, rendering, filtering, and display.

[0042] In some embodiments, the client 102 and the central storage and edge CDN nodes in the video playback system 104 use the same video file, so as to abandon the previous video playback system's need to save video files of multiple bit rates. Only one video file is stored for each video content, thereby saving storage resources.

[0043] In some embodiments, the success rate and efficiency of P2P can be greatly improved by using the same video file in both the client 102 and the central storage and edge CDN nodes in the video playback system 104. Furthermore, since the central storage and edge CDN nodes in the video playback system 104 use the same video file, playback failures caused by the loss of the corresponding bitrate file can be avoided.

[0044] Figure 2 FIG. 2 shows a flow chart of a client encoding and decoding method 200 based on a video playback system according to an embodiment of the present disclosure. Figure 1 Executed by client 102 in .

[0045] In block 210 , an originally encoded video stream is obtained. The video stream is a video file that generates only one bit rate in the entire video playback system.

[0046] In some embodiments, the original encoded video stream generates only one bit rate video file in the entire video playback system, so that when the client plays the video content, no matter what bit rate is played, the content downloaded from the central storage or edge CDN node is the same, and the client can adjust different bit rates through the new encoding and decoding scheme.

[0047] In some embodiments, the central storage, edge CDN nodes, and the client in the video playback system use the same video file. This saves transcoding and storage resources, reduces transcoding and storage costs, and improves playback success rates and P2P sharing rates, as each video content corresponds to a unique video file.

[0048] In block 220 , the video stream is decoded to obtain a decoded video stream.

[0049] In some embodiments, the original encoded video stream and the acquisition and decoding of the video stream are all controlled in the client's playback software, that is, the encoding and decoding method of the video stream does not change during the playback process, so that only a filter is added in reality later, similar to frosted glass. This filter can be set with thousands or even tens of thousands of meshes to facilitate the control of resolution when displaying video content.

[0050] In block 230 , filter processing is performed on the image of the decoded video stream according to a preset filter processing rule.

[0051] In some embodiments, the preset filter processing rules can be set according to the actual needs of the user.

[0052] like Figure 3 As shown, the purpose of performing filter processing on the decoded video stream image according to the preset filter processing rule is to process the video stream image from high resolution to low resolution, that is, to blur the video content.

[0053] In some embodiments, the filtering of the decoded video stream image according to the preset filtering rules specifically includes:

[0054] Determine filter parameters according to the preset display definition, the filter parameters including the number of filter grids;

[0055] Dividing the decoded video stream image according to the number of filter grids and determining the pixels of each filter grid;

[0056] Apply filter processing to the pixels of each filter grid.

[0057] In some embodiments, the filtering process performed on the pixels of each filter grid specifically includes:

[0058] Based on the preset filter algorithm, the pixels of each filter grid are filtered.

[0059] In some embodiments, the definition of the processed video stream increases as the number of filter grids increases until it is consistent with the definition of the decoded video stream.

[0060] In some embodiments, the image of the decoded video stream is filtered according to a preset filter processing rule, aiming to reduce the number of pixels in the video stream.

[0061] In some embodiments, before actually discarding pixels, the original high-resolution image is first low-pass filtered, i.e., blurred. This filter is typically a Gaussian blur or similar algorithm, which attenuates or removes details in the image that are above the highest spatial frequency that can be represented by the new resolution, i.e., the finest parts.

[0062] For example, a high-resolution (1920×1080) HD image contains approximately 2.07 million pixels, each of which stores the color and brightness information of that location. More pixels mean that finer details, sharper edges, and smoother gradients can be recorded. A low-resolution (1280×720) image contains approximately 920,000 pixels, which is only about 44% of the number of high-resolution pixels. When the information of 2.07 million pixels needs to be crammed into a frame of only 920,000 pixels, a large amount of original information is inevitably discarded, making it impossible to retain the pixel information representing fine textures, sharp edges, and small objects in a one-to-one correspondence in the low-resolution grid. To display a high-definition image with a resolution of 1920*1080 as an image with a resolution of 1280×720, that is, each image displays 1280×720 pixels, it is necessary to first divide it into a 1280×720 grid on the filter, and then divide the high-definition image into a 1280×720 grid. Then, the pixels in the grids in the high-definition image are merged, optimized, and rendered, and finally displayed as a resolution of 1280×720.

[0063] In some embodiments, the filter parameters further include a frame extraction number; and before dividing the images of the decoded video stream according to the filter grid number, the method further includes:

[0064] According to the number of frames extracted, the images of the decoded video stream are subjected to frame extraction processing.

[0065] For example, a high-definition image with a resolution of 1920*1080 and a frame rate of 30 needs to be displayed as an image with a resolution of 1280×720, that is, each image displays 1280×720 pixels. It is first divided into 1280×720 grids on the filter, and then the high-definition image is also divided into 1280×720 grids. Then, the pixels in the grids in the high-definition image are merged, optimized, and rendered. Finally, combined with the frame extraction process, 25 frames are extracted from 30 frames, and finally it is displayed as an image with a resolution of 1280×720 and a frame rate of 25, which is further blurred.

[0066] In some embodiments, the preset filter algorithm may include a nearest neighbor interpolation algorithm, a bilinear interpolation algorithm, or a bicubic interpolation algorithm.

[0067] Specifically, using the nearest neighbor interpolation algorithm, we can simply take the value of the nearest single pixel in the original image for each target pixel on the low-resolution grid. Using bilinear interpolation, we can take the weighted average of the 2×2 neighboring pixels in the original image for the target pixel. Using bicubic interpolation, we can take the weighted average of the 4×4 neighboring pixels in the original image for the target pixel.

[0068] At block 240, the processed video stream is displayed.

[0069] In some embodiments, in response to the clarity selected by the user, the lower the clarity selected by the user, the fewer grids of the filter and the larger the meshes; conversely, the higher the clarity selected by the user, the more grids of the filter and the smaller the meshes, until it becomes consistent with the clarity of the video content, that is, showing full clarity.

[0070] According to the embodiments of the present disclosure, the following technical effects are achieved:

[0071] The above-mentioned client encoding and decoding method based on the video playback system is applied to the client, that is, the transcoding of the server is placed on the client terminal, which can obtain the original encoded video stream, which is a video file that only generates one bit rate in the entire video playback system; then decode the video stream to obtain a decoded video stream; then filter the image of the decoded video stream according to the preset filter processing rules to blur the video content; display the processed video stream; based on this, a new encoding and decoding scheme can be used to solve the redundancy problem of multiple links in the video playback system. Specifically, when the client plays the video content, no matter what bit rate is played, the content downloaded from the central storage or CDN edge node is the same, and the new encoding and decoding scheme is used to achieve the adjustment of different bit rates of the client. Since each video content corresponds to only one video file, transcoding and storage resources are saved, the cost of transcoding and storage is reduced, and the playback success rate and P2P sharing rate are also improved.

[0072] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present disclosure is not limited by the order of the actions described, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present disclosure.

[0073] The above is an introduction to the method embodiment. The following is a further explanation of the solution disclosed in the present disclosure through an apparatus embodiment.

[0074] Figure 4 FIG. 4 shows a block diagram of a client encoding and decoding device 400 based on a video playback system according to an embodiment of the present disclosure. The device 400 may be included in Figure 1 The client 102 of the embodiment of the present invention is implemented as the client 102. Figure 4 As shown, the apparatus 400 includes:

[0075] The acquisition module 410 is used to obtain the original encoded video stream, which is a video file with only one bit rate in the entire video playback system;

[0076] The decoding module 420 is used to decode the video stream to obtain a decoded video stream;

[0077] The filter module 430 is used to perform filter processing on the image of the decoded video stream according to preset filter processing rules;

[0078] The display module 440 is configured to display the processed video stream.

[0079] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the described module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0080] In the technical solutions disclosed herein, the acquisition, storage, and application of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0081] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0082] Figure 5 A block diagram of an exemplary electronic device 500 capable of implementing embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0083] The electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a ROM 502 or a computer program loaded from a storage unit 508 into a RAM 503. The RAM 503 may also store various programs and data required for the operation of the electronic device 500. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An I / O interface 505 is also connected to the bus 504.

[0084] Multiple components in the electronic device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a magnetic disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the electronic device 500 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0085] The computing unit 501 may be a variety of general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as method 200. For example, in some embodiments, method 200 may be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as storage unit 508.

[0086] In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the method 200 described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the method 200 in any other appropriate manner (e.g., by means of firmware).

[0087] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system comprising at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0088] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0089] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0090] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0091] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0092] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0093] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0094] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A client encoding and decoding method based on a video playback system, characterized in that: Applied to the client, including: Obtaining an originally encoded video stream, wherein the video stream is a video file that generates only one bit rate in the entire video playback system; Decoding the video stream to obtain a decoded video stream; According to the preset filter processing rules, the image of the decoded video stream is filtered; Display the processed video stream.

2. The method according to claim 1, characterized in that The filtering of the decoded video stream image according to the preset filter processing rules includes: Determining filter parameters according to a preset display definition, wherein the filter parameters include the number of filter grids; Dividing the decoded video stream image according to the number of filter grids, and determining pixels of each filter grid; Apply filter processing to the pixels of each filter grid.

3. The method according to claim 2, characterized in that The filtering of the pixels of each filter grid includes: Based on the preset filter algorithm, the pixels of each filter grid are filtered.

4. The method according to claim 3, characterized in that The preset filter algorithm includes a nearest neighbor interpolation algorithm, a bilinear interpolation algorithm or a bicubic interpolation algorithm.

5. The method according to claim 2, characterized in that The filter parameters further include a frame extraction number; and before dividing the image of the decoded video stream according to the filter grid number, the method further includes: According to the frame extraction number, frame extraction processing is performed on the images of the decoded video stream.

6. The method according to claim 2, characterized in that The definition of the processed video stream increases as the number of filter grids increases, until it is consistent with the definition of the decoded video stream.

7. The method according to any one of claims 2 to 6, characterized in that The central storage and edge CDN nodes in the video playback system, as well as the client, use the same video file.

8. A client encoding and decoding device based on a video playback system, characterized in that: Applied to the client, including: An acquisition module is used to acquire an originally encoded video stream, wherein the video stream is a video file that generates only one bit rate in the entire video playback system; A decoding module, configured to decode the video stream to obtain a decoded video stream; The filter module is used to perform filter processing on the image of the decoded video stream according to preset filter processing rules; The display module is used to display the processed video stream.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

Images