Block effect detection method, apparatus, device, and storage medium

Abstract

The application discloses a method, device and equipment for block effect detection and a storage medium, and belongs to the technical field of computers. The method comprises the following steps: receiving an encoded video frame sent by a server, decoding the encoded video frame to obtain a decoded video frame; obtaining pixel values of pixel points in a middle region of a macro block in the decoded video frame; determining the number of adjacent macro block pairs with block effects in the decoded video frame based on the pixel values of the pixel points in the middle region of the macro block; determining a ratio of the number of the adjacent macro block pairs with block effects to the total number of adjacent macro block pairs in the decoded video frame as a block effect saliency index; and sending the block effect saliency index to the server, wherein the block effect saliency index is used to instruct the server to adjust an encoding code rate of a video stream sent by a terminal based on the block effect saliency index. The application can improve the video quality.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device and storage medium for block effect detection. Background Technology

[0002] With the development of communication technology and the improvement of computer processing power, cloud gaming has made groundbreaking progress. Video compression technology is indispensable in cloud gaming. During video compression and transmission, encoding and decoding are generally performed on a macroblock basis. This can cause adjacent macroblocks with previously continuous pixel values ​​to become discontinuous after encoding and decoding, resulting in users seeing individual blocks, a phenomenon known as block artifacts. Related technologies detect block artifacts and then employ appropriate methods to resolve the problem.

[0003] Generally, the method for detecting block artifacts is as follows: Identify the boundary regions of two adjacent macroblocks and obtain the pixel values ​​of all pixels within these two boundary regions. Then, calculate the average of the pixel values ​​for each of these two boundary regions and subtract the two averages. If the absolute value of this difference is greater than a preset threshold, block artifacts are determined to exist. The server can then adjust certain encoding parameters to resolve the block artifact problem in the image.

[0004] In some codec standards, the decoding process includes loop filtering. Loop filtering smooths the boundary regions of adjacent macroblocks in an image, reducing the pixel value differences between these boundary regions. In such cases, the aforementioned techniques may fail to detect block artifacts in the decoded image. However, images processed by loop filtering often still show individual blocks. This prevents the block artifact problem from being resolved promptly, thus affecting video quality. Summary of the Invention

[0005] This application provides a method, apparatus, device, and storage medium for detecting block effects, which can solve the problems of related technologies. The technical solution is as follows:

[0006] Firstly, a method for detecting block artifacts is provided, the method comprising:

[0007] Receive encoded video frames sent by the server, and decode the encoded video frames to obtain decoded video frames;

[0008] Obtain the pixel values ​​of the pixels in the middle region of the macroblock in the decoded video frame;

[0009] Based on the pixel values ​​of the pixels in the central region of the macroblock, the number of adjacent macroblock pairs with block artifacts in the decoded video frame is determined.

[0010] The ratio of the number of adjacent macroblock pairs exhibiting block effects to the total number of adjacent macroblock pairs in the decoded video frame is determined as an indicator of the significance of block effects.

[0011] The block effect saliency index is sent to the server, wherein the block effect saliency index is used to instruct the server to adjust the encoding bitrate of the video stream sent to the terminal based on the block effect saliency index.

[0012] In one possible implementation, determining the number of adjacent macroblock pairs with block artifacts in the decoded video frame based on the pixel values ​​of pixels in the central region of the macroblock includes:

[0013] Determine the variance and mean of the pixel values ​​of the pixels in the central region of the macroblock;

[0014] For two macroblocks in an adjacent macroblock pair of the decoded video frame, determine whether the adjacent macroblock pair has block effects based on the variance and mean of the two macroblocks respectively;

[0015] Determine the number of adjacent macroblock pairs in the decoded video frame that exhibit block artifacts.

[0016] In one possible implementation, determining whether the adjacent macroblock pair exhibits block effects based on the variance and mean of the two macroblocks respectively includes:

[0017] In response to the fact that the variances corresponding to the two macroblocks are both less than a variance threshold, and the absolute value of the difference between the means corresponding to the two macroblocks is less than a first mean threshold and greater than a second mean threshold, it is determined that the adjacent macroblock pair has a block effect.

[0018] In response to the following: the variance of at least one of the two macroblocks is greater than a variance threshold, the absolute value of the difference between the means of the two macroblocks is greater than a first mean threshold, or the absolute value of the difference between the means of the two macroblocks is less than a second mean threshold, it is determined that the adjacent macroblock pair does not have a block effect.

[0019] In one possible implementation, determining whether the adjacent macroblock pair exhibits block effects based on the variance and mean of the two macroblocks respectively includes:

[0020] Based on the mean values ​​corresponding to the two macroblocks, the target value range is determined;

[0021] Determine the pixel values ​​of pixels in the boundary region adjacent to the two macroblocks;

[0022] In response to the fact that the variances corresponding to the two macroblocks are both less than the variance threshold, the absolute value of the difference between the means corresponding to the two macroblocks is less than the first mean threshold and greater than the second mean threshold, and the pixel values ​​of the pixels in the boundary region are all within the target value range, it is determined that the adjacent macroblock pair has a block effect;

[0023] In response to the following: the variance of at least one of the two macroblocks is greater than a variance threshold; the absolute value of the difference between the means of the two macroblocks is greater than a first mean threshold; the absolute value of the difference between the means of the two macroblocks is less than a second mean threshold; or there is a pixel value of at least one pixel in the boundary region that is not in the target value range, it is determined that the adjacent macroblock pair does not have block effects.

[0024] In one possible implementation, determining the target value range based on the mean of the two macroblocks includes:

[0025] The upper limit of the interval is determined by the sum of the larger mean of the two macroblocks and the target value, and the lower limit of the interval is determined by the difference between the smaller mean of the two macroblocks and the target value.

[0026] In one possible implementation, the adjacent boundary region of the two macroblocks is a rectangular region between the boundaries of the two macroblocks and the central region of the two macroblocks, wherein the central region of the macroblock is a square region centered on the center of the macroblock.

[0027] In one possible implementation, before obtaining the pixel values ​​of pixels in the central region of a macroblock in the decoded video frame, the method further includes:

[0028] Receive the quantization parameters of the encoded video frame sent by the server;

[0029] The quantization parameter is determined to be greater than the quantization parameter threshold.

[0030] Secondly, a device for detecting block effects is provided, the device comprising:

[0031] The decoding module is used to receive encoded video frames sent by the server and decode the encoded video frames to obtain decoded video frames.

[0032] The acquisition module is used to acquire the pixel values ​​of pixels in the central region of macroblocks in the decoded video frame;

[0033] The first determining module is used to determine the number of adjacent macroblock pairs with block artifacts in the decoded video frame based on the pixel values ​​of the pixels in the central region of the macroblock.

[0034] The second determining module is used to determine the ratio of the number of adjacent macroblock pairs with block effects to the total number of adjacent macroblock pairs in the decoded video frame, as a block effect significance index.

[0035] A sending module is used to send the block effect saliency index to the server, wherein the block effect saliency index is used to instruct the server to adjust the encoding bitrate of the video stream sent to the terminal based on the block effect saliency index.

[0036] In one possible implementation, the first determining module is configured to:

[0037] Determine the variance and mean of the pixel values ​​of the pixels in the central region of the macroblock;

[0038] For two macroblocks in an adjacent macroblock pair of the decoded video frame, determine whether the adjacent macroblock pair has block effects based on the variance and mean of the two macroblocks respectively;

[0039] Determine the number of adjacent macroblock pairs in the decoded video frame that exhibit block artifacts.

[0040] In one possible implementation, the first determining module is configured to:

[0041] In response to the fact that the variances corresponding to the two macroblocks are both less than a variance threshold, and the absolute value of the difference between the means corresponding to the two macroblocks is less than a first mean threshold and greater than a second mean threshold, it is determined that the adjacent macroblock pair has a block effect.

[0042] In response to the following: the variance of at least one of the two macroblocks is greater than a variance threshold, the absolute value of the difference between the means of the two macroblocks is greater than a first mean threshold, or the absolute value of the difference between the means of the two macroblocks is less than a second mean threshold, it is determined that the adjacent macroblock pair does not have a block effect.

[0043] In one possible implementation, the first determining module is configured to:

[0044] Based on the mean values ​​corresponding to the two macroblocks, the target value range is determined;

[0045] Determine the pixel values ​​of pixels in the boundary region adjacent to the two macroblocks;

[0046] In response to the fact that the variances corresponding to the two macroblocks are both less than the variance threshold, the absolute value of the difference between the means corresponding to the two macroblocks is less than the first mean threshold and greater than the second mean threshold, and the pixel values ​​of the pixels in the boundary region are all within the target value range, it is determined that the adjacent macroblock pair has a block effect;

[0047] In response to the following: the variance of at least one of the two macroblocks is greater than a variance threshold; the absolute value of the difference between the means of the two macroblocks is greater than a first mean threshold; the absolute value of the difference between the means of the two macroblocks is less than a second mean threshold; or there is a pixel value of at least one pixel in the boundary region that is not in the target value range, it is determined that the adjacent macroblock pair does not have block effects.

[0048] In one possible implementation, the first determining module is configured to:

[0049] The upper limit of the interval is determined by the sum of the larger mean of the two macroblocks and the target value, and the lower limit of the interval is determined by the difference between the smaller mean of the two macroblocks and the target value.

[0050] In one possible implementation, the adjacent boundary region of the two macroblocks is a rectangular region between the boundaries of the two macroblocks and the central region of the two macroblocks, wherein the central region of the macroblock is a square region centered on the center of the macroblock.

[0051] In one possible implementation, the acquisition module is further configured to:

[0052] Receive the quantization parameters of the encoded video frame sent by the server;

[0053] The quantization parameter is determined to be greater than the quantization parameter threshold.

[0054] Thirdly, a computer device is provided, comprising a memory and a processor, the memory for storing computer instructions; the processor executes the computer instructions stored in the memory to cause the computer device to perform the method of the first aspect and its possible implementations.

[0055] Fourthly, a computer-readable storage medium is provided that stores computer program code, and in response to the computer program code being executed by a computer device, the computer device executes the method of the first aspect and its possible implementations.

[0056] Fifthly, a computer program product is provided, the computer program product including computer program code, and a method for the computer device to execute the first aspect and its possible implementations in response to the computer program code being executed by a computer device.

[0057] The technical solutions provided by the embodiments of this application may include the following beneficial effects:

[0058] The method provided in this application embodiment allows a terminal to decode an encoded video frame after receiving it from a server. The terminal then obtains the pixel values ​​of pixels in the central region of macroblocks within the decoded video frame. Based on these pixel values, the terminal determines the number of adjacent macroblock pairs exhibiting block artifacts in the decoded video frame. Finally, the terminal determines the ratio of the number of adjacent macroblock pairs exhibiting block artifacts to the total number of adjacent macroblock pairs in the decoded video frame, and sends this ratio as a block artifact saliency index to the server. The server can adjust the encoding bitrate and other parameters of the video stream sent by the terminal based on the block artifact saliency index. Thus, for video frames where the boundary regions of adjacent macroblocks have been smoothed, the terminal can determine whether block artifacts exist in adjacent macroblocks by using the pixel values ​​of pixels in the central region of the adjacent macroblocks, providing a block artifact saliency index, and sending it to the server. The server can then perform corresponding processing based on this index, enabling timely resolution of block artifact problems and improving video quality. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 This is a schematic diagram of the structure of a system provided in an embodiment of this application;

[0061] Figure 2 This is a schematic diagram of the structure of a terminal provided in an embodiment of this application;

[0062] Figure 3 This is a schematic diagram of the structure of a server provided in an embodiment of this application;

[0063] Figure 4 This is a schematic flowchart of a block effect detection method provided in an embodiment of this application;

[0064] Figure 5 This is a schematic diagram of the central region of a macroblock provided in an embodiment of this application;

[0065] Figure 6 This is a schematic diagram of the boundary region between two adjacent macroblocks provided in an embodiment of this application;

[0066] Figure 7 This is a schematic flowchart of a block effect detection method provided in an embodiment of this application;

[0067] Figure 8This is a schematic diagram of the structure of a block effect detection device provided in an embodiment of this application;

[0068] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0069] This application provides a method for detecting block artifacts. This method is used to detect whether block artifacts exist in an image after encoding and decoding. It can be applied to applications with video playback functions, especially applications with the function of playing real-time network video streams, such as cloud gaming applications and live streaming applications.

[0070] This method can be implemented by a system consisting of a terminal and a server, such as... Figure 1 As shown. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal is used to decode video streams and can also be used to communicate with other devices, such as communicating with a server. An application is installed on the terminal, which can be a video calling application, a video player application, a screen sharing application, or a cloud gaming application with the function of decoding and playing network video streams. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal and the server can be connected directly or indirectly through wired or wireless communication, which is not limited in this application.

[0071] From a hardware perspective, the terminal's structure can be as follows: Figure 2 As shown, it includes a processor 210, a memory 220, a display unit 230, and a communication unit 240.

[0072] The processor 210 can be a CPU (central processing unit) or a SoC (system on chip), etc. The processor 210 can be used to execute various instructions involved in the method.

[0073] The memory 220 may include various volatile or non-volatile memories, such as SSD (solid state disk) and DRAM (dynamic random access memory). The memory 220 can be used to store pre-stored data, intermediate data, and result data during the video decoding process, such as pixel values.

[0074] The display component 230 can be a standalone screen, or a screen integrated with the terminal body, a projector, etc. The screen can be a touch screen or a non-touch screen. The display component is used to display system interfaces, application interfaces, etc., such as game interfaces.

[0075] The communication component 240 can be a wired network connector, a WiFi (wireless fidelity) module, a Bluetooth module, a cellular network communication module, etc. The communication component can be used to transmit data with other devices, such as servers or other terminals.

[0076] In addition to processors and memory, terminals may also include audio acquisition components, audio output components, etc.

[0077] The audio acquisition component can be a microphone, used to capture the user's voice. The audio output component can be a speaker, headphones, etc., used to play audio.

[0078] From a hardware perspective, the server's structure can be as follows: Figure 3 As shown, it includes a processor 310, a memory 320, and a communication component 330.

[0079] The processor 310 can be a CPU (central processing unit) or a SoC (system on chip), etc. The processor 310 can be used to execute various instructions involved in the method.

[0080] The memory 320 may include various volatile or non-volatile memories, such as SSD (solid state disk) and DRAM (dynamic random access memory). The memory 320 can be used to store pre-stored data, intermediate data, and result data of various messages sent and received by the server, such as video frames and block effect significance indicators.

[0081] The communication component 330 can be a wired network connector, a WiFi (wireless fidelity) module, a Bluetooth module, a cellular network communication module, etc. The communication component can be used to transmit data with other devices, such as servers or other terminals. For example, a server can receive block effect significance indicators transmitted by a terminal, or send video frames to a terminal.

[0082] This application uses a mobile phone running and having a cloud gaming application installed as an example to illustrate the solution in detail. Other situations are similar and will not be described in detail here.

[0083] The following is a description of several terms used in the embodiments of this application:

[0084] Cloud gaming, also known as gaming on demand, is an online gaming technology based on cloud computing. It enables thin clients with relatively limited graphics processing and data processing capabilities to run high-quality games. In cloud gaming, the game does not reside on the player's terminal but runs on a cloud server. The cloud server renders the game scene as a video and audio stream, which is then transmitted to the player's terminal via the network. The player's terminal does not need powerful graphics processing and data processing capabilities; it only needs basic streaming media playback capabilities and the ability to receive player input commands and send them to the cloud server.

[0085] Macroblock: A fundamental concept in video encoding and decoding technology. In video coding, a encoded image is typically divided into several macroblocks.

[0086] Block artifacts: In macroblock-based encoding and decoding techniques, as the bit rate decreases, quantization becomes coarser, and discontinuities appear at the boundaries of blocks, resulting in obvious defects in the reconstructed image, which is called block artifacts.

[0087] H.264: Also known as MPEG-4 Part 10 (Moving Picture Experts Group-4 Part 10), AVC (Advanced Video Coding) is a block-oriented, motion-compensated video coding standard. By 2014, it had become one of the most commonly used formats for high-precision video recording, compression, and distribution.

[0088] Qstep (quantization step): A crucial parameter in image coding, Qstep determines the compression ratio and image precision of the quantizer. A larger Qstep results in a shorter code length and greater loss of image detail; conversely, a smaller Qstep results in a longer code length and less loss of image detail.

[0089] QP (quantization parameter): This reflects the degree of spatial detail compression. A smaller value results in finer quantization, higher image quality, and a longer bitstream. If QP is small, most details are preserved; if QP is large, some details are lost, the bitrate decreases, and image distortion and quality deteriorate. QP is the Qstep number. For luminance coding, there are 52 possible Qstep values, and QP values ​​range from 0 to 51.

[0090] Bit rate, also known as bit rate, refers to the number of bits of data transmitted or processed per unit of time in the telecommunications and computing fields. In telecommunications, bit rate is frequently used synonymously for connection speed, transmission speed, information transmission rate, and digital bandwidth capacity, often written as bit / sec (bits per second). The speed of digital image information transmission can be calculated based on the number of bits occupied during the storage of each image frame and the transmission bit rate.

[0091] Rate Control (RC) is a crucial technical module in practical video encoders. Depending on the specific application scenario (e.g., real-time video encoding, offline video encoding, traditional broadcast television services, internet streaming media transmission services, etc.), there are various specific requirements and limitations on the output bitrate of the video encoder, such as bitrate size, buffer size, and initial delay. Rate control addresses these different application scenarios by controlling the QP (Queries Per Frame) of each frame, and even each encoding unit, to ensure that the output encoded video bitrate meets various specific constraints, while simultaneously optimizing encoding performance (including encoding efficiency and subjective quality) as much as possible.

[0092] In cloud gaming scenarios, servers need to transmit large amounts of game visuals to terminals in real time. When transmitting video data related to game visuals, video frames need to be encoded and compressed. The mainstream compression technology, H.264, compresses images on a macroblock basis. During compression, a low bitrate may result in a large QStep (Quick Step), leading to discontinuous pixel values ​​in adjacent macroblocks in the decoded video data received by the terminal, a phenomenon known as block artifacts. The terminal performs block artifact detection on the decoded image, generates a block artifact significance index, and sends this index to the server. Upon receiving this index, the server performs appropriate processing based on it.

[0093] This application provides a method for detecting block artifacts for the aforementioned application scenarios. The processing flow of this method is as follows: Figure 4 As shown, the processing steps include the following:

[0094] 401, The terminal receives encoded video frames sent by the server.

[0095] In practice, during cloud gaming on the terminal, the server renders the game images, encodes them, and sends the encoded video frames to the terminal. The server-rendered game images in cloud gaming technology are the same as those rendered locally on the terminal in other technologies, including various control elements, windows, and other content.

[0096] 402, decode the encoded video frame to obtain the decoded video frame.

[0097] In implementation, the terminal decodes the received video frames according to the set codec standard and displays them, allowing the user to see the game screen on the terminal. The codec standard for cloud gaming can be H.264, and H.264 bitrate control technologies include CBR (constant bit rate), VBR (variable bit rate), and CVBR (constrained variable bit rate). During the server's encoding of video frames, bitrate control technology controls the bitrate of each video frame. In this process, there may be instances where the encoded bitrate of a video frame is too low, leading to blockiness or other artifacts in the decoded video frames after the terminal decodes them.

[0098] The terminal detects block artifacts in the decoded video frames. The terminal can detect every single video frame or every certain number of frames. The terminal can also determine whether to detect video frames based on the interface type, which is sent by the server. For example, if the interface is a game screen, every frame of the game screen can be detected; if the interface is a settings screen, the terminal can only detect the first video frame of that interface sent by the server.

[0099] 403, The terminal obtains the pixel value of the pixel point in the middle region of the macroblock in the decoded video frame.

[0100] A macroblock is a square region. The central region of a macroblock is a square region centered on the center of the macroblock. For example, if a macroblock consists of 16×16 pixels, the central region is the 10×10 pixel area located at the center of the macroblock. Figure 5 The shaded area in the image.

[0101] For each macroblock in a decoded video frame, the terminal can determine the center position of the region where the macroblock is located. Then, based on a preset size (i.e., the size of the central region), the central region is determined with this center position as the center. This allows the terminal to obtain the pixel values ​​of all pixels within the central region of each macroblock.

[0102] 404. The terminal determines the number of adjacent macroblock pairs with block artifacts in the decoded video frame based on the pixel values ​​of the pixels in the central region of the macroblock.

[0103] Each adjacent macroblock pair consists of two adjacent macroblocks, and different adjacent macroblock pairs can share a single macroblock.

[0104] In implementation, for each macroblock in the decoded video frame, the terminal obtains the pixel values ​​of the pixels in the central region of the macroblock and calculates the mean and variance of the pixel values ​​in the central region. Then, for all adjacent macroblock pairs, it determines whether the variance and mean of the two macroblocks meet certain conditions. If they do, an adjacent macroblock pair with block artifacts is recorded; otherwise, it is not recorded, or an adjacent macroblock pair without block artifacts is recorded.

[0105] The specific steps for detecting block effects are as follows:

[0106] Step 1: Determine the variance and mean of the pixel values ​​of the pixels in the central region of the macroblock.

[0107] In implementation, after obtaining the pixel values ​​of the pixels in the central region of the macroblock, the mean and variance of these pixel values ​​are calculated. For example, two adjacent macroblocks include a first macroblock and a second macroblock, where the mean value corresponding to the first macroblock is μ. s The variance is σ s The mean value corresponding to the second macroblock is μ. t The variance is σ t .

[0108] Step 2: For two macroblocks in an adjacent macroblock pair of a decoded video frame, determine whether there is block artifacts in the adjacent macroblock pair based on the variance and mean of the two macroblocks respectively.

[0109] There are several ways to determine whether adjacent macroblock pairs have block effects in this step:

[0110] Method 1: If the variances of both macroblocks are less than a variance threshold, and the absolute value of the difference between the means of the two macroblocks is less than a first mean threshold and greater than a second mean threshold, then a block effect is determined to exist in the adjacent macroblock pair. If the variance of at least one macroblock is greater than a variance threshold, the absolute value of the difference between the means of the two macroblocks is greater than a first mean threshold, or the absolute value of the difference between the means of the two macroblocks is less than a second mean threshold, then a block effect is determined to not exist in the adjacent macroblock pair.

[0111] Among them, the variance threshold, the first mean threshold, and the second mean threshold can be preset and stored by technicians, and their values ​​can be empirical values.

[0112] In practice, block artifacts often appear on flat textures. Flat textures have two characteristics: first, the pixel values ​​within a macroblock are basically equal; second, the pixel value differences between macroblocks are small. Therefore, we can determine whether a macroblock is in a flat texture region by checking if its variance is less than a variance threshold. When the variance of a macroblock is less than the variance threshold, it can be considered to be in a flat texture region. We can determine whether adjacent macroblocks are in the same texture region by checking if the absolute value of the difference between the means of two adjacent macroblocks is less than a first mean threshold and greater than a second mean threshold. When the pixel value difference between two macroblocks is too large, i.e., the absolute value of the difference between the means of the two macroblocks is greater than the first mean threshold, it can be considered that the difference is not due to block artifacts, but rather a difference in the texture itself. When the pixel values ​​of two macroblocks have no significant difference, i.e., the absolute value of the difference between the means of the two macroblocks is less than the second mean threshold, it can be considered that there is no block artifact between the two macroblocks. When the absolute value of the difference between the means of two macroblocks is less than the first mean threshold and greater than the second mean threshold, the difference in pixel values ​​between adjacent macroblocks can be considered to be caused by block effects.

[0113] For example, if the variance threshold is 5, the first threshold is 10, and the second threshold is 2, then when the variance of the first macroblock and the variance of the second macroblock are both less than the variance threshold, and the absolute value of the difference between the means of the two macroblocks is less than the first mean threshold and greater than the second mean threshold, i.e., σ s <5 and σ t <5 and 2≥|μ s -μ t When |≥10, it can be considered that adjacent macroblock pairs have block effects.

[0114] Method 2: Determine the target value range based on the means corresponding to two macroblocks. Determine the pixel values ​​of pixels in the boundary regions adjacent to the two macroblocks. If the variances of both macroblocks are less than a variance threshold, the absolute value of the difference between the means of the two macroblocks is less than a first mean threshold and greater than a second mean threshold, and the pixel values ​​of pixels in the boundary regions are all within the target value range, then the adjacent macroblock pair is determined to have block effects. If the variance of at least one macroblock is greater than a variance threshold, the absolute value of the difference between the means of the two macroblocks is greater than a first mean threshold, the absolute value of the difference between the means of the two macroblocks is less than a second mean threshold, or the pixel value of at least one pixel in the boundary region is not within the target value range, then the adjacent macroblock pair is determined to have no block effects.

[0115] In this context, the boundary region between two adjacent macroblocks is a rectangular area between the boundaries of the two macroblocks and their respective central regions. Three sides of this rectangular region can be collinear with three sides of the macroblock, and another side of this rectangular region can be collinear with one side of the central region. For example, as... Figure 6As shown, a macroblock is composed of 16×16 pixels. The central region is the area containing 10×10 pixels at the center of the macroblock, and the boundary region between two adjacent macroblocks is the area containing 6×16 pixels.

[0116] In practice, in the image before video frame encoding, the boundary region of two adjacent macroblocks may have a different texture from the central region. For example, a line might pass through the edge region but not the central region. In this case, the two adjacent macroblocks are not involved in the block artifact problem. Therefore, Method 2, after determining whether there is a block artifact between adjacent macroblocks by using the mean and variance of the pixel values ​​of the pixels in the central region of the adjacent macroblocks, further determines whether there is a block artifact by checking whether the pixel values ​​of the pixels in the boundary region of the adjacent macroblocks are all within the target value range. If the pixel values ​​of the pixels in the boundary region of the adjacent macroblocks are all within the target value range, it means that the boundary region and the central region have the same texture. Based on Method 1, the difference in pixel values ​​between these adjacent macroblocks can be considered to be caused by the block artifact. If the pixel values ​​of some pixels in the boundary region of the adjacent macroblocks are not within the target value range, it means that the boundary region may have a different texture from the central region. It can be considered that there is no block artifact between these adjacent macroblocks. The target value range can be determined by summing the larger of the means of the two macroblocks and the target value as the upper limit of the range, and by subtracting the smaller of the means of the two macroblocks from the target value as the lower limit of the range. The target value is preset by the terminal. For example, if the target value is 3, the target value range is [min(μ s ,μ t )-3, max(μ s ,μ t The target value range can also be obtained by determining the larger mean of the two macroblocks as the upper limit of the range and the smaller mean of the two macroblocks as the lower limit of the range. For example, the target value range is [min(μ s ,μ t ), max(μ s ,μ t )).

[0117] Step 3: Determine the number of adjacent macroblock pairs with block artifacts in the decoded video frame.

[0118] In practice, when the result of step two is that adjacent macroblock pairs have block artifacts, the terminal records one adjacent macroblock pair with block artifacts. The terminal accumulates the number of adjacent macroblock pairs with block artifacts to determine the total number of adjacent macroblock pairs with block artifacts in the video frame.

[0119] 405. The ratio of the number of adjacent macroblock pairs with block effects determined by the terminal to the total number of adjacent macroblock pairs in the decoded video frame is used as an indicator of the significance of block effects.

[0120] For example, the number of adjacent macroblock pairs with block artifacts in the decoded video frame is 900, the total number of adjacent macroblock pairs is 1000, and the block artifact significance index is 0.9.

[0121] In implementation, the total number of adjacent macroblock pairs in a video frame can be determined using the following formula: (Int(M / A)-1)×Int(N / A)+(Int(N / A)-1)×Int(M / A). Where A is the side length of the macroblock, M is the number of pixels in the horizontal direction, and N is the number of pixels in the vertical direction; Int() refers to the floor function, for example, Int(1080 / 16)=Int(67.5)=67. For a video frame with a resolution of 1920×1080, the number of pixels in the horizontal direction is 1920, and the number of adjacent macroblock pairs is Int(1920 / 16). The number of pixels in the vertical direction is 1080, and the number of adjacent macroblock pairs is Int(1080 / 16). The total number of adjacent macroblock pairs is (Int(1920 / 16)-1)×Int(1080 / 16)+(Int(1080 / 16)-1)×Int(1920 / 16)=15893.

[0122] The more adjacent macroblock pairs with block artifacts there are, and the higher the ratio of the number of adjacent macroblock pairs with block artifacts to the total number of adjacent macroblock pairs in the decoded video frame, the higher the significance of block artifacts in the video frame; conversely, the lower the ratio, the less significant the block artifacts.

[0123] 406, The terminal sends the block effect significance index to the server.

[0124] In implementation, the terminal can generate an indicator reporting message, add the block effect significance indicator to the indicator reporting message, and also add the account of the locally logged-in application to the block effect significance indicator. Then, the terminal sends the block effect significance indicator to the server.

[0125] 407. The server adjusts the encoding bitrate of the video stream sent by the terminal based on the block effect significance index.

[0126] In implementation, technicians can pre-establish a correspondence between coding bitrate and block effect significance index, and store this correspondence on a server. In this correspondence, a smaller block effect significance index corresponds to a lower coding bitrate, and a larger block effect significance index corresponds to a higher coding bitrate.

[0127] Whenever the server receives a metric reporting message from a terminal, it can retrieve the block effect significance metric and the account information contained within. The server then looks up the encoding bitrate corresponding to that block effect significance metric in the mapping table. Using this bitrate, the server encodes the game image for that account to obtain encoded video frames, which are then sent to the terminal.

[0128] This application also provides a method for detecting block artifacts, addressing the aforementioned application scenarios. This method combines cloud information for judgment, reducing the impact of block artifact detection on system latency. The processing flow of this method is as follows: Figure 7 As shown, the processing steps include the following:

[0129] 701, The terminal receives the encoded video frames and quantization parameters of the encoded video frames sent by the server.

[0130] In practice, during the encoding of video frames, the server determines quantization parameters based on certain logic, and then encodes the video frames based on these parameters to obtain encoded video frames. The encoded video frames and quantization parameters are then sent to the terminal together.

[0131] 702, The terminal decodes the encoded video frame to obtain the decoded video frame.

[0132] 703, the terminal determines that the quantization parameter is greater than the quantization parameter threshold.

[0133] The quantization parameter threshold can be preset and stored by technicians, and its value can be obtained through experiments. In the experiment, the quantization parameter is set from small to large to encode video frames. When the quantization parameter increases to a certain value, block artifacts appear in the decoded video frame, and this value is used as the quantization parameter threshold.

[0134] In practice, the quantization parameter reflects the quality of the decoded video frame. A smaller quantization parameter indicates that the server retains more detail during encoding, resulting in a higher quality decoded video frame less prone to block artifacts. The terminal evaluates the quantization parameter sent by the server. When the quantization parameter exceeds a threshold, the decoded video frame quality is relatively poor, requiring subsequent block artifact detection steps to calculate a block artifact significance index, allowing the server to adjust the subsequent encoding bitrate. When the quantization parameter is less than or equal to the threshold, the decoded video frame quality is relatively good, and the terminal returns a preset reference value.

[0135] 704, The terminal obtains the pixel value of the pixel point in the middle region of the macroblock in the decoded video frame.

[0136] 705. The terminal determines the number of adjacent macroblock pairs with block artifacts in the decoded video frame based on the pixel values ​​of the pixels in the central region of the macroblock.

[0137] 706. The ratio of the number of adjacent macroblock pairs with block effects determined by the terminal to the total number of adjacent macroblock pairs in the decoded video frame is used as an indicator of the significance of block effects.

[0138] 707, The terminal sends the block effect significance index to the server.

[0139] 708, the server adjusts the encoding bitrate of the video stream sent by the terminal based on the block effect significance index.

[0140] The specific processing of steps 704-708 is similar to that of steps 403-407. Please refer to the relevant explanations of steps 403-407, which will not be repeated here.

[0141] The method provided in this application embodiment allows a terminal to decode an encoded video frame after receiving it from a server. The terminal then obtains the pixel values ​​of pixels in the central region of macroblocks within the decoded video frame. Based on these pixel values, the terminal determines the number of adjacent macroblock pairs exhibiting block artifacts in the decoded video frame. Finally, the terminal determines the ratio of the number of adjacent macroblock pairs exhibiting block artifacts to the total number of adjacent macroblock pairs in the decoded video frame, and sends this ratio as a block artifact saliency index to the server. The server can adjust the encoding bitrate and other parameters of the video stream sent by the terminal based on the block artifact saliency index. Thus, for video frames where the boundary regions of adjacent macroblocks have been smoothed, the terminal can determine whether block artifacts exist in adjacent macroblocks by using the pixel values ​​of pixels in the central region of the adjacent macroblocks, providing a block artifact saliency index, and sending it to the server. The server can then perform corresponding processing based on this index, enabling timely resolution of block artifact problems and improving video quality.

[0142] Based on the same technical concept, embodiments of this application also provide a device for detecting block effects, such as... Figure 8 As shown, the device includes:

[0143] The decoding module 810 is used to receive encoded video frames sent by the server and decode the encoded video frames to obtain decoded video frames.

[0144] The acquisition module 820 is used to acquire the pixel values ​​of the pixels in the middle region of the macroblock in the decoded video frame;

[0145] The first determining module 830 is used to determine the number of adjacent macroblock pairs with block artifacts in the decoded video frame based on the pixel values ​​of the pixels in the central region of the macroblock.

[0146] The second determining module 840 is used to determine the ratio of the number of adjacent macroblock pairs with block effects to the total number of adjacent macroblock pairs in the decoded video frame, as a block effect significance index.

[0147] The sending module 850 is used to send the block effect saliency index to the server, wherein the block effect saliency index is used to instruct the server to adjust the encoding bitrate of the video stream sent to the terminal based on the block effect saliency index.

[0148] In one possible implementation, the first determining module 830 is configured to:

[0149] Determine the variance and mean of the pixel values ​​of the pixels in the central region of the macroblock;

[0150] For two macroblocks in an adjacent macroblock pair of the decoded video frame, determine whether the adjacent macroblock pair has block effects based on the variance and mean of the two macroblocks respectively;

[0151] Determine the number of adjacent macroblock pairs in the decoded video frame that exhibit block artifacts.

[0152] In one possible implementation, the first determining module 830 is configured to:

[0153] In response to the fact that the variances corresponding to the two macroblocks are both less than a variance threshold, and the absolute value of the difference between the means corresponding to the two macroblocks is less than a first mean threshold and greater than a second mean threshold, it is determined that the adjacent macroblock pair has a block effect.

[0154] In response to the following: the variance of at least one of the two macroblocks is greater than a variance threshold, the absolute value of the difference between the means of the two macroblocks is greater than a first mean threshold, or the absolute value of the difference between the means of the two macroblocks is less than a second mean threshold, it is determined that the adjacent macroblock pair does not have a block effect.

[0155] In one possible implementation, the first determining module 830 is configured to:

[0156] Based on the mean values ​​corresponding to the two macroblocks, the target value range is determined;

[0157] Determine the pixel values ​​of pixels in the boundary region adjacent to the two macroblocks;

[0158] In response to the fact that the variances corresponding to the two macroblocks are both less than the variance threshold, the absolute value of the difference between the means corresponding to the two macroblocks is less than the first mean threshold and greater than the second mean threshold, and the pixel values ​​of the pixels in the boundary region are all within the target value range, it is determined that the adjacent macroblock pair has a block effect;

[0159] In response to the following: the variance of at least one of the two macroblocks is greater than a variance threshold; the absolute value of the difference between the means of the two macroblocks is greater than a first mean threshold; the absolute value of the difference between the means of the two macroblocks is less than a second mean threshold; or there is a pixel value of at least one pixel in the boundary region that is not in the target value range, it is determined that the adjacent macroblock pair does not have block effects.

[0160] In one possible implementation, the first determining module 830 is configured to:

[0161] The upper limit of the interval is determined by the sum of the larger mean of the two macroblocks and the target value, and the lower limit of the interval is determined by the difference between the smaller mean of the two macroblocks and the target value.

[0162] In one possible implementation, the adjacent boundary region of the two macroblocks is a rectangular region between the boundaries of the two macroblocks and the central region of the two macroblocks, wherein the central region of the macroblock is a square region centered on the center of the macroblock.

[0163] In one possible implementation, the acquisition module 820 is further configured to:

[0164] Receive the quantization parameters of the encoded video frame sent by the server;

[0165] The quantization parameter is determined to be greater than the quantization parameter threshold.

[0166] The apparatus provided in this application embodiment allows a terminal to decode an encoded video frame after receiving it from a server. The terminal then obtains the pixel values ​​of pixels in the central region of macroblocks within the decoded video frame. Based on these pixel values, the terminal determines the number of adjacent macroblock pairs exhibiting block artifacts in the decoded video frame. Finally, the terminal determines the ratio of the number of adjacent macroblock pairs exhibiting block artifacts to the total number of adjacent macroblock pairs in the decoded video frame, and sends this ratio as a block artifact saliency index to the server. The server can adjust the encoding bitrate and other parameters of the video stream sent by the terminal based on the block artifact saliency index. Thus, for video frames where the boundary regions of adjacent macroblocks have been smoothed, the terminal can determine whether block artifacts exist in adjacent macroblocks by using the pixel values ​​of pixels in the central region of the adjacent macroblocks, providing a block artifact saliency index, and sending it to the server. The server can then perform corresponding processing based on this index, enabling timely resolution of block artifact problems and improving video quality.

[0167] It should be noted that the block effect detection apparatus provided in the above-described embodiments is only illustrated by the division of the functional modules described above. In practical applications, the functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the block effect detection apparatus and the block effect detection method embodiments provided in the above-described embodiments belong to the same concept, and their specific implementation process is detailed in the method embodiments, which will not be repeated here.

[0168] Figure 9 A structural block diagram of an electronic device 900 provided in an embodiment of this application is shown. This electronic device can be any of the terminals described in the above embodiments. The electronic device 900 can be a portable mobile terminal, such as a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. The electronic device 900 may also be referred to as a user device, portable terminal, laptop terminal, desktop terminal, or other names.

[0169] Typically, electronic device 900 includes a processor 901 and a memory 902.

[0170] Processor 901 may include one or more processing cores, such as a quad-core processor or an octa-core processor. Processor 901 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). Processor 901 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 901 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 901 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0171] The memory 902 may include one or more computer-readable storage media, which may be non-transitory. The memory 902 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 902 are used to store at least one instruction, which is executed by the processor 901 to implement the method provided in the embodiments of this application.

[0172] In some embodiments, the electronic device 900 may optionally include a peripheral device interface 903 and at least one peripheral device. The processor 901, memory 902, and peripheral device interface 903 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 903 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 904, a display screen 905, a camera assembly 906, an audio circuit 907, a positioning assembly 908, and a power supply 909.

[0173] Peripheral device interface 903 can be used to connect at least one I / O (Input / Output) related peripheral device to processor 901 and memory 902. In some embodiments, processor 901, memory 902 and peripheral device interface 903 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 901, memory 902 and peripheral device interface 903 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0174] The radio frequency (RF) circuit 904 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The RF circuit 904 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 904 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 904 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 904 can communicate with other terminals through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: the World Wide Web, metropolitan area networks, intranets, various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks, and / or WiFi (Wireless Fidelity) networks. In some embodiments, the RF circuit 904 may also include circuitry related to NFC (Near Field Communication), which is not limited in this application.

[0175] Display screen 905 is used to display a UI (User Interface). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 905 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 901 for processing. In this case, display screen 905 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 905, disposed on the front panel of electronic device 900; in other embodiments, there may be at least two display screens 905, disposed on different surfaces of electronic device 900 or in a folded design; in other embodiments, display screen 905 may be a flexible display screen, disposed on a curved or folded surface of electronic device 900. Furthermore, display screen 905 may be configured as a non-rectangular irregular shape, i.e., a non-rectangular screen. Display screen 905 may be made of materials such as LCD (Liquid Crystal Display) or OLED (Organic Light-Emitting Diode).

[0176] The camera assembly 906 is used to acquire images or videos. Optionally, the camera assembly 906 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (Virtual Reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 906 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0177] The audio circuit 907 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting the sound waves into electrical signals that are input to the processor 901 for processing, or input to the radio frequency circuit 904 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each located in a different part of the electronic device 900. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert the electrical signals from the processor 901 or the radio frequency circuit 904 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 907 may also include a headphone jack.

[0178] The positioning component 908 is used to locate the current geographical location of the electronic device 900 in order to enable navigation or LBS (Location Based Service). The positioning component 908 can be a positioning component based on GPS (Global Positioning System) or BeiDou system.

[0179] Power supply 909 is used to supply power to various components in electronic device 900. Power supply 909 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 909 includes a rechargeable battery, the rechargeable battery can be a wired rechargeable battery or a wireless rechargeable battery. A wired rechargeable battery is a battery that is charged via a wired line, while a wireless rechargeable battery is a battery that is charged via a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0180] In some embodiments, the electronic device 900 further includes one or more sensors 910. The one or more sensors 910 include, but are not limited to: an accelerometer 911, a gyroscope 912, a pressure sensor 913, a fingerprint sensor 914, an optical sensor 915, and a proximity sensor 916.

[0181] Accelerometer 911 can detect the magnitude of acceleration on the three coordinate axes of a coordinate system established by electronic device 900. For example, accelerometer 911 can be used to detect the components of gravitational acceleration on the three coordinate axes. Processor 901 can control display screen 905 to display the user interface in either a landscape or portrait view based on the gravitational acceleration signal acquired by accelerometer 911. Accelerometer 911 can also be used for games or for acquiring user motion data.

[0182] The gyroscope sensor 912 can detect the orientation and rotation angle of the electronic device 900. The gyroscope sensor 912, in conjunction with the accelerometer sensor 911, can collect 3D motion data from the user on the electronic device 900. Based on the data collected by the gyroscope sensor 912, the processor 901 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0183] The pressure sensor 913 can be disposed on the side bezel of the electronic device 900 and / or on the lower layer of the display screen 905. When the pressure sensor 913 is disposed on the side bezel of the electronic device 900, it can detect the user's grip signal on the electronic device 900, and the processor 901 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 913. When the pressure sensor 913 is disposed on the lower layer of the display screen 905, the processor 901 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 905. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0184] The fingerprint sensor 914 is used to collect a user's fingerprint. The processor 901 identifies the user based on the fingerprint collected by the fingerprint sensor 914, or vice versa. When the user's identity is verified as trusted, the processor 901 authorizes the user to perform relevant sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, making payments, and changing settings. The fingerprint sensor 914 can be located on the front, back, or side of the electronic device 900. When the electronic device 900 has a physical button or manufacturer logo, the fingerprint sensor 914 can be integrated with the physical button or manufacturer logo.

[0185] An optical sensor 915 is used to collect ambient light intensity. In one embodiment, the processor 901 can control the display brightness of the display screen 905 based on the ambient light intensity collected by the optical sensor 915. Specifically, when the ambient light intensity is high, the display brightness of the display screen 905 is increased; when the ambient light intensity is low, the display brightness of the display screen 905 is decreased. In another embodiment, the processor 901 can also dynamically adjust the shooting parameters of the camera assembly 906 based on the ambient light intensity collected by the optical sensor 915.

[0186] A proximity sensor 916, also known as a distance sensor, is typically located on the front panel of an electronic device 900. The proximity sensor 916 is used to detect the distance between the user and the front of the electronic device 900. In one embodiment, when the proximity sensor 916 detects that the distance between the user and the front of the electronic device 900 is gradually decreasing, the processor 901 controls the display screen 905 to switch from a screen-on state to a screen-off state; when the proximity sensor 916 detects that the distance between the user and the front of the electronic device 900 is gradually increasing, the processor 901 controls the display screen 905 to switch from a screen-off state to a screen-on state.

[0187] Those skilled in the art will understand that Figure 9 The structure shown does not constitute a limitation on the electronic device 900, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0188] In this embodiment, a computer-readable storage medium is also provided, such as a memory including instructions that can be executed by a processor in a terminal to complete the method for performing interactive operations as described in the above embodiments. This computer-readable storage medium can be non-transitory. For example, the computer-readable storage medium can be ROM (Read-Only Memory), RAM (Random Access Memory), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0189] It should be noted that the information (including but not limited to user equipment information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals (including but not limited to signals transmitted between user terminals and other devices) involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0190] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0191] The above description is only some possible embodiments of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for detecting block effects, characterized in that, The method includes: Receive encoded video frames sent by the server, and decode the encoded video frames to obtain decoded video frames; Obtain the pixel values ​​of the pixels in the middle region of the macroblock in the decoded video frame; Based on the pixel values ​​of the pixels in the central region of the macroblock, the number of adjacent macroblock pairs with block artifacts in the decoded video frame is determined. The ratio of the number of adjacent macroblock pairs exhibiting block effects to the total number of adjacent macroblock pairs in the decoded video frame is determined as an indicator of the significance of block effects. The block effect saliency index is sent to the server, wherein the block effect saliency index is used to instruct the server to adjust the encoding bitrate of the video stream sent to the terminal based on the block effect saliency index.

2. The method according to claim 1, characterized in that, Determining the number of adjacent macroblock pairs with block artifacts in the decoded video frame based on the pixel values ​​of pixels in the central region of the macroblock includes: Determine the variance and mean of the pixel values ​​of the pixels in the central region of the macroblock; For two macroblocks in an adjacent macroblock pair of the decoded video frame, determine whether the adjacent macroblock pair has block effects based on the variance and mean of the two macroblocks respectively; Determine the number of adjacent macroblock pairs in the decoded video frame that exhibit block artifacts.

3. The method according to claim 2, characterized in that, Determining whether there is a block effect in the adjacent macroblock pair based on the variance and mean of the two macroblocks respectively includes: In response to the fact that the variances corresponding to the two macroblocks are both less than a variance threshold, and the absolute value of the difference between the means corresponding to the two macroblocks is less than a first mean threshold and greater than a second mean threshold, it is determined that the adjacent macroblock pair has a block effect. In response to the following: the variance of at least one of the two macroblocks is greater than a variance threshold, the absolute value of the difference between the means of the two macroblocks is greater than a first mean threshold, or the absolute value of the difference between the means of the two macroblocks is less than a second mean threshold, it is determined that the adjacent macroblock pair does not have a block effect.

4. The method according to claim 2, characterized in that, Determining whether there is a block effect in the adjacent macroblock pair based on the variance and mean of the two macroblocks respectively includes: Based on the mean values ​​corresponding to the two macroblocks, the target value range is determined; Determine the pixel values ​​of pixels in the boundary regions adjacent to the two macroblocks; In response to the fact that the variances corresponding to the two macroblocks are both less than the variance threshold, the absolute value of the difference between the means corresponding to the two macroblocks is less than the first mean threshold and greater than the second mean threshold, and the pixel values ​​of the pixels in the boundary region are all within the target value range, it is determined that the adjacent macroblock pair has a block effect; In response to the following: the variance of at least one of the two macroblocks is greater than a variance threshold; the absolute value of the difference between the means of the two macroblocks is greater than a first mean threshold; the absolute value of the difference between the means of the two macroblocks is less than a second mean threshold; or there is a pixel value of at least one pixel in the boundary region that is not in the target value range, it is determined that the adjacent macroblock pair does not have block effects.

5. The method according to claim 4, characterized in that, The determination of the target value range based on the mean of the two macroblocks includes: The upper limit of the interval is determined by the sum of the larger mean of the two macroblocks and the target value, and the lower limit of the interval is determined by the difference between the smaller mean of the two macroblocks and the target value.

6. The method according to claim 4, characterized in that, The adjacent boundary region of the two macroblocks is a rectangular region between the boundaries of the two macroblocks and the central region of the two macroblocks, and the central region of the macroblock is a square region centered on the center of the macroblock.

7. The method according to any one of claims 1-6, characterized in that, Before obtaining the pixel values ​​of the pixels in the central region of the macroblock in the decoded video frame, the method further includes: Receive the quantization parameters of the encoded video frame sent by the server; The quantization parameter is determined to be greater than the quantization parameter threshold.

8. A device for detecting block effects, characterized in that, The device includes: The decoding module is used to receive encoded video frames sent by the server and decode the encoded video frames to obtain decoded video frames. The acquisition module is used to acquire the pixel values ​​of pixels in the central region of macroblocks in the decoded video frame; The first determining module is used to determine the number of adjacent macroblock pairs with block artifacts in the decoded video frame based on the pixel values ​​of the pixels in the central region of the macroblock. The second determining module is used to determine the ratio of the number of adjacent macroblock pairs with block effects to the total number of adjacent macroblock pairs in the decoded video frame, as a block effect significance index. A sending module is used to send the block effect saliency index to the server, wherein the block effect saliency index is used to instruct the server to adjust the encoding bitrate of the video stream sent to the terminal based on the block effect saliency index.

9. A computer device, characterized in that, The computer device includes a memory and a processor, the memory being used to store computer instructions; The processor executes computer instructions stored in the memory to cause the computer device to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program code, which, in response to execution by a computer device, performs the method according to any one of claims 1 to 7.

Images