Live picture evaluation method and apparatus, computer device, and storage medium

Abstract

Embodiments of the present application disclose a live picture evaluation method and device, computer equipment and a storage medium. The method comprises the following steps: after starting the code rate adaptation at the streaming end, adjusting the network bandwidth corresponding to the streaming end; then obtaining the live picture displayed by the streaming end under the adjusted network bandwidth; then evaluating the clarity of the live picture to obtain the definition value corresponding to the live picture; finally, according to the definition value, determining whether the streaming end switches the live code rate to the target live code rate corresponding to the adjusted network bandwidth. Thus, the definition of the live picture displayed by the streaming end is detected after the network bandwidth of the streaming end is switched, so that whether the streaming end successfully starts the code rate adaptation effect is accurately detected.

Description

Technical Field

[0001] This application relates to the field of computer technology, specifically to a method, apparatus, computer equipment, and storage medium for evaluating live broadcast footage. Background Technology

[0002] With the development of internet technology, live streaming has become a common form of media dissemination. However, during live streaming, network fluctuations often cause buffering or stuttering in the viewer's browser.

[0003] In related technologies, adaptive algorithms are used to switch the image quality of live streams.

[0004] However, during the process of switching the image quality of the live broadcast, it is often observed with the naked eye, which cannot accurately test the switching effect of the live broadcast's adaptive switching. Summary of the Invention

[0005] This application provides a live stream evaluation method, apparatus, computer device, and storage medium. The live stream evaluation method can accurately detect whether the streaming end has successfully enabled bitrate adaptation after switching network bandwidth.

[0006] The live stream evaluation method provided in this application embodiment is applied to a streaming client with adaptive bitrate enabled, including:

[0007] Adjust the network bandwidth corresponding to the streaming end;

[0008] To capture the live stream displayed on the streaming end under the adjusted network bandwidth;

[0009] The clarity of the live stream is evaluated to obtain the corresponding clarity value.

[0010] Based on the resolution value, determine whether the streaming client should switch the live stream bitrate to the target live stream bitrate corresponding to the adjusted network bandwidth.

[0011] In some implementations, the clarity of the live stream is evaluated to obtain a corresponding clarity value, including:

[0012] Perform grayscale processing on the live stream to obtain a grayscale image corresponding to the live stream;

[0013] Image edge detection is performed on the grayscale image to obtain the sharpness value corresponding to the live broadcast.

[0014] In some implementations, image edge detection is performed on the grayscale image to obtain the sharpness value corresponding to the live broadcast image, including:

[0015] The variance value corresponding to the grayscale image is determined by convolution operation in order to perform image edge detection on the grayscale image;

[0016] The variance value was determined as the resolution value corresponding to the live stream image.

[0017] In some implementations, the clarity of the live stream is evaluated to obtain a corresponding clarity value, including:

[0018] Determine multiple sub-region images corresponding to the live stream screen;

[0019] Determine the sub-resolution value corresponding to each sub-region image;

[0020] The resolution value of the live broadcast is determined based on the sub-resolution value corresponding to each sub-region image.

[0021] In some implementations, determining multiple sub-region images corresponding to the live stream view includes:

[0022] Obtain depth information corresponding to the live stream footage;

[0023] The live stream is divided into multiple sub-regions based on depth information to obtain multiple sub-region images.

[0024] In some implementations, the resolution value of the live stream is determined based on the sub-resolution value corresponding to each sub-region image, including:

[0025] Determine the area corresponding to each sub-region of the image;

[0026] Weights are assigned to the sub-resolution values ​​of each sub-region image based on its area.

[0027] The resolution value of the live stream is determined based on the sub-resolution value and weight value of each sub-region image.

[0028] In some implementations, determining whether the streaming client should switch the live stream bitrate to the target live stream bitrate corresponding to the adjusted network bandwidth, based on the resolution value, includes:

[0029] Determine the target resolution range corresponding to the target live stream bitrate;

[0030] If the resolution value is within the target resolution range, then the streaming end will switch the live stream bitrate to the target live stream bitrate corresponding to the adjusted network bandwidth.

[0031] In some implementations, after adjusting the network bandwidth corresponding to the streaming end, the method further includes:

[0032] At the same time, the first screen displayed on the push streaming end and the second screen displayed on the pull streaming end are obtained. The first screen contains the first time information, and the second screen contains the second time information.

[0033] Text recognition is performed on the first screen to obtain first-time information, and text recognition is performed on the second screen to obtain second-time information;

[0034] Based on the first and second time information, determine the delay at which the pushing end pushes the live stream to the pulling end.

[0035] In some implementations, text recognition is performed on a first screen to obtain first time information, and text recognition is performed on a second screen to obtain second time information, including:

[0036] Identify the first target region in the first frame, and perform text recognition on the image in the first target region to determine the first time information;

[0037] Identify the second target region in the second image, and perform text recognition on the image in the second target region to determine the second time information.

[0038] In some implementations, text recognition is performed on a first screen to obtain first time information, and text recognition is performed on a second screen to obtain second time information, including:

[0039] Convert the text content in the first screen and the text content in the second screen to the first color;

[0040] The background in the first frame and the background in the second frame are converted to a second color, and the first color and the second color are different;

[0041] Text recognition is performed on the first image after color conversion to obtain first-time information, and text recognition is performed on the second image after color conversion to obtain second-time information.

[0042] In some implementations, text recognition is performed on the first image after color conversion to obtain first time information, and text recognition is performed on the second image after color conversion to obtain second time information, including:

[0043] Text recognition is performed on the first image after color conversion to obtain the first text recognition result, and text recognition is performed on the second image after color conversion to obtain the second text recognition result;

[0044] If the first text recognition result contains text recognition errors, the erroneous text recognition results are corrected to obtain the first corrected result;

[0045] Determine the first-time information based on the first correction result; and / or

[0046] If the second text recognition result contains text recognition errors, the erroneous text recognition results are corrected to obtain a second corrected result;

[0047] The second time information is determined based on the second correction result.

[0048] In some implementations, if the first text recognition result contains erroneous text recognition results, the erroneous text recognition results are corrected to obtain a first corrected result, including:

[0049] If the first text recognition result contains partially missing text, the partially missing text is filled in to obtain the complete text, thus obtaining the first correction result.

[0050] The live stream evaluation device provided in this application embodiment is applied to a streaming end with adaptive bitrate enabled, including:

[0051] The adjustment module is used to adjust the network bandwidth corresponding to the streaming end;

[0052] The image acquisition module is used to acquire the live video displayed by the streaming client under the adjusted network bandwidth.

[0053] The resolution acquisition module is used to evaluate the resolution of the live stream and obtain the corresponding resolution value.

[0054] The determination module is used to determine, based on the resolution value, whether the streaming end should switch the live bitrate to the target live bitrate corresponding to the adjusted network bandwidth.

[0055] The computer-readable storage medium provided in the embodiments of this application stores a computer program thereon. When the computer program is run on a computer, it causes the computer to perform the steps in the live broadcast evaluation method provided in any embodiment of this application.

[0056] The computer device provided in this application includes a processor and a memory. The memory contains a computer program, and the processor executes the steps in the live broadcast evaluation method provided in any embodiment of this application by calling the computer program.

[0057] In this embodiment, after enabling bitrate adaptation at the streaming end, the network bandwidth corresponding to the streaming end is adjusted. Then, the live stream image displayed by the streaming end under the adjusted network bandwidth is acquired. The clarity of the live stream image is then evaluated to obtain a corresponding clarity value. Finally, based on the clarity value, it is determined whether the streaming end has switched the live stream bitrate to the target live stream bitrate corresponding to the adjusted network bandwidth. This allows for accurate detection of whether the streaming end has successfully enabled bitrate adaptation by performing clarity detection on the live stream image displayed at the streaming end after switching the network bandwidth. Attached Figure Description

[0058] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0059] Figure 1 This is a schematic diagram of a live streaming scenario provided in an embodiment of this application.

[0060] Figure 2 This is another live streaming scenario diagram provided in the embodiments of this application.

[0061] Figure 3 This is a flowchart illustrating the live stream evaluation method provided in the embodiments of this application.

[0062] Figure 4 This is another flowchart illustrating the live stream evaluation method provided in this application embodiment.

[0063] Figure 5 This is another flowchart illustrating the live stream evaluation method provided in the embodiments of this application.

[0064] Figure 6 These are schematic diagrams of the first and second screens provided in the embodiments of this application.

[0065] Figure 7 This is another flowchart illustrating the live stream evaluation method provided in the embodiments of this application.

[0066] Figure 8 This is a schematic diagram of the text recognition screen provided in the embodiments of this application.

[0067] Figure 9 This is another schematic diagram of the text recognition screen provided in the embodiments of this application.

[0068] Figure 10 This is another schematic diagram of the text recognition screen provided in the embodiments of this application.

[0069] Figure 11 This is another schematic diagram of the live broadcast image evaluation method provided in the embodiments of this application.

[0070] Figure 12 This is a data curve provided in the embodiments of this application.

[0071] Figure 13 This is another data curve provided in the embodiments of this application.

[0072] Figure 14 This is yet another data curve provided in the embodiments of this application.

[0073] Figure 15 This is a schematic diagram of the structure of the live broadcast image evaluation device provided in the embodiments of this application.

[0074] Figure 16 This is a schematic diagram of the structure of the computer device provided in the embodiments of this application. Detailed Implementation

[0075] It should be noted that the principles of this application are illustrated by example in a suitable computing environment. The following description is based on the specific embodiments of this application exemplified, and should not be considered as limiting other specific embodiments not detailed herein. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0076] The relational terms such as "first" and "second" used in the following embodiments of this application are only used to distinguish one object or operation from another, and are not intended to limit the actual order of these objects or operations. In the description of the embodiments of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0077] Please see Figure 1 , Figure 1 This is a schematic diagram of a live streaming scenario provided in an embodiment of this application. For example... Figure 1 As shown, in a live streaming scenario, the pushing client transmits the live video data to the server. The server can process the live video data, such as encoding / decoding and generating video files of multiple resolutions. The server then sends the video files to the pulling client, which, upon receiving the video files, plays them, allowing the live stream to be viewed on the pulling client.

[0078] Please refer to the following: Figure 2 , Figure 2 This is another live streaming scenario diagram provided in the embodiments of this application.

[0079] When the broadcaster (pushing end) pushes the live video data to the server, the server can perform certain processing on the live video data, such as encoding and decoding, and generating video files with multiple resolutions.

[0080] To address the issue of high latency during live streaming, this embodiment of the application utilizes a server to transcode the live video data into multiple TS (Transport Stream) segments, manages them using m3u8 files, and then streams them downstream via the HTTP (Hypertext Transfer Protocol) protocol. The user end (streaming end) then decapsulates, decodes, and plays the video, thereby enabling the user to watch the live stream.

[0081] In some implementations, each resolution has a corresponding TS segment. For example, a video stream may have 7 segments from 1 to 7. 1080p resolution corresponds to 7 TS segments, 720p resolution corresponds to 7 TS segments, and 360p resolution corresponds to 7 TS segments. Furthermore, the TS segments corresponding to each resolution are different.

[0082] During live streaming, the user's network connection may fluctuate, such as changes in network speed and bandwidth. In this situation, the system can adjust the playback resolution based on these network fluctuations. For example, when network conditions are unstable, the user can stream a TS (Translation Lookalike) segment of the video at the appropriate resolution, and then the player can process this segment to play the video at the corresponding resolution. This reduces buffering during the live stream.

[0083] For example, when a user is watching a 720p live stream but the internet speed is poor, it will switch to 480p. If the internet speed is still poor, it will switch to 360p. Conversely, when the internet speed starts to improve, it will switch back to 720p, thereby reducing buffering when watching the live stream.

[0084] In some implementations, to further reduce latency during live streaming, the duration of each TS segment can be reduced. For example, a traditional TS segment is often 5 to 10 seconds long, but the duration of each TS segment can be reduced to 2 seconds. This reduces latency for users watching the live stream.

[0085] In some implementations, a low-latency streaming protocol, such as LHLS (Low Latency HLS), can be used between the client and the server. LHLS (HTTP Live Streaming) is an improvement on the HLS protocol, but it has lower latency.

[0086] The HLS protocol is an HTTP-based streaming media network transmission protocol proposed by Apple. It's part of Apple's QuickTime X and iPhone software systems. Its working principle is to break the entire stream into small HTTP-based files for download, downloading only a few at a time. While the media stream is playing, the client can choose to download the same resource from many different alternative sources at different rates, allowing the streaming session to adapt to different data rates. At the start of a streaming session, the client downloads an extended M3U (m3u8) playlist file containing metadata to locate available media streams.

[0087] Based on the above implementation method, latency during live streaming can be reduced to some extent. However, if the network at the streaming end is unstable, such as due to fluctuations in network bandwidth, stuttering may occur. Therefore, it is necessary to enable adaptive bitrate on the streaming end.

[0088] However, during the process of switching the image quality of the live broadcast, it is often observed with the naked eye, which cannot accurately test the switching effect of the live broadcast's adaptive switching.

[0089] To address this technical problem, embodiments of this application provide a live stream image evaluation method, apparatus, computer device, and storage medium. This can accurately detect whether the streaming end has successfully enabled bitrate adaptation after switching network bandwidth.

[0090] Please see Figure 3 Figure 3 is a flowchart illustrating the live stream evaluation method provided in this embodiment. The live stream evaluation method may include the following steps:

[0091] 110. Adjust the network bandwidth corresponding to the streaming end.

[0092] In some implementations, after enabling adaptive bitrate control on the streaming end, the bandwidth of the network used by the streaming end can be controlled by a network parameter control tool during the display of the live stream on the streaming end.

[0093] For example, during the display of a live stream on the streaming end, the network bandwidth corresponding to the streaming end can be switched at regular intervals, thereby changing the network quality of the streaming end. The computer device can be set with multiple time points, and at each time point from the current time, the network bandwidth corresponding to the streaming end is adjusted. The time interval between any two adjacent time points is greater than a preset duration, thus allowing sufficient time for the streaming end to acquire and display the live stream after each bandwidth switch.

[0094] When adaptive bitrate is enabled on the streaming client, different video qualities will be played depending on the network bandwidth. For example, network bandwidths include 480bps, 600bps, 1500bps, 2500bps, and 8000bps. Among these, 480bps corresponds to 144P resolution, 600bps corresponds to 360P resolution, 1500bps corresponds to 480P resolution, 2500bps corresponds to 720P resolution, and 8000bps corresponds to 1080P resolution.

[0095] 120. Obtain the live stream image displayed by the streaming client under the adjusted network bandwidth.

[0096] In some implementations, when the streaming client uses an adaptive bitrate to play live streams, it can reduce latency and stuttering during playback to some extent. To analyze whether the adaptive bitrate at the streaming client can automatically switch the live stream quality based on fluctuations in network parameters, the computer device needs to capture multiple frames of the live stream during its display and analyze the adaptive quality effect using these multiple frames.

[0097] In some implementations, the computer device can determine the specific time of the network bandwidth switch, and then acquire the live stream displayed on the streaming client under the adjusted network bandwidth 30 seconds after that specific time. This allows the streaming client to finish playing the buffered footage before the network bandwidth switch. For example, the live stream displayed on the streaming client under the adjusted network bandwidth can be acquired 30 seconds after the network bandwidth is adjusted.

[0098] It should be noted that the live stream footage captured by computer equipment can be a single frame or multiple frames.

[0099] 130. Evaluate the clarity of the live stream and obtain the corresponding clarity value.

[0100] In some implementations, the computer device can perform grayscale processing on the live stream to obtain a grayscale image corresponding to the live stream. Then, image edge detection is performed on the grayscale image to obtain the sharpness value corresponding to the live stream.

[0101] For example, if the live stream is a color image, it can be processed into grayscale to obtain a grayscale image. Then, the grayscale value information corresponding to the grayscale image can be obtained to perform image edge detection in order to obtain the corresponding sharpness value of the live stream.

[0102] Specifically, computer equipment can determine the variance value corresponding to a grayscale image through convolution operations to perform image edge detection on the grayscale image; the variance value is then used to determine the sharpness value corresponding to the live broadcast image. For example, computer equipment can obtain the variance corresponding to the live broadcast image by performing a convolution operation on the grayscale image using a Laplacian mask.

[0103] Specifically, a higher variance in the live stream indicates a wider frequency response range, meaning the live stream is clearer. Conversely, a lower variance indicates a narrower frequency response range, meaning there are fewer edges in the live stream, resulting in a blurrier image.

[0104] In some implementations, the computer device can input each frame of the live broadcast into a resolution detection model and output the resolution value corresponding to each frame of the live broadcast. The higher the resolution value, the higher the resolution.

[0105] For example, the sharpness detection model is a trained model that can output a corresponding sharpness value for each frame of the live broadcast. This sharpness value is positively correlated with the sharpness of the live broadcast, that is, the higher the sharpness value, the higher the sharpness of the live broadcast.

[0106] 140. Based on the resolution value, determine whether the streaming end should switch the live bitrate to the target live bitrate corresponding to the adjusted network bandwidth.

[0107] In some implementations, the computer device can determine the target resolution range corresponding to the target live bitrate; if the resolution value is within the target resolution range, it determines that the streaming end will switch the live bitrate to the target live bitrate corresponding to the adjusted network bandwidth.

[0108] For example, under different network bandwidths, the streaming client has a target live bitrate when playing live streams, and this target live bitrate corresponds to a target resolution range.

[0109] After the computer device obtains the resolution value of the live stream after the network bandwidth switch at the streaming end, it can directly determine whether the resolution value is within the target resolution range. If the resolution value is within the target resolution range, the streaming end will switch the live stream bitrate to the target live stream bitrate corresponding to the adjusted network bandwidth.

[0110] In some implementations, the computer device can also obtain the historical live stream images displayed by the streaming terminal before adjusting the network parameters corresponding to the streaming terminal, and then obtain the resolution value of the historical live stream images.

[0111] Computer equipment can compare the resolution values ​​of historical live stream footage with the resolution values ​​of the live stream footage after adjusting the network parameters of the streaming client. This allows the system to determine whether adaptive quality control is enabled on the streaming client.

[0112] For example, if the original network bandwidth of the streaming end is 1500bps, obtain the resolution value of the historical live stream under the 1500bps network bandwidth. Then switch the network bandwidth to 2500bps and obtain the resolution value of the live stream under the 2500bps network bandwidth.

[0113] Comparing these two resolution values, if the resolution of the live stream at 2500bps network bandwidth is higher than the resolution of the historical live stream at 1500bps network bandwidth, it indicates that the streaming client has enabled adaptive quality control. In other words, when the streaming client plays the live stream, it adjusts the bitrate of the live stream after the network bandwidth changes.

[0114] In this embodiment, after enabling bitrate adaptation at the streaming end, the network bandwidth corresponding to the streaming end is adjusted. Then, the live stream image displayed by the streaming end under the adjusted network bandwidth is acquired. The clarity of the live stream image is then evaluated to obtain a corresponding clarity value. Finally, based on the clarity value, it is determined whether the streaming end has switched the live stream bitrate to the target live stream bitrate corresponding to the adjusted network bandwidth. This allows for accurate detection of whether the streaming end has successfully enabled bitrate adaptation by performing clarity detection on the live stream image displayed at the streaming end after switching the network bandwidth.

[0115] For a more detailed understanding of the live stream evaluation method provided in the embodiments of this application, please continue reading. Figure 4 , Figure 4 This is another schematic flowchart of the live stream evaluation method provided in this application embodiment. The live stream evaluation method may include the following steps:

[0116] 201. Adjust the network bandwidth corresponding to the streaming end.

[0117] In some implementations, after enabling adaptive bitrate control on the streaming end, the bandwidth of the network used by the streaming end can be controlled by a network parameter control tool during the display of the live stream on the streaming end.

[0118] For example, during the display of a live stream on the streaming end, the network bandwidth corresponding to the streaming end can be switched at regular intervals, thereby changing the network quality of the streaming end. The computer device can be set with multiple time points, and at each time point from the current time, the network bandwidth corresponding to the streaming end is adjusted. The time interval between any two adjacent time points is greater than a preset duration, thus allowing sufficient time for the streaming end to acquire and display the live stream after each bandwidth switch.

[0119] 202. Obtain the live stream image displayed by the streaming client under the adjusted network bandwidth.

[0120] In some implementations, the computer device can determine the specific time of the network bandwidth switch, and then acquire the live stream displayed on the streaming client under the adjusted network bandwidth 30 seconds after that specific time. This allows the streaming client to finish playing the buffered footage before the network bandwidth switch. For example, the live stream displayed on the streaming client under the adjusted network bandwidth can be acquired 30 seconds after the network bandwidth is adjusted.

[0121] It should be noted that the live stream footage captured by computer equipment can be a single frame or multiple frames.

[0122] 203. Obtain the depth information corresponding to the live broadcast image, and divide the live broadcast image into multiple sub-regions based on the depth information to obtain multiple sub-region images.

[0123] In some implementations, in order to obtain the resolution value of the entire live broadcast more accurately, the computer device can determine multiple sub-region images corresponding to the live broadcast; then determine the sub-resolution value corresponding to each sub-region image; and finally determine the resolution value corresponding to the live broadcast based on the sub-resolution value corresponding to each sub-region image.

[0124] In some implementations, the computer device acquires depth information corresponding to the live stream image. For example, if the live stream image contains subjects at different depth levels, the computer device can use the depth information to determine multiple sub-regions, each containing the corresponding subject. This results in multiple sub-region images.

[0125] 204. Determine the sub-resolution value corresponding to each sub-region image.

[0126] In some implementations, the computer device can perform grayscale processing on the sub-region image to obtain a grayscale image corresponding to the sub-region image. Then, image edge detection is performed on the grayscale image corresponding to the sub-region image to obtain the sub-resolution value corresponding to the sub-region image.

[0127] 205. Determine the area corresponding to each sub-region image, and set a weight value for the sub-resolution value of each sub-region image based on the area.

[0128] In some implementations, the size of different sub-region images has varying impacts on the overall clarity of the live stream. For example, the larger the area of ​​the sub-region image, the greater its impact on the clarity of the overall live stream. Conversely, the smaller the area of ​​the sub-region image, the smaller its impact on the clarity of the overall live stream.

[0129] Computer devices can assign weights to the sub-resolution values ​​of each sub-region image based on its area. For example, if there are three sub-region images, A, B, and C, and the area of ​​the live broadcast is 100, the area of ​​sub-region image A is 50, the area of ​​sub-region image B is 30, and the area of ​​sub-region image C is 20.

[0130] Then the weight of sub-region image A is 0.5, the weight of sub-region image B is 0.3, and the weight of sub-region image C is 0.2.

[0131] 206. Determine the resolution value of the live broadcast image based on the sub-resolution value and weight value of each sub-region image.

[0132] After obtaining the sub-resolution value of each region image, the sub-resolution value of each region image is multiplied by the corresponding weight value, and then the calculation result of each region image is obtained. Finally, the calculation results of each region image are added together to obtain the resolution value corresponding to the live broadcast.

[0133] 207. Determine the target resolution range corresponding to the target live stream bitrate.

[0134] For example, under different network bandwidths, the streaming client has a target live bitrate when playing live streams, and this target live bitrate corresponds to a target resolution range. After the computer device obtains the resolution value of the live stream after the network bandwidth switch at the streaming client, it can directly determine whether the resolution value is within the target resolution range.

[0135] 208. If the resolution value is within the target resolution range, then the streaming end will switch the live bitrate to the target live bitrate corresponding to the adjusted network bandwidth.

[0136] For example, if the target resolution range corresponding to the target live stream bitrate is 15-20, and the resolution value corresponding to the live stream image is 18, then if the resolution value is within the target resolution range, the streaming client will switch the live stream bitrate to the target live stream bitrate corresponding to the adjusted network bandwidth.

[0137] In other words, after this network bandwidth switch, the corresponding bitrate adaptation of the streaming end took effect, successfully switching the picture quality of the live broadcast.

[0138] In this embodiment, the computer device adjusts the network bandwidth corresponding to the streaming endpoint. It then acquires the live stream image displayed by the streaming endpoint under the adjusted network bandwidth. Next, it acquires the depth information corresponding to the live stream image and divides the live stream image into multiple sub-regions based on the depth information to obtain multiple sub-region images.

[0139] Next, determine the sub-resolution value corresponding to each sub-region image. Then, determine the area corresponding to each sub-region image, and assign a weight value to the sub-resolution value of each sub-region image based on the area. Finally, determine the resolution value corresponding to the live stream image based on the sub-resolution value and weight value of each sub-region image.

[0140] Finally, determine the target resolution range corresponding to the target live stream bitrate. If the resolution value is within the target resolution range, then the streaming client will switch the live stream bitrate to the target live stream bitrate corresponding to the adjusted network bandwidth. This accurately detects whether the streaming client has successfully enabled the bitrate adaptation effect.

[0141] Correspondingly, during actual live streaming, there is still a certain delay between the streaming source and the streaming source. Calculating this delay using existing technologies is complex and prone to errors.

[0142] The live stream evaluation method provided in this application can also accurately determine the latency during the live stream. Please refer to the following: Figure 5 , Figure 5 This is another flowchart illustrating the live stream evaluation method provided in this application. The live stream evaluation method may include the following steps:

[0143] 310. At the same time, acquire the first screen displayed on the push streaming end and the second screen displayed on the pull streaming end. The first screen contains the first time information, and the second screen contains the second time information.

[0144] In some implementations, at the same time point, the first screen displayed by the push end and the second screen displayed by the pull end can be acquired simultaneously. Since the first screen and the second screen are acquired at the same time, the delay of the live streaming process from the push end to the pull end can be determined based on the first time information in the first screen and the second time information in the second screen.

[0145] For example, a timer can be set up so that when the timer is triggered, a screenshot is taken of the screen displayed on the pushing end to obtain the first screen. A screenshot is then taken of the screen displayed on the pulling end to obtain the second screen.

[0146] The first screen contains first-time information, and the second screen contains second-time information. The first-time information and the second-time information can be displayed in the corresponding screen in the form of timestamps.

[0147] In some implementations, before the computer device acquires the first screen displayed on the push end and the second screen displayed on the pull end, the following steps are also included:

[0148] Set the corresponding standard time in the real-time display on the streaming end to obtain the live video stream;

[0149] The server processes the live video stream uploaded by the push client and sends the processed live video stream to the pull client for display.

[0150] For example, a clock program can be set up in the streaming client. This clock program is connected to the standard time. When the streaming client is broadcasting live, the clock program can set the corresponding time information, such as a timestamp, on the screen displayed by the streaming client.

[0151] Since the live stream uploaded to the server by the streaming client contains time information, the live stream obtained from the server by the streaming client also contains the corresponding time information.

[0152] In this way, throughout the entire live stream, the computer equipment can simultaneously acquire the first screen displayed by the pushing stream and the second screen displayed by the pulling stream at any given moment. Furthermore, the first screen contains first-time information, and the second screen contains second-time information. This allows the computer equipment to evaluate and analyze the live stream latency between the pushing and pulling streams at any given time.

[0153] It should be noted that the computer device can be independent of the push, pull, and server ends. This computer device is used to analyze the entire live streaming process. For example, this computer device can be the device of the developer corresponding to the live streaming platform.

[0154] 320. Perform text recognition on the first screen to obtain first-time information, and perform text recognition on the second screen to obtain second-time information.

[0155] The computer equipment can use Optical Character Recognition (OCR) to perform text recognition on the first and second screens, thereby obtaining first time information from the first screen and second time information from the second screen.

[0156] Please refer to the following: Figure 6 , Figure 6 These are schematic diagrams of the first and second screens provided in the embodiments of this application.

[0157] Simultaneously, the first frame is captured at the pushing end, and the second frame is captured at the pulling end. Because there is a certain delay in transmitting the live stream from the pushing end to the pulling end, the first time information in the first frame and the second time information in the second frame are different.

[0158] like Figure 6As shown, the first time information in the first screen is 10:50:18:203, while the second time information in the second screen is 10:50:15:303. The first and second time information are different.

[0159] The computer device can perform text recognition on the first screen to obtain first-time information, and perform text recognition on the second screen to obtain second-time information.

[0160] In some implementations, text recognition is performed on a first screen to obtain first time information, and text recognition is performed on a second screen to obtain second time information, including:

[0161] Identify the first target region in the first frame, and perform text recognition on the image in the first target region to determine the first time information;

[0162] Identify the second target region in the second image, and perform text recognition on the image in the second target region to determine the second time information.

[0163] For example, when performing text recognition on the entire first and second screens, the large area of ​​the screen to be recognized will take a long time and will also affect the accuracy of text recognition.

[0164] Therefore, a first target region can be identified in the first frame, containing the text corresponding to the first time information. The computer then performs text recognition on the image within this first target region to obtain the first time information. Similarly, a second target region can be identified in the second frame, containing the text corresponding to the second time information. The computer then performs text recognition on the image within this second target region to obtain the second time information. This improves the efficiency of text recognition and enhances the accuracy of both the acquired first and second time information.

[0165] In some implementations, text recognition is performed on a first screen to obtain first time information, and text recognition is performed on a second screen to obtain second time information, including:

[0166] Convert the text content in the first screen and the text content in the second screen to the first color;

[0167] The background in the first frame and the background in the second frame are converted to a second color, and the first color and the second color are different;

[0168] Text recognition is performed on the first image after color conversion to obtain first-time information, and text recognition is performed on the second image after color conversion to obtain second-time information.

[0169] For example, if the text content and background content in the first frame are different, the computer device can convert the text content and background content to different colors, such as converting the text content to black and the background content to white, so that the text content and background content have high contrast, which is more conducive to the recognition of the text content and thus obtaining the information in real time.

[0170] Similarly, computer devices can convert the text content and background content of the second screen into different colors, thereby giving the text content and background content high contrast, which is more conducive to the recognition of text in the text content, thus obtaining the second time information.

[0171] Please refer to the following: Figure 8 and Figure 9 , Figure 8 This is a schematic diagram of the text recognition screen provided in the embodiments of this application. Figure 9 This is another schematic diagram of the text recognition screen provided in the embodiments of this application.

[0172] in, Figure 8 and Figure 9 The scene in the video can be either the first scene or the second scene. Figure 8 In the image shown, the text and background content are completely different. The computer device can convert the text content to black (the first color) and the background content to white (the second color), thus creating a new image.

[0173] The computer device performs text recognition on the new screen to obtain first-time or second-time information. For example... Figure 9 As shown, the computer device can obtain the time information, namely "10:44:29:001" in the figure, by directly recognizing the text content in the new screen.

[0174] It should be noted that in some implementations, the time information corresponding to the first and / or second frames acquired at the same time may be inaccurate. For example, if the first and second frames are captured by screenshots, the time display in the frames may contain missing text.

[0175] Please refer to the following for details. Figure 10 , Figure 10 This is another schematic diagram of the text recognition screen provided in the embodiments of this application.

[0176] in, Figure 10 The text recognition screen shown is the second screen acquired at the streaming end. In this second screen, there is a clear missing element in the text corresponding to the time. Figure 10The portion shown in the middle box is missing. If this image is used for subsequent text recognition to obtain time information, the recognized time information will most likely be incorrect and unusable.

[0177] Therefore, the second frame can be discarded, and the first frame captured at the same time corresponding to the second frame can also be discarded to prevent errors in calculating live stream latency later. Then, at another time, both the first frame displayed on the pushing end and the second frame displayed on the pulling end can be captured simultaneously.

[0178] In some implementations, text recognition is performed on the first image after color conversion to obtain first time information, and text recognition is performed on the second image after color conversion to obtain second time information, including:

[0179] Text recognition is performed on the first image after color conversion to obtain the first text recognition result, and text recognition is performed on the second image after color conversion to obtain the second text recognition result;

[0180] If the first text recognition result contains text recognition errors, the erroneous text recognition results are corrected to obtain the first corrected result;

[0181] Determine the first-time information based on the first correction result; and / or

[0182] If the second text recognition result contains text recognition errors, the erroneous text recognition results are corrected to obtain a second corrected result;

[0183] The second time information is determined based on the second correction result.

[0184] For example, taking the first image as an example, performing text recognition on the first image after color conversion yields the first text recognition result. However, the first text recognition result may not be accurate. For instance, there may be omissions or misidentifications in the first text recognition result.

[0185] If the first text recognition result contains text recognition errors, the computer device can correct the erroneous text recognition result. For example, if there is missing text in the first text recognition result, the missing text can be filled in to obtain the first corrected result.

[0186] For example, the first text recognition result is "10:4:29:001". Then, the current standard time is determined to be "10:44:49:001". By comparing the first text result with the current standard time, it is found that the time difference of the "minute" unit in "10:4:29:001" is too large. Therefore, the first text recognition result can be corrected to obtain the first corrected result "10:44:29:001".

[0187] If an error occurs in the first text recognition result, it can be corrected. For example, if the first text recognition result is "70:44:29:001", while the actual first time information is "10:44:29:001", and the "1" was recognized as "7" during the text recognition process, the first text recognition result can be corrected to obtain the first corrected result "10:44:29:001".

[0188] It should be noted that the second text recognition result corresponding to the second image can also be corrected using a similar method as the correction of the first text recognition result, thus obtaining the second corrected result.

[0189] In some implementations, after obtaining the first correction result and the second correction result, the computing device can directly determine the text in the first correction result as the first time information and the text in the second correction result as the second time information.

[0190] 330. Based on the first and second time information, determine the delay at which the push end will push the live stream to the pull end.

[0191] In some implementations, before determining the delay at which the pushing end pushes the live stream to the pulling end based on the first time information and the second time information, the computer device can convert the first time information and the second time information to obtain a time in seconds.

[0192] For example, if the time unit for the first time information and the second time information is t0, the minute unit is t1, the second unit is t2, and the microsecond unit is t3, then the calculation rule t0*3600+t1*60+t2+t3 / 1000 can be used to calculate the first time information in seconds. The second time information in seconds can then be calculated.

[0193] The computer equipment can then obtain the absolute value of the difference between the first and second times, thereby determining the delay between the push and pull streams during the live broadcast.

[0194] As described above, determining the latency of the live stream being pushed from the pushing end to the pulling end using this method is relatively simple, and the latency can be obtained at any time during the live stream. Furthermore, the calculated latency is quite accurate and free from error.

[0195] Please see Figure 7 , Figure 7 This is another flowchart illustrating the live stream evaluation method provided in this application embodiment. The live stream evaluation method may further include the following steps:

[0196] 401. At the same time, acquire the first screen displayed on the push streaming end and the second screen displayed on the pull streaming end. The first screen contains the first time information, and the second screen contains the second time information.

[0197] In some implementations, at the same time point, the first screen displayed by the push end and the second screen displayed by the pull end can be acquired simultaneously. Since the first screen and the second screen are acquired at the same time, the delay of the live streaming process from the push end to the pull end can be determined based on the first time information in the first screen and the second time information in the second screen.

[0198] For example, a screenshot is taken of the screen displayed on the pushing end at the same time to obtain the first screen. A screenshot is taken of the screen displayed on the pulling end to obtain the second screen. The first screen contains first time information, and the second screen contains second time information. The first and second time information can be displayed in the corresponding screens in the form of timestamps.

[0199] 402. Convert the text content in the first screen and the text content in the second screen to the first color.

[0200] like Figure 8 As shown, Figure 8 The image shown is either the first or second image. Figure 6 The text content in the image is white, which is not conducive to subsequent text recognition. Therefore, the text content needs to be converted to a primary color. The primary color can be black.

[0201] like Figure 9 As shown, computer equipment can Figure 8 The text content in the image is changed to black.

[0202] 403. Convert the background in the first image and the background in the second image to a second color, so that the first color and the second color are different.

[0203] like Figure 8 As shown, Figure 8 The image shown is either the first or second image. Figure 8 The background of the image is black, which can interfere with subsequent text recognition. Therefore, the background needs to be converted to a second color. The second color can be white.

[0204] like Figure 9 As shown, computer equipment can Figure 8 The background color of the image in the video is changed to white.

[0205] 404. Perform text recognition on the first image after color conversion to obtain the first text recognition result, and perform text recognition on the second image after color conversion to obtain the second text recognition result.

[0206] In some implementations, the computer device can convert the text content and background content of the first screen and the second screen into different colors, thereby giving the text content and background content high contrast, which is more conducive to the recognition of text in the text content, so as to obtain first time information and second time information.

[0207] 405. Determine the first time information based on the first identification result, and determine the second time information based on the second identification result.

[0208] In some implementations, if the first text recognition result contains a text recognition result with an error, the error text recognition result is corrected to obtain a first corrected result;

[0209] Determine the first-time information based on the first correction result; and / or

[0210] If the second text recognition result contains text recognition errors, the erroneous text recognition results are corrected to obtain a second corrected result;

[0211] The second time information is determined based on the second correction result.

[0212] For example, taking the first image as an example, performing text recognition on the first image after color conversion yields the first text recognition result. However, the first text recognition result may not be accurate. For instance, there may be omissions or misidentifications in the first text recognition result.

[0213] If the first text recognition result contains text recognition errors, the computer device can correct the erroneous text recognition result. For example, if there is missing text in the first text recognition result, the missing text can be filled in to obtain the first corrected result.

[0214] For example, the first text recognition result is "10:4:29:001". Then, the current standard time is determined to be "10:44:49:001". By comparing the first text result with the current standard time, it is found that the time difference of the "minute" unit in "10:4:29:001" is too large. Therefore, the first text recognition result can be corrected to obtain the first corrected result "10:44:29:001".

[0215] If an error occurs in the first text recognition result, it can be corrected. For example, if the first text recognition result is "70:44:29:001", while the actual first time information is "10:44:29:001", and the "1" was recognized as "7" during the text recognition process, the first text recognition result can be corrected to obtain the first corrected result "10:44:29:001".

[0216] It should be noted that the second text recognition result corresponding to the second image can also be corrected using a similar method as the correction of the first text recognition result, thus obtaining the second corrected result.

[0217] In some implementations, after obtaining the first correction result and the second correction result, the computing device can directly determine the text in the first correction result as the first time information and the text in the second correction result as the second time information.

[0218] 406. Based on the first and second time information, determine the delay at which the push end will push the live stream to the pull end.

[0219] In some implementations, before determining the delay at which the pushing end pushes the live stream to the pulling end based on the first time information and the second time information, the computer device can convert the first time information and the second time information to obtain a time in seconds.

[0220] For example, if the time unit for the first time information and the second time information is t0, the minute unit is t1, the second unit is t2, and the microsecond unit is t3, then the calculation rule t0*3600+t1*60+t2+t3 / 1000 can be used to calculate the first time information in seconds. The second time information in seconds can then be calculated.

[0221] The computer equipment can then obtain the absolute value of the difference between the first and second times, thereby determining the delay between the push and pull streams during the live broadcast.

[0222] In this embodiment, the computer device simultaneously acquires a first screen displayed on the push streaming end and a second screen displayed on the pull streaming end. The first screen contains first time information, and the second screen contains second time information. Then, the text content in both the first and second screens is converted to a first color, and the backgrounds in both screens are converted to a second color. The first and second colors are different.

[0223] Then, text recognition is performed on the first image after color conversion to obtain the first text recognition result, and text recognition is performed on the second image after color conversion to obtain the second text recognition result. Based on the first recognition result, the first time information is determined, and based on the second recognition result, the second time information is determined. Based on the first and second time information, the delay at which the pushing end pushes the live stream to the pulling end is determined. This accurately determines the delay at which the pushing end pushes the live stream to the pulling end.

[0224] Please see Figure 11 , Figure 11 This is another schematic diagram of the live broadcast image evaluation method provided in the embodiments of this application.

[0225] During the display of a live video stream on the streaming end, the streaming end can save playback data during the display of the live screen, and then report the saved playback data at certain nodes. For example, playback data can be reported when the live stream ends, when the user switches the resolution, when the page is closed, or when an error occurs. This playback data can include various types of data such as the average bitrate, average bandwidth, total playback duration, first frame duration, secondary buffering duration, number of secondary buffering operations, average transcoding latency, average push latency, and average player buffering duration during the display of the live screen on the streaming end.

[0226] In some implementations, during the display of a live video stream on the streaming end, the computer device can acquire playback data uploaded by the streaming end, which is the data corresponding to the display of the live video stream on the streaming end; then the playback data is analyzed and processed to determine abnormal playback data during the display of the live video stream on the streaming end.

[0227] like Figure 11 As shown, during the process of displaying the live video stream at the streaming end, this live video evaluation method may also include the following steps:

[0228] 501. Based on the playback data, determine at least one of the following data during the display of the live video stream: average bitrate, average number of stutters per unit time, and secondary buffering ratio.

[0229] Among them, the average bitrate is the average bitrate corresponding to the entire live broadcast process, the average number of stutters per unit time is the average number of stutters per hour during the live broadcast process, and the secondary buffering ratio is the number of stutters during the live broadcast process divided by the total number of playbacks.

[0230] Computer equipment can filter various types of data in the playback data and then obtain at least one of the following data: average bitrate, average number of stutters per unit time, and secondary buffer ratio.

[0231] For example, computer devices can filter data based on different system platforms, such as filtering playback data for web, Android, and iOS devices, or filtering playback data within a specific time period.

[0232] 502. Generate a data curve based on at least one of the following: average bitrate, average number of stutters per unit time, and secondary buffer ratio.

[0233] Please refer to the following: Figure 12 , Figure 12 This is a data curve provided in the embodiments of this application.

[0234] in, Figure 12 The data curve in the middle is a curve corresponding to the average bitrate during the live broadcast. The vertical axis is the average bitrate, and the horizontal axis is time. This data curve shows the average bitrate of the live broadcast over a certain period of time.

[0235] like Figure 12 As shown, A1 represents the average bitrate under adaptive image quality, and B1 represents the average bitrate under non-adaptive image quality.

[0236] Please refer to the following: Figure 13 , Figure 13 This is yet another data curve provided in the embodiments of this application.

[0237] in, Figure 13 The data curve in the middle is a curve showing the average number of stutters per unit time during a live broadcast. The vertical axis represents the average number of stutters per unit time, and the horizontal axis represents time. This data curve shows the average number of stutters per unit time during a live broadcast over a given period.

[0238] like Figure 13 As shown, A2 represents the average bitrate under adaptive image quality, and B2 represents the average bitrate under non-adaptive image quality.

[0239] Please refer to the following: Figure 14 , Figure 14 This is another data curve provided in the embodiments of this application.

[0240] in, Figure 14 The data curve in the middle is a curve corresponding to the secondary buffering ratio during the live broadcast. The vertical axis is the secondary buffering ratio, and the horizontal axis is time. This data curve shows the secondary buffering ratio of the live broadcast over a certain period of time.

[0241] like Figure 14 As shown, A3 represents the average bitrate under adaptive image quality, and B3 represents the average bitrate under non-adaptive image quality.

[0242] Computer equipment can process playback data to generate data graphs, allowing for a more direct display of abnormal data during the live stream.

[0243] 503. Analyze the corresponding abnormal data in the data curve chart and identify the abnormal data as abnormal playback data.

[0244] In some implementations, the computer device can compare the data corresponding to each date with other data to identify abnormal data.

[0245] like Figure 12 As shown, the data corresponding to the date "Mar 16" is empty, indicating that there may be no playback data for that day, so this data can be identified as abnormal data.

[0246] like Figure 13 As shown, during the live stream, with adaptive quality settings, the average number of stutters on April 22, 2022 was 15.62. With non-adaptive quality settings, the average number of stutters on the same day was 1.84. Logically, the average number of stutters should be lower with adaptive quality and higher with non-adaptive quality. However, the data for April 22, 2022, is exactly the opposite.

[0247] Therefore, the average number of stutters corresponding to "April 22, 2022" can be identified as outlier data.

[0248] like Figure 14 As shown, during the live stream, with adaptive quality, the secondary buffering ratio on April 13, 2022, was 0.85. With non-adaptive quality, the ratio on the same day was 0.11. Logically, with adaptive quality, the average number of stutters should be lower, and with non-adaptive quality, it should be higher. However, the data for April 13, 2022, is exactly the opposite.

[0249] Therefore, the secondary buffer ratio data corresponding to "April 13, 2022" can be identified as outlier data.

[0250] In some implementations, after identifying outliers in the data graph, these outliers can be filtered out, which helps developers analyze them subsequently.

[0251] In this embodiment, by simultaneously acquiring the first screen displayed on the pushing end and the second screen displayed on the pulling end, the latency during the live stream is determined using the time information in the first and second screens. Simultaneously, during the display of the live stream on the pulling end, the adaptive image quality switching effect can be evaluated by assessing the clarity of the live stream before and after switching network parameters on the pulling end. Furthermore, during the display of the live stream on the pulling end, the playback data uploaded by the pulling end can be analyzed to identify abnormal data during the live stream. Ultimately, developers can evaluate the entire live stream process from multiple perspectives, thereby optimizing existing live stream issues in subsequent development.

[0252] Please see Figure 15 , Figure 15 This is a schematic diagram of the structure of the live stream evaluation device provided in this application embodiment. The live stream evaluation device 600 may include:

[0253] The adjustment module 610 is used to adjust the network bandwidth corresponding to the streaming end.

[0254] The image acquisition module 620 is used to acquire the live video displayed by the streaming end under the adjusted network bandwidth.

[0255] The clarity acquisition module 630 is used to evaluate the clarity of the live broadcast image and obtain the corresponding clarity value of the live broadcast image.

[0256] The resolution acquisition module 630 is also used to perform grayscale processing on the live broadcast image to obtain a grayscale image corresponding to the live broadcast image.

[0257] Image edge detection is performed on the grayscale image to obtain the sharpness value corresponding to the live broadcast.

[0258] The sharpness acquisition module 630 is also used to determine the variance value corresponding to the grayscale image through convolution operation, so as to perform image edge detection on the grayscale image;

[0259] The variance value was determined as the resolution value corresponding to the live stream image.

[0260] The resolution acquisition module 630 is also used to determine multiple sub-region images corresponding to the live broadcast image;

[0261] Determine the sub-resolution value corresponding to each sub-region image;

[0262] The resolution value of the live broadcast is determined based on the sub-resolution value corresponding to each sub-region image.

[0263] The resolution acquisition module 630 is also used to acquire depth information corresponding to the live broadcast image;

[0264] The live stream is divided into multiple sub-regions based on depth information to obtain multiple sub-region images.

[0265] The sharpness acquisition module 630 is also used to determine the area corresponding to each sub-region of the image;

[0266] Weights are assigned to the sub-resolution values ​​of each sub-region image based on its area.

[0267] The resolution value of the live stream is determined based on the sub-resolution value and weight value of each sub-region image.

[0268] The determination module 640 is used to determine, based on the resolution value, whether the streaming end should switch the live bitrate to the target live bitrate corresponding to the adjusted network bandwidth.

[0269] The determination module 640 is also used to determine the target resolution range corresponding to the target live stream bitrate;

[0270] If the resolution value is within the target resolution range, then the streaming end will switch the live stream bitrate to the target live stream bitrate corresponding to the adjusted network bandwidth.

[0271] The image acquisition module 620, after adjusting the network bandwidth corresponding to the streaming end, acquires the first image displayed by the streaming end and the second image displayed by the streaming end at the same time. The first image contains the first time information, and the second image contains the second time information.

[0272] Text recognition is performed on the first screen to obtain first-time information, and text recognition is performed on the second screen to obtain second-time information;

[0273] Based on the first and second time information, determine the delay at which the pushing end pushes the live stream to the pulling end.

[0274] The image acquisition module 620 is also used to determine the first target area in the first image, and to perform text recognition on the image in the first target area to determine the first time information;

[0275] Identify the second target region in the second image, and perform text recognition on the image in the second target region to determine the second time information.

[0276] The image acquisition module 620 is also used to convert the text content in the first image and the text content in the second image into a first color;

[0277] The background in the first frame and the background in the second frame are converted to a second color, and the first color and the second color are different;

[0278] Text recognition is performed on the first image after color conversion to obtain first-time information, and text recognition is performed on the second image after color conversion to obtain second-time information.

[0279] The image acquisition module 620 is also used to perform text recognition on the first image after color conversion to obtain a first text recognition result, and to perform text recognition on the second image after color conversion to obtain a second text recognition result.

[0280] If the first text recognition result contains text recognition errors, the erroneous text recognition results are corrected to obtain the first corrected result;

[0281] Determine the first-time information based on the first correction result; and / or

[0282] If the second text recognition result contains text recognition errors, the erroneous text recognition results are corrected to obtain a second corrected result;

[0283] The second time information is determined based on the second correction result.

[0284] The image acquisition module 620 is also used to complete the missing text into complete text if there is some missing text in the first text recognition result, so as to obtain the first correction result.

[0285] During the process of displaying the live stream on the streaming end, the determination module 640 is also used to obtain the playback data corresponding to the streaming end. The playback data is the data corresponding to the process of displaying the live stream on the streaming end.

[0286] The playback data is analyzed and processed to identify any abnormal playback data during the display of the live stream on the streaming end.

[0287] The determination module 640 is also used to determine at least one of the following data based on the playback data: average bitrate, average number of stutters per unit time, and secondary buffer ratio, during the process of displaying the live broadcast image at the streaming end.

[0288] Abnormal playback data is determined based on at least one of the following: average bitrate, average number of stutters per unit time, and secondary buffer ratio.

[0289] The determination module 640 is also used to generate a data curve based on at least one of the following: average bit rate, average number of stutters per unit time, and secondary buffer ratio.

[0290] Analyze the data curves to identify the corresponding abnormal data and determine the abnormal data as abnormal playback data.

[0291] In this embodiment, after enabling bitrate adaptation at the streaming end, the network bandwidth corresponding to the streaming end is adjusted. Then, the live stream image displayed by the streaming end under the adjusted network bandwidth is acquired. The clarity of the live stream image is then evaluated to obtain a corresponding clarity value. Finally, based on the clarity value, it is determined whether the streaming end has switched the live stream bitrate to the target live stream bitrate corresponding to the adjusted network bandwidth. This allows for accurate detection of whether the streaming end has successfully enabled bitrate adaptation by performing clarity detection on the live stream image displayed at the streaming end after switching the network bandwidth.

[0292] In this embodiment, by simultaneously acquiring the first screen displayed on the pushing end and the second screen displayed on the pulling end, the latency during the live stream is determined using the time information in the first and second screens. Simultaneously, during the display of the live stream on the pulling end, the adaptive image quality switching effect can be evaluated by assessing the clarity of the live stream before and after switching network parameters on the pulling end. Furthermore, during the display of the live stream on the pulling end, the playback data uploaded by the pulling end can be analyzed to identify abnormal data during the live stream. Ultimately, developers can evaluate the entire live stream process from multiple perspectives, thereby optimizing existing live stream issues in subsequent development.

[0293] Please refer to Figure 16 , Figure 16 This is a schematic diagram of the structure of a computer device 700 provided in an embodiment of this application.

[0294] The computer device 700 may include components such as a network interface 701, a memory 702, a processor 703, and a screen assembly. Those skilled in the art will understand that... Figure 16 The structure of the computer device 700 shown does not constitute a limitation on the computer device 700, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0295] Network interface 701 can be used for network connections between devices.

[0296] Memory 702 can be used to store computer programs and data. The computer program stored in memory 702 contains executable code. The computer program can be divided into various functional modules. Processor 703 executes various functional applications and data processing by running the computer program stored in memory 702.

[0297] The processor 703 is the control center of the computer device 700. It connects various parts of the computer device 700 through various interfaces and lines. By running or executing computer programs stored in the memory 702 and calling data stored in the memory 702, it performs various functions of the computer device 700 and processes data, thereby controlling the computer device 700 as a whole.

[0298] In this embodiment, the processor 703 in the computer device 700 loads executable code corresponding to one or more computer programs into the memory 702 according to the following instructions, and the processor 703 executes the steps in the live broadcast image evaluation method provided in this application, such as:

[0299] Adjust the network bandwidth corresponding to the streaming end;

[0300] To capture the live stream displayed on the streaming end under the adjusted network bandwidth;

[0301] The clarity of the live stream is evaluated to obtain the corresponding clarity value.

[0302] Based on the resolution value, determine whether the streaming client should switch the live stream bitrate to the target live stream bitrate corresponding to the adjusted network bandwidth.

[0303] It should be noted that the computer device 700 provided in this application embodiment and the live broadcast image evaluation method in the above embodiment belong to the same concept. The specific implementation process can be found in the above related embodiments, and will not be repeated here.

[0304] This application also provides a computer-readable storage medium storing a computer program thereon. When the computer program stored thereon is executed on the processor of the computer device provided in the embodiments of this application, the processor of the computer device performs any of the steps in the live broadcast image evaluation method suitable for the computer device described above. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0305] This application also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps in the live broadcast image evaluation method provided in this application, such as:

[0306] Adjust the network bandwidth corresponding to the streaming end;

[0307] To capture the live stream displayed on the streaming end under the adjusted network bandwidth;

[0308] The clarity of the live stream is evaluated to obtain the corresponding clarity value.

[0309] Based on the resolution value, determine whether the streaming client should switch the live stream bitrate to the target live stream bitrate corresponding to the adjusted network bandwidth.

[0310] It should be noted that the computer program product provided in this application embodiment and the live broadcast image evaluation method in the above embodiment belong to the same concept. The specific implementation process can be found in the above related embodiments, and will not be repeated here.

[0311] The above provides a detailed description of a live broadcast image evaluation method, apparatus, computer device, and storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A live stream image evaluation method, applied to a streaming client with adaptive bitrate enabled, characterized in that, include: Adjust the network bandwidth corresponding to the streaming endpoint; Acquire the live stream displayed by the streaming end under the adjusted network bandwidth; The clarity of the live stream is evaluated to obtain a corresponding clarity value. Determine the target resolution range corresponding to the target live bitrate. Under different network bandwidths, the streaming end has a target live bitrate when playing the live stream, and this target live bitrate corresponds to a target resolution range. If the resolution value is within the target resolution range, then the streaming end will switch the live stream bitrate to the target live stream bitrate corresponding to the adjusted network bandwidth.

2. The live broadcast image evaluation method according to claim 1, characterized in that, The process of evaluating the clarity of the live stream image to obtain a corresponding clarity value includes: The live stream image is processed to obtain a grayscale image corresponding to the live stream image. Image edge detection is performed on the grayscale image to obtain the sharpness value corresponding to the live broadcast.

3. The live broadcast image evaluation method according to claim 2, characterized in that, The step of performing image edge detection on the grayscale image to obtain the sharpness value corresponding to the live broadcast image includes: The variance value corresponding to the grayscale image is determined by convolution operation, so as to perform image edge detection on the grayscale image; The variance value is determined as the resolution value corresponding to the live stream image.

4. The live broadcast image evaluation method according to claim 1, characterized in that, The process of evaluating the clarity of the live stream image to obtain a corresponding clarity value includes: Determine multiple sub-region images corresponding to the live stream screen; Determine the sub-resolution value corresponding to each of the sub-region images; The resolution value of the live broadcast image is determined based on the sub-resolution value corresponding to each sub-region image.

5. The live broadcast image evaluation method according to claim 4, characterized in that, The step of determining the multiple sub-region images corresponding to the live broadcast screen includes: Obtain the depth information corresponding to the live stream image; The live stream is divided into multiple sub-regions based on the depth information to obtain images of the multiple sub-regions.

6. The live broadcast image evaluation method according to claim 4, characterized in that, Determining the resolution value of the live stream based on the sub-resolution value corresponding to each sub-region image includes: Determine the area corresponding to each of the sub-region images; A weight value is set for the sub-resolution value of each sub-region image based on the area; The resolution value corresponding to the live broadcast image is determined based on the sub-resolution value and weight value of each sub-region image.

7. The live broadcast image evaluation method according to any one of claims 1 to 6, characterized in that, After adjusting the network bandwidth corresponding to the streaming end, the method further includes: At the same time, the first screen displayed by the push streaming end and the second screen displayed by the pull streaming end are acquired. The first screen contains first time information, and the second screen contains second time information. Text recognition is performed on the first image to obtain the first time information, and text recognition is performed on the second image to obtain the second time information; Based on the first time information and the second time information, the delay at which the pushing end pushes the live stream to the pulling end is determined.

8. The live broadcast image evaluation method according to claim 7, characterized in that, The step of performing text recognition on the first image to obtain the first time information, and performing text recognition on the second image to obtain the second time information, includes: A first target region in the first image is determined, and text recognition is performed on the image in the first target region to determine the first time information; A second target region in the second image is identified, and text recognition is performed on the image in the second target region to determine the second time information.

9. The live broadcast image evaluation method according to claim 7, characterized in that, The step of performing text recognition on the first image to obtain the first time information, and performing text recognition on the second image to obtain the second time information, includes: The text content in the first screen and the text content in the second screen are converted to a first color; The backgrounds in the first and second images are converted to a second color, and the first and second colors are different. Text recognition is performed on the first image after color conversion to obtain the first time information, and text recognition is performed on the second image after color conversion to obtain the second time information.

10. The live broadcast image evaluation method according to claim 9, characterized in that, The step of performing text recognition on the first image after color conversion to obtain the first time information, and performing text recognition on the second image after color conversion to obtain the second time information, includes: Text recognition is performed on the first image after color conversion to obtain a first text recognition result, and text recognition is performed on the second image after color conversion to obtain a second text recognition result; If the first text recognition result contains a text recognition result with an error, the error text recognition result is corrected to obtain a first corrected result; Determine the first time information based on the first correction result; and / or If the second text recognition result contains a text recognition result with an error, the error text recognition result is corrected to obtain a second corrected result; The second time information is determined based on the second correction result.

11. The live broadcast image evaluation method according to claim 10, characterized in that, If the first text recognition result contains erroneous text recognition results, the erroneous text recognition results are corrected to obtain a first corrected result, including: If the first text recognition result contains partially missing text, the partially missing text is filled in to form complete text, so as to obtain the first correction result.

12. A live stream evaluation device, applied to a streaming terminal with adaptive bitrate enabled, characterized in that, include: The adjustment module is used to adjust the network bandwidth corresponding to the streaming end; The image acquisition module is used to acquire the live video displayed by the streaming end under the adjusted network bandwidth. The clarity acquisition module is used to evaluate the clarity of the live stream and obtain the clarity value corresponding to the live stream. The determination module is used to determine the target resolution range corresponding to the target live bitrate. Under different network bandwidths, the streaming end has a target live bitrate when playing live, and the target live bitrate corresponds to a target resolution range. If the resolution value is within the target resolution range, then the streaming end is determined to switch the live bitrate to the adjusted target live bitrate corresponding to the network bandwidth.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is run on a computer, it causes the computer to perform the steps in the live broadcast evaluation method as described in any one of claims 1 to 11.

14. A computer device comprising a processor and a memory, the memory storing a computer program, characterized in that, The processor invokes the computer program to perform the steps in the live broadcast evaluation method as described in any one of claims 1 to 11.

Images