Live broadcast optimization method, device, equipment and storage medium

By acquiring image quality information from live stream frames for anomaly detection and rendering log analysis, the problem of quickly repairing live stream image quality anomalies was solved, improving the stability and display effect of live stream image quality.

CN117615211BActive Publication Date: 2024-11-05SHUXING TECH (BEIJING) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202311656917.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-05
Publication Date
2024-11-05
Estimated Expiration
2043-12-05

AI Technical Summary

Technical Problem

In existing technologies, abnormal live stream quality cannot be quickly identified and repaired, affecting the live stream display effect.

Method used

By acquiring image quality information from live stream frames, abnormal images are detected, abnormal rendering stages are located, and optimization is performed based on rendering log information, including optimization strategies for the shooting, rendering, and encoding modules, to improve image quality.

Benefits of technology

It enables the rapid and accurate identification of abnormal image quality factors during live streaming and allows for targeted optimization, thereby improving the stability of image quality during live streaming.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117615211B_ABST
    Figure CN117615211B_ABST
Patent Text Reader

Abstract

This application provides a live streaming optimization method, apparatus, device, and storage medium. The method includes: acquiring live streaming frames displayed in a target live streaming room to be detected, and live streaming quality information of the live streaming frames; performing abnormal image detection on the live streaming frames in the target live streaming room based on the live streaming quality information to determine abnormal frames in the target live streaming room; acquiring rendering log information of each live streaming rendering stage associated with the abnormal frames, comparing the rendering log information of each live streaming rendering stage, and locating the abnormal rendering stage of the target live streaming room; and optimizing the abnormal rendering stage to obtain a quality optimization result. The technical solution of this application can accurately and quickly locate abnormal quality factors in the live streaming process, and specifically optimize the abnormal rendering stage corresponding to the abnormal quality factors, thereby improving the stability of image quality during the live streaming process.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of live broadcast technology, and in particular to a live broadcast optimization method, device, equipment and storage medium. Background Art

[0002] At present, with the rapid development of Internet technology and live broadcast technology, more and more users (i.e. anchors or bloggers) are broadcasting live through live broadcast applications in smart terminals such as mobile phones, tablets and smart computers to share games / talent shows / products and other programs with the audience in real time. However, when the anchor is broadcasting live, the picture needs to go through multiple live broadcast rendering links from shooting to rendering in the live broadcast room. During this rendering process, the live broadcast room image quality may be abnormal due to an abnormal live broadcast rendering link. The anchor or related personnel cannot quickly and effectively locate the abnormal rendering link and repair it, which affects the live broadcast display effect. Summary of the invention

[0003] The embodiments of the present application provide a live broadcast optimization method, device, equipment and storage medium, which are intended to solve the technical problem in the prior art that it is impossible to quickly troubleshoot and repair the abnormal live broadcast quality when the live broadcast quality in the live broadcast room is abnormal.

[0004] On the one hand, an embodiment of the present application provides a live broadcast optimization method, the live broadcast optimization method comprising the following steps:

[0005] Acquire a live picture frame displayed in the target live broadcast room to be detected, and live picture quality information of the live picture frame;

[0006] Perform abnormal image detection on the live picture frames in the target live broadcast room according to the live picture quality information, and determine the abnormal picture frames in the target live broadcast room;

[0007] Acquire rendering log information of each live rendering stage associated with the abnormal picture frame, compare the rendering log information of each live rendering stage, and locate the abnormal rendering stage of the target live room;

[0008] The abnormal rendering stage is optimized to obtain an image quality optimization result.

[0009] In a possible implementation of the present application, the live broadcast rendering stage includes a first live broadcast rendering stage and a second live broadcast rendering stage, and the first live broadcast rendering stage is a previous live broadcast rendering stage of the second live broadcast rendering stage;

[0010] The obtaining of rendering log information associated with the abnormal picture frame in each live rendering stage, comparing the rendering log information of each live rendering stage, and locating the abnormal rendering stage of the target live room includes:

[0011] Acquire first rendering log information associated with the abnormal picture frame in a first live broadcast rendering phase in the target live broadcast room, and second rendering log information associated with the abnormal picture frame in a second live broadcast rendering phase;

[0012] Calculating a first image quality evaluation parameter of a first rendered image in the first rendering log information and a second image quality evaluation parameter of a second rendered image in the second rendering log;

[0013] If the second image quality evaluation parameter is lower than the first image quality evaluation parameter and lower than a preset image quality evaluation threshold, the second live broadcast rendering stage is set as an abnormal rendering stage.

[0014] In a possible implementation of the present application, obtaining rendering log information associated with the abnormal picture frame in each live rendering stage, comparing the rendering log information of each live rendering stage, and locating the abnormal rendering stage of the target live room includes:

[0015] Acquire shooting log information of the image shooting stage of the target live broadcast room, wherein the shooting log information includes a target shot image, and image noise data and image sharpening data corresponding to the target shot image;

[0016] If the image noise data is greater than a preset noise threshold and / or the image sharpening data is greater than a preset sharpening parameter threshold, the image capture stage is set as an abnormal rendering stage of the target live broadcast room.

[0017] In a possible implementation of the present application, the optimizing process for the abnormal rendering stage to obtain an image quality optimization result includes:

[0018] Acquire an abnormal link module corresponding to the abnormal rendering stage, wherein the abnormal link module includes any one or more of an abnormal shooting module, an abnormal rendering module and an abnormal encoding module;

[0019] The image quality optimization strategy corresponding to the abnormal link module is obtained, and the abnormal rendering stage is optimized based on the image quality optimization strategy to obtain an image quality optimization result.

[0020] In a possible implementation of the present application, the acquiring the image quality optimization strategy corresponding to the abnormal link module, optimizing the abnormal rendering stage based on the image quality optimization strategy, and obtaining the image quality optimization result includes:

[0021] If the abnormal link module is an abnormal shooting module, determining that the image quality optimization strategy is a shooting optimization strategy;

[0022] Optimizing the abnormal shooting module based on the focus optimization parameters, white balance optimization parameters and / or camera switching parameters in the shooting optimization strategy to obtain an optimized shooting module;

[0023] The optimized shooting image captured by the optimized shooting module is obtained, and the image quality optimization result of the optimized shooting module is output according to the optimized shooting image.

[0024] In a possible implementation of the present application, the acquiring the image quality optimization strategy corresponding to the abnormal link module, optimizing the abnormal rendering stage based on the image quality optimization strategy, and obtaining the image quality optimization result includes:

[0025] If the abnormal link module is an abnormal rendering module, determining that the image quality optimization strategy is a rendering optimization strategy;

[0026] Identifying an abnormal beautification item associated with the abnormal picture frame in the abnormal rendering module, and an abnormal beautification parameter of the abnormal beautification item;

[0027] The abnormal beautification parameters are adjusted based on the target beautification parameters of the target beautification project in the rendering optimization strategy to obtain an optimized rendering module;

[0028] An optimized rendering image rendered by the optimized rendering module is obtained, and a picture quality optimization result of the optimized rendering module is determined according to the optimized rendering image.

[0029] In a possible implementation of the present application, determining the image quality optimization result of the optimized rendering module according to the optimized rendered image includes:

[0030] If the image quality evaluation parameter of the optimized rendering image is lower than a preset image quality evaluation threshold, obtaining a rendering update package of the optimized rendering module;

[0031] The optimized rendering module is updated based on the rendering update package to obtain an updated rendering module;

[0032] An updated rendering image of an updated rendering module is obtained, and a picture quality optimization result corresponding to the updated rendering image is output.

[0033] In a possible implementation of the present application, the acquiring the image quality optimization strategy corresponding to the abnormal link module, optimizing the abnormal rendering stage based on the image quality optimization strategy, and obtaining the image quality optimization result includes:

[0034] If the abnormal link module is an abnormal encoding module, determining that the image quality optimization strategy is an encoding optimization strategy;

[0035] Optimizing the abnormal coding module based on the target coding strategy and the target coding format in the coding optimization strategy to obtain an optimized coding module;

[0036] The optimized coded image encoded by the optimized coding module is obtained, and a picture quality optimization result corresponding to the optimized coded image is output.

[0037] On the other hand, the present application provides a live broadcast optimization device, the live broadcast optimization device comprising:

[0038] An information acquisition module is configured to acquire a live picture frame displayed in a target live broadcast room to be detected, and live picture quality information of the live picture frame;

[0039] an abnormality identification module, configured to perform abnormal image detection on the live picture frames in the target live room according to the live picture quality information, and determine the abnormal picture frames in the target live room;

[0040] A link positioning module is configured to obtain rendering log information of each live rendering stage associated with the abnormal picture frame, compare the rendering log information of each live rendering stage, and locate the abnormal rendering stage of the target live room;

[0041] The live broadcast optimization module is configured to optimize the abnormal rendering stage to obtain a picture quality optimization result.

[0042] On the other hand, the present application also provides a live broadcast optimization device, the live broadcast optimization device comprising:

[0043] one or more processors;

[0044] Memory; and

[0045] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the steps of the live broadcast optimization method.

[0046] On the other hand, the present application also provides a computer-readable storage medium on which a computer program is stored, and the computer program is loaded by a processor to execute the steps in the live broadcast optimization method.

[0047] In this application, by obtaining the live picture frame displayed in the target live broadcast room and the live picture quality information in the live picture frame, and performing image quality detection on the live picture frame according to the live picture quality information, it is determined whether the live picture frame is an abnormal picture frame, and when it is determined that there is an abnormal picture frame in the target live broadcast room, the rendering log information of each live rendering stage associated with the abnormal picture frame in the live rendering link is obtained, and the rendering log information of each live rendering stage is compared to determine the abnormal rendering stage of the target live broadcast room, and the abnormal rendering stage is optimized to obtain the picture quality optimization result. It is possible to accurately and quickly locate the abnormal picture quality factors in the live broadcast process, and to optimize the abnormal rendering stage corresponding to the abnormal picture quality factors in a targeted manner, so as to improve the picture quality stability during the live broadcast process. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0049] Figure 1 A schematic diagram of a scenario of a live broadcast optimization method according to an embodiment of the present application;

[0050] Figure 2 A flowchart of an embodiment of the live broadcast optimization method in the embodiment of the present application;

[0051] Figure 3a A schematic diagram of a scene of an embodiment of each live broadcast rendering stage in a live broadcast rendering link of a target live broadcast room in a live broadcast optimization method provided in an embodiment of the present application;

[0052] Figure 3b A schematic diagram of a scene of a specific embodiment of rendering each live picture frame by a live broadcast rendering link of a target live broadcast room provided in an embodiment of the present application;

[0053] Figure 4 A timing diagram of an embodiment of the live broadcast optimization method provided in an embodiment of the present application;

[0054] Figure 5 A flow chart of an embodiment of optimizing the image quality of an abnormal shooting module in the live broadcast optimization method provided in an embodiment of the present application;

[0055] Figure 6 A flow chart of an embodiment of optimizing the image quality of an abnormal rendering module in the live broadcast optimization method provided in an embodiment of the present application;

[0056] Figure 7A schematic diagram of the structure of an embodiment of a live broadcast optimization device provided in an embodiment of the present application;

[0057] Figure 8 A structural diagram of an embodiment of a live broadcast optimization device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0059] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0060] In this application, the word "exemplary" is used to mean "serving as an example, illustration, or illustration." Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any person skilled in the art to implement and use the invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the invention can be implemented without using these specific details. In other instances, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in this application.

[0061] At present, with the rapid development of Internet technology and live broadcast technology, more and more users (i.e. anchors or bloggers) are broadcasting live through live broadcast applications in smart terminals such as mobile phones, tablets and smart computers to share games / talent shows / products and other programs with the audience in real time. However, when the anchor is broadcasting live, the picture needs to go through multiple live broadcast rendering links from shooting to rendering in the live broadcast room. During this rendering process, the live broadcast room image quality may be abnormal due to an abnormal live broadcast rendering link. The anchor or related personnel cannot quickly and effectively locate the abnormal rendering link and repair it, which affects the live broadcast display effect.

[0062] Based on this, the present application proposes a live broadcast optimization method, device, equipment and computer-readable storage medium to solve the technical problem in the prior art that when the live broadcast quality in the live broadcast room is abnormal, it is impossible to quickly troubleshoot and repair the abnormal live broadcast quality.

[0063] The live broadcast optimization method in the embodiment of the present invention is applied to a live broadcast optimization device, which is arranged in the live broadcast optimization device. The live broadcast optimization device is provided with one or more processors, memories, and one or more applications, wherein the one or more applications are stored in the memories and configured to be executed by the processor to implement the live broadcast optimization method; wherein the live broadcast optimization device can be an intelligent terminal, such as a mobile phone, a tablet computer, a network device, and a smart computer; optionally, the live broadcast optimization device can also be a server, or a service cluster composed of multiple servers.

[0064] like Figure 1 As shown, Figure 1 The following is a scene diagram of the live broadcast optimization method according to an embodiment of the present application. The live broadcast optimization scene according to the embodiment of the present invention includes a live broadcast optimization device 100 (a live broadcast optimization device is integrated in the live broadcast optimization device 100) and a target service terminal 200. A computer-readable storage medium corresponding to the live broadcast optimization method is run in the live broadcast optimization device 100 to execute the steps of the live broadcast optimization method. The target service terminal 200 is an intelligent terminal that communicates and interacts with the live broadcast optimization device 100 and is installed with a live broadcast client. Optionally, in a specific embodiment, the live broadcast optimization device 100 and the target service terminal can be the same intelligent terminal.

[0065] Understandably, Figure 1 The live broadcast optimization equipment in the live broadcast optimization method scenario shown, or the devices included in the live broadcast optimization equipment, does not constitute a limitation on the embodiments of the present invention, that is, the number of devices and types of devices of the live broadcast optimization equipment included in the scenario of the live broadcast optimization method, or the number of devices and types of devices included in each device do not affect the overall implementation of the technical solution in the embodiments of the present invention, and can all be regarded as equivalent replacements or derivatives of the technical solution claimed to be protected by the embodiments of the present invention.

[0066] The live broadcast optimization device 100 in the embodiment of the present invention is mainly used to: obtain the live broadcast picture frames displayed in the target live broadcast room to be detected, and the live broadcast picture quality information of the live broadcast picture frames; perform abnormal image detection on the live broadcast picture frames in the target live broadcast room according to the live broadcast picture quality information, and determine the abnormal picture frames in the target live broadcast room; obtain the rendering log information of each live broadcast rendering stage associated with the abnormal picture frame, compare the rendering log information of each live broadcast rendering stage, and locate the abnormal rendering stage of the target live broadcast room; optimize the abnormal rendering stage to obtain the picture quality optimization result.

[0067] The live broadcast optimization device 100 in the embodiment of the present invention can be an independent live broadcast optimization device, such as a smart terminal such as a mobile phone, a tablet computer, a network device, a server, and a smart computer, or it can be a live broadcast optimization network or a live broadcast optimization cluster composed of multiple live broadcast optimization devices.

[0068] The embodiments of the present application provide a live broadcast optimization method, apparatus, device, and computer-readable storage medium, which are described in detail below.

[0069] It can be understood by those skilled in the art that Figure 1 The application environment shown in is only one of the application scenarios related to the present application solution and does not constitute a limitation on the application scenario of the present application solution. Other application environments may also include Figure 1 More or fewer live broadcast optimization devices, or live broadcast optimization network connection relationships shown, such as Figure 1 Only one live broadcast optimization device is shown. It can be understood that the scenario of the live broadcast optimization method can also include one or more live broadcast optimization devices, which are not specifically limited here. The live broadcast optimization device 100 can also include a memory for storing live broadcast image data and other data.

[0070] It should be noted that Figure 1 The scenario diagram of the live broadcast optimization method shown is only an example. The scenario of the live broadcast optimization method described in the embodiment of the present invention is to more clearly illustrate the technical solution of the embodiment of the present invention, and does not constitute a limitation on the technical solution provided by the embodiment of the present invention.

[0071] Based on the scenario of the above live broadcast optimization method, various embodiments of the live broadcast optimization method disclosed in the present invention are proposed.

[0072] like Figure 2 As shown, Figure 2 This is a flow chart of an embodiment of the live broadcast optimization method in the embodiment of the present application. The live broadcast optimization method includes the following steps 201 to 204:

[0073] 201. Acquire a live picture frame displayed in a target live broadcast room to be detected, and live picture quality information of the live picture frame;

[0074] The live broadcast optimization method in this embodiment is applied to live broadcast optimization equipment. The type and number of live broadcast optimization equipment are not specifically limited. That is, the live broadcast optimization equipment can be one or more intelligent terminals or servers that perform image quality detection and image quality optimization on the target live broadcast room in the target business terminal installed with the target live broadcast application. In a specific embodiment, the live broadcast optimization equipment is a live broadcast server (including streaming media servers, cloud servers, and physical servers, etc.), and the target business terminal is a live broadcast intelligent device (such as smart terminals such as mobile phones, tablets, and personal computers). In other embodiments, the live broadcast optimization equipment can also be the same intelligent device as the target business terminal. That is, a certain intelligent device can be a live broadcast optimization device or a target business terminal for live broadcast.

[0075] The target live broadcast room is a virtual space that transmits audio and video content in real time through the Internet in the target live broadcast application. The anchor user can use the target live broadcast room to share games / talent shows / product promotion programs in real time. The audience user can enter the target live broadcast room to watch the audio and video program content displayed by the anchor user.

[0076] Specifically, during operation, the live broadcast optimization device responds to the live broadcast optimization event of the target live broadcast room, wherein the live broadcast optimization event is an operation instruction event that drives the live broadcast optimization device to detect the live broadcast picture frame of the target live broadcast room and optimize the picture quality when the live broadcast picture frame has abnormal picture quality. Optionally, the triggering method of the live broadcast optimization event is not specifically limited here, that is, the live broadcast optimization event can be triggered by a user. For example, in a specific embodiment, the user is an anchor user, and during the live broadcast of the target live broadcast room, the live broadcast optimization button in the target live broadcast application is clicked by a control method such as a mouse, keyboard, controller or touch control to actively trigger the live broadcast optimization event. In another specific embodiment, the user can also be an audience user, and during the live broadcast of the target live broadcast room, the live broadcast optimization button in the target live broadcast application is clicked by a control method such as a mouse, keyboard, controller or touch control to actively trigger the live broadcast optimization event. Optionally, the live broadcast optimization event can also be actively triggered by the live broadcast optimization device. For example, a live broadcast detection process is pre-set in the live broadcast optimization device, and the live broadcast detection process is used to perform real-time picture quality detection on the target live broadcast room in the broadcast, and the live broadcast optimization event is automatically triggered when the picture quality is abnormal.

[0077] Specifically, after acquiring the live broadcast optimization event, the live broadcast optimization device responds to the live broadcast optimization event, analyzes the live broadcast optimization event, determines the target live broadcast room to be detected to which the live broadcast optimization event points, collects the live broadcast picture frames displayed by the target live broadcast room, and obtains the live broadcast picture quality information of the live broadcast picture frames. Among them, the live broadcast picture frame is a sequence of picture frames currently displayed and / or already displayed by the target live broadcast room in the current live broadcast. The live broadcast picture quality information is a picture quality parameter that characterizes the picture frame quality state of each live broadcast picture frame. Optionally, in a specific embodiment, the live broadcast picture quality information includes noise information, blur information, sharpening information, etc. In other embodiments, in order to improve the accuracy of live broadcast picture quality detection, the live broadcast picture quality information can also include picture quality parameters of more dimensions. For example, in a specific embodiment, the live broadcast picture quality information also includes beautification parameters such as skin grinding amplitude, whitening amplitude, color level amplitude, and contrast amplitude.

[0078] 202. Perform abnormal image detection on the live picture frames in the target live broadcast room according to the live broadcast picture quality information to determine the abnormal picture frames in the target live broadcast room;

[0079] Specifically, after the live broadcast optimization device obtains the live broadcast picture frames of the target live broadcast room and the live broadcast picture quality information corresponding to each live broadcast picture frame, it also performs abnormal image detection on the live broadcast picture frames in the target live broadcast room according to the live broadcast picture quality information, thereby determining the abnormal picture frames of the target live broadcast room. The abnormal picture frames are live broadcast picture frames whose live broadcast picture quality is determined to be abnormal after abnormal image detection.

[0080] Among them, the abnormal image detection operation is to drive the live broadcast optimization device to perform image quality detection on the live broadcast picture frame according to a preset image quality detection algorithm. Optionally, the image quality detection algorithm may include manual detection, structural similarity index detection, peak signal-to-noise ratio detection, visual perception quality detection, VQA (Visual Quality Assessment) quality detection and other image quality detection algorithms.

[0081] Specifically, the live broadcast optimization device performs image quality detection on each live broadcast picture frame based on the live broadcast picture quality information of the live broadcast picture frame, that is, the live broadcast picture quality evaluation parameters of each live broadcast picture frame are calculated by the picture quality parameters in the live broadcast picture quality information and the image quality detection algorithm, and the live broadcast picture frame is determined to be an abnormal picture frame by the preset picture quality evaluation thresholds corresponding to the live broadcast picture quality evaluation parameters and each picture quality detection algorithm. Among them, the live broadcast picture quality evaluation parameter is an indicator parameter calculated based on the live broadcast picture quality information and the image detection algorithm, and is used to characterize the picture quality of the live broadcast picture frame. The picture quality evaluation threshold is the indicator parameter threshold that characterizes the image quality in each image detection algorithm.

[0082] The live broadcast quality evaluation parameters may include a structural similarity index (SSIM) and a peak signal-to-noise ratio (PSNR).

[0083] Specifically, after acquiring the live picture quality evaluation parameters of each live picture frame, the live picture quality evaluation parameters are compared with the picture quality evaluation parameters, so as to determine whether the live picture frame is an abnormal picture frame according to the comparison result.

[0084] Optionally, if the live broadcast image quality evaluation parameter is greater than a preset image quality evaluation threshold, the live broadcast optimization device determines that the live broadcast picture frame corresponding to the live broadcast image quality evaluation parameter is a normal picture frame, that is, the live broadcast image quality of the target live broadcast room is normal.

[0085] Optionally, if the live broadcast image quality evaluation parameter is less than a preset image quality evaluation threshold, the live broadcast optimization device determines that the live broadcast picture frame corresponding to the live broadcast image quality evaluation parameter is an abnormal picture frame, and the live broadcast optimization device determines that there is an abnormality in the live broadcast rendering link of the target live broadcast room.

[0086] Optionally, in other embodiments, the live broadcast optimization device is also communicatively connected to a preset streaming media server, in which a picture quality detection model is preset. The picture quality detection model is used to collect live screen frames displayed in the target live broadcast room in real time, and VQA picture quality detection is performed on the live screen frames, thereby determining abnormal picture frames in the target live broadcast room.

[0087] 203. Obtain rendering log information of each live rendering stage associated with the abnormal picture frame, compare the rendering log information of each live rendering stage, and locate the abnormal rendering stage of the target live room;

[0088] Specifically, when the live broadcast optimization device detects that the image quality of the target live broadcast room is abnormal, it retrieves the rendering log information of the live broadcast rendering stage associated with the abnormal picture frame, and performs rendering anomaly detection on the live broadcast rendering link associated with the target live broadcast room through the rendering log information, thereby locating the abnormal rendering stage of the rendering abnormality in the target live broadcast room.

[0089] like Figure 3a and Figure 3b It can be seen that Figure 3a A scene diagram of an embodiment of each live broadcast rendering stage in the live broadcast rendering link of the target live broadcast room in the live broadcast optimization method provided in the embodiment of the present application. Figure 3bA scene diagram of a specific embodiment of rendering each live screen frame for the live broadcast rendering link of the target live broadcast room provided in the embodiment of the present application. Specifically, the live broadcast rendering link of the target live broadcast room includes at least a live broadcast shooting module, a live broadcast rendering module, a live broadcast encoding module and a live broadcast distribution module. Among them, the live broadcast shooting module is a shooting module that collects live screen images for the anchor user. The live broadcast rendering module is a rendering module that performs beautification and rendering operations on the live screen images collected by the live broadcast shooting module. The live broadcast encoding module is an encoding and streaming module that encodes, compresses and streams the rendered live screen images. The live broadcast distribution module is a content distribution module that transmits the encoded live screen images to the audience client of the target live broadcast room based on the content distribution network.

[0090] Depend on Figure 3b It can be seen that the rendering process of each live picture frame includes at least the following steps:

[0091] The live broadcast shooting module collects the live broadcast screen images associated with the target live broadcast room, and sends each frame of the live broadcast screen image to the live broadcast rendering module;

[0092] The live broadcast rendering module detects whether the beautification filter processor is turned on in the target live broadcast room. If the beautification filter processor is turned on, the live broadcast rendering module renders and beautifies the live broadcast screen image according to the beautification items and preset rendering methods set by the beautification filter processor, and when the original stream recording instruction is received, it also renders the live broadcast screen image according to the default rendering logic and preset rendering method, and also beautifies the rendered live broadcast screen image according to the beautification items set by the beautification filter processor. After rendering is completed, the rendered live broadcast screen image is output to the live broadcast encoding module.

[0093] If the beautification filter processor is not enabled, the live broadcast rendering module renders the live broadcast screen image according to the default rendering logic and the preset rendering method, and after the rendering is completed, it outputs the rendered live broadcast screen image to the live broadcast encoding module. Among them, the preset rendering method is the SurfaceTexture.updateTextlmage() method. The default rendering logic is to render the OES texture as a 2D texture.

[0094] The live shooting module also distributes the rendered live screen images to the preset display screen for preview display.

[0095] The live encoding module encodes the rendered live screen image through a preset streaming SDK (Software Development Kit) to obtain a live screen frame and transmits the live screen frame to the live distribution module.

[0096] The live broadcast distribution module distributes the encoded live broadcast screen images to the corresponding audience end of the target live broadcast room for display.

[0097] Specifically, the live broadcast rendering link of the target live broadcast room is also provided with a first live broadcast rendering stage and a second live broadcast rendering stage, and the first live broadcast rendering stage is earlier than the second live broadcast rendering stage. Taking the live broadcast rendering link of the live broadcast shooting module, the live broadcast rendering module, the live broadcast encoding module and the live broadcast distribution module as an example, in an optional embodiment, the live broadcast rendering stage from the live broadcast shooting module to the live broadcast rendering module is the first live broadcast rendering stage, and the live broadcast rendering stage from the live broadcast rendering module to the live broadcast encoding module is the second live broadcast rendering stage. In another optional embodiment, the live broadcast rendering stage from the live broadcast rendering module to the live broadcast encoding module is the first live broadcast rendering stage, and the live broadcast rendering stage from the live broadcast encoding module to the live broadcast distribution module is the second live broadcast rendering stage.

[0098] Specifically, the live broadcast optimization device calculates the image quality evaluation parameters corresponding to the abnormal picture frame in each live broadcast rendering stage according to the rendering log information, thereby determining the abnormal rendering stage in the live broadcast rendering link according to the image quality evaluation parameters.

[0099] Specifically, the live broadcast optimization device obtains the first rendering log information associated with the abnormal picture frame in the first live broadcast rendering stage and the first rendering log information associated with the abnormal picture frame in the second live broadcast rendering stage in the target live broadcast room. The first rendering log information includes the first rendered image of the abnormal picture frame in the first live broadcast rendering stage and the first image quality information. The second rendering log information includes the second rendered image and the second image quality information of the abnormal picture frame in the second live broadcast rendering stage.

[0100] Specifically, the live broadcast optimization device calculates the first image quality evaluation parameter of the first rendered image corresponding to the abnormal picture frame in the first live broadcast rendering stage according to the first image quality information in the first rendering log, and calculates the second image quality evaluation parameter of the second rendered image corresponding to the abnormal picture frame in the second live broadcast rendering stage according to the second image quality information in the second rendering log. The live broadcast optimization device locates the abnormal rendering stage in the live broadcast rendering link that causes the live broadcast picture to degrade according to the first image quality evaluation parameter, the second image quality evaluation parameter and the preset image quality evaluation threshold.

[0101] Specifically, the live broadcast optimization device compares the acquired first image quality evaluation parameter, the second image quality evaluation parameter, and a preset image quality evaluation threshold, and determines the abnormal rendering stage in the live broadcast rendering link according to the comparison result.

[0102] Optionally, if the first image quality evaluation parameter and the second image quality evaluation parameter are both greater than a preset image quality evaluation threshold, and the difference between the first image quality evaluation parameter and the second image quality evaluation parameter is less than a preset difference threshold, the live broadcast optimization device determines that the first live broadcast rendering stage and the second live broadcast rendering stage are normal rendering stages.

[0103] Optionally, if the second picture quality evaluation parameter is lower than the first picture quality evaluation parameter and lower than a preset picture quality evaluation threshold, that is, the picture quality of the second rendered image in the second live broadcast rendering stage is lower than the picture quality of the first rendered image in the first live broadcast rendering stage, that is, the live broadcast image has abnormal picture quality problems after being processed in the second live broadcast rendering stage. Therefore, the live broadcast optimization device sets the second live broadcast rendering stage as an abnormal rendering stage.

[0104] Optionally, in other embodiments, before comparing the rendering log information of each live rendering stage, the live broadcast optimization device also performs anomaly detection on the starting stage of the live broadcast rendering link corresponding to the target live broadcast room, that is, the image capture stage, so as to determine whether the live broadcast picture abnormality is caused by the image capture stage.

[0105] The live broadcast optimization device obtains the shooting log information in the image shooting stage of the target live broadcast room, wherein the shooting log information is a log record recording the target shot image and the image information of the target shot image during the live broadcast shooting process. The image information includes image noise data and image sharpening data.

[0106] Specifically, the live broadcast optimization device compares the image noise data with a preset noise threshold, and compares the image sharpening data with a preset sharpening parameter threshold, so as to determine whether the target captured image is an abnormal captured image with poor image quality.

[0107] Optionally, if the image noise data is less than a preset noise threshold and the image sharpening data is less than a preset sharpening parameter threshold, the live broadcast optimization device determines that the target captured image is a normal captured image with normal picture quality, and determines that the image capture stage is a normal image capture stage.

[0108] Optionally, if the image noise data is greater than a preset noise threshold, and / or the image sharpening data is greater than a preset sharpening parameter threshold, the target captured image is determined to be a captured image with abnormal image quality, and the image capturing stage is determined to be an abnormal rendering stage of the target live broadcast room.

[0109] 204. Optimize the abnormal rendering stage to obtain an image quality optimization result.

[0110] Specifically, after locating the abnormal rendering stage in the live rendering link corresponding to the target live broadcast room, the live broadcast optimization device also optimizes the abnormal rendering stage according to the image quality optimization strategy corresponding to the abnormal rendering stage, thereby obtaining an image quality optimization result. Among them, the image quality optimization strategy is a set of optimization rules for optimizing the image quality of the abnormal link module in the abnormal rendering stage. Optionally, the image quality optimization strategy includes a shooting optimization strategy, a beautification optimization strategy, and an encoding optimization strategy.

[0111] Specifically, after determining the abnormal rendering stage of the target live broadcast room, the live broadcast optimization device determines the abnormal link module corresponding to the abnormal rendering stage, obtains the image quality optimization strategy corresponding to the abnormal link module, optimizes the abnormal rendering stage based on the image quality optimization strategy, and obtains the image quality optimization result. Among them, the abnormal link module is the live broadcast rendering module in the live broadcast rendering link that causes the abnormal image quality to decrease. Among them, the abnormal link module in the live broadcast rendering link includes at least any one or more of an abnormal shooting module, an abnormal rendering module, and an abnormal encoding module.

[0112] Optionally, in a specific embodiment, the live broadcast optimization device determines that the abnormal rendering stage is an abnormal encoding stage, then determines that the abnormal link module is an abnormal encoding module, and obtains the encoding optimization strategy corresponding to the abnormal encoding module, wherein the encoding optimization strategy includes a target encoding strategy and a target encoding format.

[0113] Specifically, the live broadcast optimization device checks the coding compression format and initial coding strategy of the abnormal coding module. After detecting the abnormal coding compression format and / or abnormal coding strategy, the abnormal coding module is optimized through the target coding strategy and target coding format in the coding optimization strategy to obtain an optimized coding module.

[0114] Specifically, after obtaining the optimized coding module, the live broadcast optimization device also obtains the optimized coded image encoded by the optimized coding module, and performs image quality detection on the optimized coded image. If the image quality evaluation parameter of the optimized coded image is greater than the preset image quality evaluation threshold, the image quality optimization result of the optimization of the coding module is output.

[0115] like Figure 4 As shown, Figure 4 A timing diagram of an embodiment of the live broadcast optimization method provided in the present application. Figure 4 It can be seen that in the live broadcast optimization process of the embodiment, the live broadcast optimization device is a live broadcast server, and the target service terminal is a live broadcast client. The live broadcast optimization device can perform live broadcast picture detection on the target service terminal, drive the target service terminal to collect the live broadcast picture frames of the target live broadcast room and the live broadcast picture quality information corresponding to the live broadcast picture frames, and receive the uploaded live broadcast picture frames and live broadcast picture quality information sent by the target service focus, and perform picture quality detection on the live broadcast picture frames and live broadcast picture quality information. If an abnormal picture frame is detected, the rendering log information of each live broadcast rendering stage associated with the abnormal picture frame of the target live broadcast room is retrieved, and the abnormal rendering stage in the target live broadcast room is analyzed through the rendering log information, and a picture quality optimization strategy is sent to the target service terminal, driving the target service focus to perform live broadcast rendering optimization processing based on the picture quality optimization strategy, and obtaining a picture quality optimization result to improve the live broadcast picture quality of the target live broadcast room.

[0116] In this embodiment, the live broadcast optimization device obtains the live broadcast picture frame displayed in the target live broadcast room, as well as the live broadcast picture quality information in the live broadcast picture frame, and performs image quality detection on the live broadcast picture frame according to the live broadcast picture quality information, thereby determining whether the live broadcast picture frame is an abnormal picture frame, and when it is determined that there is an abnormal picture frame in the target live broadcast room, the rendering log information of each live broadcast rendering stage associated with the abnormal picture frame in the live broadcast rendering link is obtained, and the rendering log information of each live broadcast rendering stage is compared to determine the abnormal rendering stage of the target live broadcast room, and the abnormal rendering stage is optimized to obtain the picture quality optimization result. It is possible to accurately and quickly locate the abnormal picture quality factors in the live broadcast process, and to optimize the abnormal rendering stage corresponding to the abnormal picture quality factors in a targeted manner, thereby improving the picture quality stability during the live broadcast process.

[0117] like Figure 5 As shown, Figure 5 This is a flow chart of an embodiment of optimizing the image quality of an abnormal shooting module in the live broadcast optimization method provided in an embodiment of the present application. Specifically, based on the above embodiment, in this embodiment, the live broadcast optimization method further includes steps 301 to 303:

[0118] 301. If the abnormal link module is an abnormal shooting module, determining that the image quality optimization strategy is a shooting optimization strategy;

[0119] Specifically, if the live broadcast optimization device analyzes the shooting log information and determines that the abnormal rendering stage of the target live broadcast room is the abnormal shooting stage, that is, the live broadcast optimization device determines that the abnormal link module of the target live broadcast room is the abnormal camera module. That is, the picture frames captured by the camera module used to collect live picture frames in the target live broadcast room are abnormal, resulting in poor picture quality of the live picture frames displayed in the subsequent live broadcast room.

[0120] Specifically, after determining that the abnormal link module is an abnormal shooting module, the live broadcast optimization device further determines that the image quality optimization strategy of the target live broadcast room is a shooting optimization strategy. The shooting optimization strategy is a set of preset optimization rules including at least one camera optimization parameter such as focus optimization parameters, white balance optimization parameters and camera switching parameters.

[0121] Among them, the focus optimization parameters are optimization parameters for debugging and optimizing the focus mode of the abnormal shooting module.

[0122] The white balance optimization parameters are optimization parameters for debugging and optimizing the white balance parameters of the abnormal shooting module.

[0123] The camera switching parameters are optimized parameters for performing camera switching on an abnormal shooting module.

[0124] 302. Optimize the abnormal shooting module based on the focus optimization parameter, the white balance optimization parameter and / or the camera switching parameter in the shooting optimization strategy to obtain an optimized shooting module;

[0125] Specifically, after acquiring the shooting optimization strategy, the live broadcast optimization device performs image quality optimization processing on the abnormal camera module according to the shooting optimization strategy to obtain an optimized shooting module.

[0126] Specifically, the live broadcast optimization device obtains the initial focus parameters and initial white balance parameters in the abnormal shooting module, and compares the initial focus parameters with the focus optimization parameters. If the initial focus parameters are different from the focus optimization parameters, the initial focus parameters are updated according to the focus optimization parameters to obtain updated focus parameters.

[0127] Specifically, the live broadcast optimization device can also compare the initial white balance parameters with the white balance optimization parameters. If the initial white balance parameters and the white balance optimization parameters are different, the initial focus parameters are updated according to the white balance optimization parameters to obtain updated white balance parameters.

[0128] Optionally, in other embodiments, the live broadcast optimization device also compares the current camera identifier of the abnormal shooting module. If the current camera identifier is different from the target camera identifier corresponding to the camera switching parameter, the current shooting unit in the abnormal shooting module is switched to the target shooting unit according to the camera switching parameter.

[0129] Specifically, after the live broadcast optimization device optimizes the image quality of the abnormal shooting module according to the shooting optimization strategy, the optimized abnormal camera module is set as the optimized shooting module.

[0130] 303. Obtain an optimized shooting image collected by the optimized shooting module, and output an image quality optimization result of the optimized shooting module according to the optimized shooting image.

[0131] Specifically, after the live broadcast optimization device optimizes the abnormal shooting module of the target live broadcast room, it also obtains the optimized shooting image captured by the optimized shooting module, and performs image quality detection on the optimized shooting image. If the image quality evaluation parameter of the optimized shooting image is greater than the preset image quality evaluation threshold, the image quality optimization result of the shooting module optimization is output.

[0132] Among them, the image quality detection method is the same as that in the above embodiment, and will not be described in detail in this embodiment.

[0133] In this embodiment, the live broadcast optimization device determines that the abnormal link module is an abnormal shooting module, and then determines that the image quality optimization strategy is a shooting optimization strategy; optimizes the abnormal shooting module based on the focus optimization parameters, white balance optimization parameters and / or camera switching parameters in the shooting optimization strategy to obtain an optimized shooting module; obtains the optimized shooting image collected by the optimized shooting module, and outputs the image quality optimization result of the optimized shooting module according to the optimized shooting image. Targeted image quality optimization is achieved for the abnormal shooting module that performs live broadcast collection to improve the image quality of the live broadcast display frame.

[0134] like Figure 6 As shown, Figure 6 This is a flow chart of an embodiment of optimizing the image quality of an abnormal rendering module in the live broadcast optimization method provided in an embodiment of the present application. Specifically, based on the above embodiment, in this embodiment, the live broadcast optimization method further includes steps 401 to 404:

[0135] 401. If the abnormal link module is an abnormal rendering module, determine that the image quality optimization strategy is a rendering optimization strategy;

[0136] Specifically, when the live broadcast optimization device compares the rendering log information and determines that the rendered image has image quality problems such as picture noise and severe sharpening, it determines that the abnormal rendering stage is an abnormal rendering stage, and the abnormal link module is an abnormal rendering module, and the live broadcast optimization device obtains the rendering optimization strategy corresponding to the abnormal rendering module. Among them, the rendering optimization strategy is a set of rules that includes target beautification projects and corresponding target beautification parameters, and can adjust and optimize the abnormal rendering module. Among them, the target beautification project is an image / portrait beautification project that improves the image quality effect. The target beautification parameters are the optimized image / portrait beautification parameters corresponding to the target beautification project. Among them, in a specific embodiment, the abnormal rendering module is a rendering SDK (Software Development Kit) of the target live broadcast application.

[0137] 402. Identify an abnormal beautification item associated with the abnormal picture frame in the abnormal rendering module, and an abnormal beautification parameter of the abnormal beautification item;

[0138] Specifically, the live broadcast optimization device checks the initial beautification items set by the user in the abnormal rendering module according to the rendering optimization strategy, locates the abnormal beautification items in the initial beautification items that cause the abnormal picture frame quality to deteriorate, and the abnormal beautification parameters of the abnormal beautification items.

[0139] Specifically, the live broadcast optimization device obtains the initial beautification items in the abnormal rendering module, and determines whether the initial beautification parameters in each initial beautification item exceed the corresponding target beautification threshold according to the rendering optimization strategy, and sets the initial beautification items corresponding to the initial beautification parameters exceeding the target beautification threshold as abnormal beautification items, and sets the initial beautification parameters exceeding the target beautification threshold as abnormal beautification parameters.

[0140] 403. Adjust the abnormal beautification parameter based on the target beautification parameter of the target beautification item in the rendering optimization strategy to obtain an optimized rendering module;

[0141] Specifically, after obtaining the abnormal beautification project and the corresponding abnormal beautification parameters, the live broadcast optimization device also determines whether the abnormal beautification project is an abnormal beautification project that can be adjusted and optimized. If the abnormal beautification parameters of the abnormal beautification project can be adjusted, the abnormal beautification project and the abnormal beautification parameters in the abnormal rendering module are corrected according to the target beautification project in the rendering optimization strategy and the target beautification parameters corresponding to the target beautification project to obtain an optimized rendering module. In a specific embodiment, the abnormal beautification project includes beautification rendering projects such as skin grinding amplitude, whitening amplitude, color scale amplitude and contrast amplitude.

[0142] Specifically, the live broadcast optimization device sends a rendering adjustment instruction to the target business terminal corresponding to the target live broadcast room according to the target beautification project and the target beautification parameter, so as to adjust and optimize the adjustable abnormal beautification project and abnormal beautification parameter in the abnormal rendering module to obtain an optimized rendering module.

[0143] 404. Obtain an optimized rendering image rendered by the optimized rendering module, and determine a picture quality optimization result of the optimized rendering module according to the optimized rendering image.

[0144] Specifically, after optimizing the image quality of the abnormal rendering module, the live broadcast optimization device also obtains the optimized rendering image rendered by the optimized rendering module, and performs image quality detection on the optimized rendering image. If the image quality evaluation parameter of the optimized rendering image is greater than the preset image quality evaluation threshold, the image quality optimization result of the rendering module optimization is output.

[0145] Optionally, if the image quality evaluation parameter of the optimized rendered image is lower than a preset image quality evaluation threshold, the live broadcast optimization device determines that the abnormal beautification item and abnormal beautification parameter of the abnormal rendering module are non-adjustable abnormal beautification items and abnormal beautification parameters. The live broadcast optimization device reads the current version number of the optimized rendering module, and obtains the rendering update package of the optimized rendering module from the application update server based on the current version number.

[0146] Specifically, the live broadcast optimization device updates the optimization rendering module based on the rendering update package, obtains the updated rendering module, obtains the updated rendering image rendered by the updated rendering module, calculates the updated image quality evaluation parameter of the updated rendering image, and if the updated image quality evaluation parameter is higher than the preset image quality evaluation threshold, outputs the image quality optimization result of the live broadcast rendering module optimization.

[0147] In this embodiment, the live broadcast optimization device determines that the image quality optimization strategy is a rendering optimization strategy by determining that the abnormal link module is an abnormal rendering module; identifies the abnormal beautification project associated with the abnormal picture frame in the abnormal rendering module, and the abnormal beautification parameters of the abnormal beautification project; adjusts the abnormal beautification parameters based on the target beautification parameters of the target beautification project in the rendering optimization strategy to obtain an optimized rendering module; obtains an optimized rendering image rendered by the optimized rendering module, and determines the image quality optimization result of the optimized rendering module according to the optimized rendering image. The image quality optimization of the abnormal beautification project in the abnormal rendering module in the live broadcast rendering link is realized to improve the image quality of the live broadcast display picture frame.

[0148] In order to better implement the live broadcast optimization method in the embodiment of the present application, based on the live broadcast optimization method, the embodiment of the present application also provides a live broadcast optimization device, such as Figure 7 As shown, Figure 7 The structure diagram of the live broadcast optimization device provided in the embodiment of the present application is as follows. Specifically, the live broadcast optimization device 500 includes:

[0149] The information acquisition module 501 is configured to acquire a live picture frame displayed in a target live broadcast room to be detected, and live picture quality information of the live picture frame;

[0150] The abnormality identification module 502 is configured to perform abnormal image detection on the live picture frames in the target live room according to the live picture quality information, and determine the abnormal picture frames in the target live room;

[0151] The link positioning module 503 is configured to obtain the rendering log information of each live rendering stage associated with the abnormal picture frame, compare the rendering log information of each live rendering stage, and locate the abnormal rendering stage of the target live room;

[0152] The live broadcast optimization module 504 is configured to optimize the abnormal rendering stage to obtain a picture quality optimization result.

[0153] In a possible implementation of this embodiment, the live broadcast optimization device obtains rendering log information of each live broadcast rendering stage associated with the abnormal picture frame, compares the rendering log information of each live broadcast rendering stage, and locates the abnormal rendering stage of the target live broadcast room, including:

[0154] Acquire first rendering log information associated with the abnormal picture frame in a first live broadcast rendering phase in the target live broadcast room, and second rendering log information associated with the abnormal picture frame in a second live broadcast rendering phase;

[0155] Calculating a first image quality evaluation parameter of a first rendered image in the first rendering log information and a second image quality evaluation parameter of a second rendered image in the second rendering log;

[0156] If the second image quality evaluation parameter is lower than the first image quality evaluation parameter and lower than a preset image quality evaluation threshold, the second live broadcast rendering stage is set as an abnormal rendering stage.

[0157] In a possible implementation of this embodiment, the live broadcast optimization device obtains rendering log information of each live broadcast rendering stage associated with the abnormal picture frame, compares the rendering log information of each live broadcast rendering stage, and locates the abnormal rendering stage of the target live broadcast room, including:

[0158] Acquire shooting log information of the image shooting stage of the target live broadcast room, wherein the shooting log information includes a target shot image, and image noise data and image sharpening data corresponding to the target shot image;

[0159] If the image noise data is greater than a preset noise threshold and / or the image sharpening data is greater than a preset sharpening parameter threshold, the image capture stage is set as an abnormal rendering stage of the target live broadcast room.

[0160] In a possible implementation of this embodiment, the live broadcast optimization device optimizes the abnormal rendering stage to obtain a picture quality optimization result, including:

[0161] Acquire an abnormal link module corresponding to the abnormal rendering stage, wherein the abnormal link module includes any one or more of an abnormal shooting module, an abnormal rendering module and an abnormal encoding module;

[0162] The image quality optimization strategy corresponding to the abnormal link module is obtained, and the abnormal rendering stage is optimized based on the image quality optimization strategy to obtain an image quality optimization result.

[0163] In a possible implementation of this embodiment, the live broadcast optimization device obtains the image quality optimization strategy corresponding to the abnormal link module, optimizes the abnormal rendering stage based on the image quality optimization strategy, and obtains the image quality optimization result, including:

[0164] If the abnormal link module is an abnormal shooting module, determining that the image quality optimization strategy is a shooting optimization strategy;

[0165] Optimizing the abnormal shooting module based on the focus optimization parameters, white balance optimization parameters and / or camera switching parameters in the shooting optimization strategy to obtain an optimized shooting module;

[0166] The optimized shooting image captured by the optimized shooting module is obtained, and the image quality optimization result of the optimized shooting module is output according to the optimized shooting image.

[0167] In a possible implementation of this embodiment, the live broadcast optimization device obtains the image quality optimization strategy corresponding to the abnormal link module, optimizes the abnormal rendering stage based on the image quality optimization strategy, and obtains the image quality optimization result, including:

[0168] If the abnormal link module is an abnormal rendering module, determining that the image quality optimization strategy is a rendering optimization strategy;

[0169] Identifying an abnormal beautification item associated with the abnormal picture frame in the abnormal rendering module, and an abnormal beautification parameter of the abnormal beautification item;

[0170] The abnormal beautification parameters are adjusted based on the target beautification parameters of the target beautification project in the rendering optimization strategy to obtain an optimized rendering module;

[0171] An optimized rendering image rendered by the optimized rendering module is obtained, and a picture quality optimization result of the optimized rendering module is determined according to the optimized rendering image.

[0172] In a possible implementation of this embodiment, the live broadcast optimization device determines the image quality optimization result of the optimization rendering module according to the optimized rendering image, including:

[0173] If the image quality evaluation parameter of the optimized rendering image is lower than a preset image quality evaluation threshold, obtaining a rendering update package of the optimized rendering module;

[0174] The optimized rendering module is updated based on the rendering update package to obtain an updated rendering module;

[0175] An updated rendering image of an updated rendering module is obtained, and a picture quality optimization result corresponding to the updated rendering image is output.

[0176] In a possible implementation of this embodiment, the live broadcast optimization device obtains the image quality optimization strategy corresponding to the abnormal link module, optimizes the abnormal rendering stage based on the image quality optimization strategy, and obtains the image quality optimization result, including:

[0177] If the abnormal link module is an abnormal encoding module, determining that the image quality optimization strategy is an encoding optimization strategy;

[0178] Optimizing the abnormal coding module based on the target coding strategy and the target coding format in the coding optimization strategy to obtain an optimized coding module;

[0179] The optimized coded image encoded by the optimized coding module is obtained, and a picture quality optimization result corresponding to the optimized coded image is output.

[0180] In this embodiment, the live broadcast optimization device obtains the live picture frame displayed in the target live broadcast room, and the live picture quality information in the live picture frame, and performs image quality detection on the live picture frame according to the live picture quality information, thereby determining whether the live picture frame is an abnormal picture frame, and when it is determined that there is an abnormal picture frame in the target live broadcast room, the rendering log information of each live rendering stage associated with the abnormal picture frame in the live broadcast rendering link is obtained, and the rendering log information of each live rendering stage is compared to determine the abnormal rendering stage of the target live broadcast room, and the abnormal rendering stage is optimized to obtain the picture quality optimization result. It can accurately and quickly locate the abnormal picture quality factors in the live broadcast process, and optimize the abnormal rendering stage corresponding to the abnormal picture quality factors in a targeted manner, thereby improving the picture quality stability during the live broadcast process.

[0181] The embodiment of the present invention also provides a live broadcast optimization device, such as Figure 8 As shown, Figure 8 A schematic diagram of the structure of an embodiment of the live broadcast optimization device provided in the embodiments of the present application.

[0182] The live broadcast optimization device integrates any one of the live broadcast optimization devices provided in the embodiments of the present invention, and the live broadcast optimization device includes:

[0183] one or more processors;

[0184] Memory; and

[0185] One or more applications, wherein the one or more applications are stored in the memory and are configured to be executed by the processor to perform the steps of the live broadcast optimization method described in any of the above-mentioned live broadcast optimization method embodiments.

[0186] Specifically, the live broadcast optimization device may include one or more processing core processors 601, one or more computer-readable storage media memories 602, a power supply 603, an input unit 604 and other components. Those skilled in the art will understand that Figure 6 The structure of the live broadcast optimization device shown in the figure does not constitute a limitation on the live broadcast optimization device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0187] The processor 601 is the control center of the live broadcast optimization device. It uses various interfaces and lines to connect various parts of the entire live broadcast optimization device. By running or executing software programs and / or modules stored in the memory 602, and calling data stored in the memory 602, it performs various functions of the live broadcast optimization device and processes data, thereby monitoring the live broadcast optimization device as a whole. Optionally, the processor 601 may include one or more processing cores; preferably, the processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 601.

[0188] The memory 602 can be used to store software programs and modules. The processor 601 executes various functional applications and data processing by running the software programs and modules stored in the memory 602. The memory 602 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the live broadcast optimization device, etc. In addition, the memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 602 may also include a memory controller to provide the processor 601 with access to the memory 602.

[0189] The live broadcast optimization device also includes a power supply 603 for supplying power to each component. Preferably, the power supply 603 can be logically connected to the processor 601 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 603 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0190] The live broadcast optimization device may also include an input unit 604, which can be used to receive input digital or character information, and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0191] Although not shown, the live broadcast optimization device may also include a display unit, etc., which will not be described in detail here. Specifically in this embodiment, the processor 601 in the live broadcast optimization device will load the executable files corresponding to the processes of one or more application programs into the memory 602 according to the following instructions, and the processor 601 will run the application programs stored in the memory 602, thereby realizing various functions, as follows:

[0192] Acquire a live picture frame displayed in the target live broadcast room to be detected, and live picture quality information of the live picture frame;

[0193] Perform abnormal image detection on the live picture frames in the target live broadcast room according to the live picture quality information, and determine the abnormal picture frames in the target live broadcast room;

[0194] Acquire rendering log information of each live rendering stage associated with the abnormal picture frame, compare the rendering log information of each live rendering stage, and locate the abnormal rendering stage of the target live room;

[0195] The abnormal rendering stage is optimized to obtain an image quality optimization result.

[0196] To this end, an embodiment of the present invention provides a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any one of the live broadcast optimization methods provided in the embodiments of the present invention. For example, the computer program can be loaded by a processor to execute the following steps:

[0197] Acquire a live picture frame displayed in the target live broadcast room to be detected, and live picture quality information of the live picture frame;

[0198] Perform abnormal image detection on the live picture frames in the target live broadcast room according to the live picture quality information, and determine the abnormal picture frames in the target live broadcast room;

[0199] Acquire rendering log information of each live rendering stage associated with the abnormal picture frame, compare the rendering log information of each live rendering stage, and locate the abnormal rendering stage of the target live room;

[0200] The abnormal rendering stage is optimized to obtain an image quality optimization result.

[0201] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the detailed description of other embodiments above, and will not be repeated here.

[0202] In specific implementation, the above units or structures can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units or structures can refer to the previous method embodiments, which will not be repeated here.

[0203] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0204] The above is a detailed introduction to a live broadcast optimization method provided in an embodiment of the present application. Specific embodiments are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A live broadcast optimization method, It is characterized in that The live broadcast optimization method comprises: Acquire a live picture frame displayed in the target live broadcast room to be detected, and live picture quality information of the live picture frame; Perform abnormal image detection on the live picture frames in the target live broadcast room according to the live picture quality information, and determine the abnormal picture frames in the target live broadcast room; Obtaining rendering log information of each live rendering stage associated with the abnormal picture frame, comparing the rendering log information of each live rendering stage, and locating the abnormal rendering stage of the target live room, wherein the live rendering stage is an image processing stage between adjacent link modules in a live rendering link of the target live room, and the live rendering link includes at least a live shooting module, a live rendering module, a live encoding module, and a live distribution module; The abnormal rendering stage is optimized to obtain an image quality optimization result.

2. The live broadcast optimization method according to claim 1, It is characterized in that The live broadcast rendering stage includes a first live broadcast rendering stage and a second live broadcast rendering stage, wherein the first live broadcast rendering stage is a previous live broadcast rendering stage of the second live broadcast rendering stage; The obtaining of rendering log information associated with the abnormal picture frame in each live rendering stage, comparing the rendering log information of each live rendering stage, and locating the abnormal rendering stage of the target live room includes: Obtaining first rendering log information associated with the abnormal picture frame in the first live rendering stage in the target live broadcast room, and second rendering log information associated with the abnormal picture frame in the second live rendering stage; the first rendering log information includes a first rendered image of the abnormal picture frame in the first live rendering stage, and first image quality information, and the second rendering log information includes a second rendered image and second image quality information of the abnormal picture frame in the second live rendering stage; Calculating a first image quality evaluation parameter of a first rendered image in the first rendering log information and a second image quality evaluation parameter of a second rendered image in the second rendering log; If the second image quality evaluation parameter is lower than the first image quality evaluation parameter and lower than a preset image quality evaluation threshold, the second live broadcast rendering stage is set as an abnormal rendering stage.

3. The live broadcast optimization method according to claim 1, It is characterized in that The obtaining of rendering log information associated with the abnormal picture frame in each live rendering stage, comparing the rendering log information of each live rendering stage, and locating the abnormal rendering stage of the target live room includes: Acquire shooting log information of the image shooting stage of the target live broadcast room, wherein the shooting log information includes a target shot image, and image noise data and image sharpening data corresponding to the target shot image; If the image noise data is greater than a preset noise threshold and / or the image sharpening data is greater than a preset sharpening parameter threshold, the image capture stage is set as an abnormal rendering stage of the target live broadcast room.

4. The live broadcast optimization method according to claim 1, It is characterized in that The optimizing process of the abnormal rendering stage to obtain an image quality optimization result includes: Acquire an abnormal link module corresponding to the abnormal rendering stage, wherein the abnormal link module includes any one or more of an abnormal shooting module, an abnormal rendering module and an abnormal encoding module; The image quality optimization strategy corresponding to the abnormal link module is obtained, and the abnormal rendering stage is optimized based on the image quality optimization strategy to obtain an image quality optimization result.

5. The live broadcast optimization method according to claim 4, It is characterized in that The obtaining of the image quality optimization strategy corresponding to the abnormal link module, and optimizing the abnormal rendering stage based on the image quality optimization strategy to obtain an image quality optimization result, includes: If the abnormal link module is an abnormal shooting module, determining that the image quality optimization strategy is a shooting optimization strategy; Optimizing the abnormal shooting module based on the focus optimization parameters, white balance optimization parameters and / or camera switching parameters in the shooting optimization strategy to obtain an optimized shooting module; The optimized shooting image captured by the optimized shooting module is obtained, and the image quality optimization result of the optimized shooting module is output according to the optimized shooting image.

6. The live broadcast optimization method according to claim 4, It is characterized in that The obtaining of the image quality optimization strategy corresponding to the abnormal link module, and optimizing the abnormal rendering stage based on the image quality optimization strategy to obtain an image quality optimization result, includes: If the abnormal link module is an abnormal rendering module, determining that the image quality optimization strategy is a rendering optimization strategy; Identifying an abnormal beautification item associated with the abnormal picture frame in the abnormal rendering module, and an abnormal beautification parameter of the abnormal beautification item; The abnormal beautification parameters are adjusted based on the target beautification parameters of the target beautification project in the rendering optimization strategy to obtain an optimized rendering module; An optimized rendering image rendered by the optimized rendering module is obtained, and a picture quality optimization result of the optimized rendering module is determined according to the optimized rendering image.

7. The live broadcast optimization method according to claim 6, It is characterized in that The step of determining the image quality optimization result of the optimized rendering module according to the optimized rendered image includes: If the image quality evaluation parameter of the optimized rendering image is lower than a preset image quality evaluation threshold, obtaining a rendering update package of the optimized rendering module; The optimized rendering module is updated based on the rendering update package to obtain an updated rendering module; An updated rendering image of an updated rendering module is obtained, and a picture quality optimization result corresponding to the updated rendering image is output.

8. The live broadcast optimization method according to claim 4, It is characterized in that The obtaining of the image quality optimization strategy corresponding to the abnormal link module, and optimizing the abnormal rendering stage based on the image quality optimization strategy to obtain an image quality optimization result, includes: If the abnormal link module is an abnormal encoding module, determining that the image quality optimization strategy is an encoding optimization strategy; Optimizing the abnormal coding module based on the target coding strategy and the target coding format in the coding optimization strategy to obtain an optimized coding module; The optimized coded image encoded by the optimized coding module is obtained, and a picture quality optimization result corresponding to the optimized coded image is output.

9. A live broadcast optimization device, It is characterized in that The live broadcast optimization device comprises: An information acquisition module is configured to acquire a live picture frame displayed in a target live broadcast room to be detected, and live picture quality information of the live picture frame; an abnormality identification module, configured to perform abnormal image detection on the live picture frames in the target live room according to the live picture quality information, and determine the abnormal picture frames in the target live room; A link positioning module is configured to obtain rendering log information of each live rendering stage associated with the abnormal picture frame, compare the rendering log information of each live rendering stage, and locate the abnormal rendering stage of the target live room, wherein the live rendering stage is an image processing stage between adjacent link modules in the live rendering link of the target live room, and the live rendering link includes at least a live shooting module, a live rendering module, a live encoding module, and a live distribution module; The live broadcast optimization module is configured to optimize the abnormal rendering stage to obtain a picture quality optimization result.

10. A live broadcast optimization device, It is characterized in that The live broadcast optimization device comprises: one or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the processor to implement the steps of the live broadcast optimization method described in any one of claims 1 to 8.

11. A computer-readable storage medium, It is characterized in that A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps of the live broadcast optimization method described in any one of claims 1 to 8.