Live broadcast data transmission method and device

Abstract

The invention discloses a live broadcast data transmission method and device. The method comprises the following steps: acquiring a current bandwidth, a preset transmission frame rate, a current real-time code rate, residual live broadcast time, data frame indication information, target maturity information and reference maturity information; the current real-time code rate is larger than the current bandwidth, and the data frame indication information indicates that the last frame of image is located in a frame extraction area; determining current maturity information based on the food material cooking result; determining a maturity threshold based on the current maturity information, the target maturity information, the residual live broadcast time, the current real-time code rate, the current bandwidth and a preset transmission frame rate; and when the first maturity difference value is greater than the maturity threshold value, the current image frame is extracted, and the live broadcast image frames in the live broadcast buffer data queue pair are transmitted in sequence based on the real-time transmission frame rate, so that the frame extraction preciseness is improved, the image quality and resolution requirements are met, the image frame transmission timeliness is improved, and the user experience is improved. And the live broadcast fluency is ensured.

Description

Technical Field

[0001] The present invention relates to the technical field of data transmission, and in particular, to a live broadcast data transmission method and apparatus. Background Art

[0002] During a live broadcast, the necessity of data transmission is self-evident. In actual situations, there are cases where the network is poor. In a weak network environment, the network bandwidth is limited. To ensure the smoothness of the live broadcast, it is necessary to reduce the amount of video data transmitted, reduce the bit rate, and thus reduce the resolution and frame rate. However, this will result in a reduction in the image quality of the live broadcast. Therefore, in the application scenario of live cooking in a steam oven, the rate of change of the outline and color of the ingredients is relatively low. How to maintain the clarity of the image quality while ensuring the resolution requires a new live broadcast data transmission method to meet the requirements of resolution and image quality. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a live broadcast data transmission method and apparatus to solve technical problems such as low resolution, low image quality, and low live broadcast smoothness during a live broadcast in a weak network environment.

[0004] One aspect of the present invention provides a live broadcast data transmission method, which includes the following steps: Obtain the current bandwidth corresponding to the current live broadcast, the preset transmission frame rate, the current real-time bit rate, the remaining live broadcast time, the data frame indication information corresponding to the live broadcast buffer data queue, the target maturity information corresponding to the cooking ingredients, and the reference maturity information corresponding to the reference image frame; the data frame indication information is used to indicate whether the last frame image in the live broadcast buffer data queue is located in the frame extraction area; the current image frame is the image frame collected after the last frame image in the live broadcast buffer data queue; When the current real-time bit rate is greater than the current bandwidth and the data frame indication information indicates that the last frame image is located in the frame extraction area, obtain the ingredient cooking result of the cooking ingredients in the current image frame; Determine the current maturity information based on the ingredient cooking result; Determine a maturity threshold based on the current maturity information, the target maturity information, the remaining live broadcast time, the current real-time bit rate, the current bandwidth, and the preset transmission frame rate; When the first maturity difference between the current maturity information and the reference maturity information is greater than the maturity threshold, extract the current image frame, and determine the real-time transmission frame rate of the current live broadcast based on the current bandwidth, the current real-time bit rate, and the preset transmission frame rate; Transmit the live broadcast image frames in the live broadcast buffer data queue in order based on the real-time transmission frame rate.

[0005] On the other hand, the present invention provides a transmission device, which comprises: A first acquisition module, configured to acquire the current bandwidth corresponding to the current live broadcast, a preset transmission frame rate, the current real-time bit rate, the remaining live broadcast time, data frame indication information corresponding to the live broadcast buffer data queue, target maturity information corresponding to cooking ingredients, and reference maturity information corresponding to a reference image frame; the data frame indication information is used to indicate whether the last frame of image in the live broadcast buffer data queue is located in the frame extraction area; the current image frame is an image frame acquired after the last frame of image in the live broadcast buffer data queue; A second acquisition module, configured to acquire the cooking result of the cooking ingredients in the current image frame when the current real-time bit rate is greater than the current bandwidth and the data frame indication information indicates that the last frame of image is located in the frame extraction area; A first determination module, configured to determine the current maturity information based on the cooking result of the ingredients; A second determination module, configured to determine a maturity threshold based on the current maturity information, the target maturity information, the remaining live broadcast time, the current real-time bit rate, the current bandwidth, and the preset transmission frame rate; A third determination module, configured to extract the current image frame when a first maturity difference between the current maturity information and the reference maturity information is greater than the maturity threshold, and determine the real-time transmission frame rate of the current live broadcast based on the current bandwidth, the current real-time bit rate, and the preset transmission frame rate; A transmission module, configured to sequentially transmit the live broadcast image frames in the live broadcast buffer data queue based on the real-time transmission frame rate.

[0006] On yet another aspect, the present invention provides an electronic device, which comprises: A processor; A memory for storing instructions executable by the processor; Wherein, the processor is configured to execute the instructions to implement the live broadcast data transmission method described in any one of the above.

[0007] On yet another aspect, the present invention provides a computer-readable storage medium, which, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to execute the live broadcast data transmission method described in any one of the above.

[0008] A live broadcast data transmission method and apparatus provided by the present invention obtain the current bandwidth, preset transmission frame rate, current real-time bit rate, remaining live broadcast time, data frame indication information corresponding to the live broadcast buffer data queue, target maturity information corresponding to cooking ingredients, and reference maturity information corresponding to the reference image frame for the current live broadcast. When the current real-time bit rate is greater than the current bandwidth and the data frame indication information indicates that the last frame image of the live broadcast buffer data queue is located in the frame extraction area, the cooking result of the cooking ingredients is obtained, and the current maturity information is determined based on the cooking result of the ingredients, which improves the rationality of determining the current maturity information. Further, by using the determined current maturity information, target maturity information, current real-time bit rate, current bandwidth, and preset transmission frame rate, a maturity threshold is calculated. Then, the maturity threshold is compared with the first maturity difference calculated from the current maturity information and the reference maturity information. When the first maturity difference is greater than the maturity threshold, the current image frame is extracted, which improves the rigor of frame extraction for live broadcast image frames and meets the requirements for the image quality and resolution of the current live broadcast. Further, based on the current bandwidth, current real-time bit rate, and preset transmission frame rate, the real-time transmission frame rate is calculated and determined, and the live broadcast image frames in the live broadcast buffer data queue are transmitted in sequence, which improves the timeliness of live broadcast image frame transmission and ensures the smoothness of the current live broadcast. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] To more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the accompanying drawings required for the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0010] Figure 1 is a flowchart of a live broadcast data transmission method provided according to an exemplary embodiment; Figure 2 is a flowchart of determining a maturity threshold based on the current maturity information, target maturity information, remaining live broadcast time, current real-time bit rate, current bandwidth, and preset transmission frame rate according to an exemplary embodiment; Figure 3 is a flowchart of determining a first maturity difference based on the current maturity information and reference maturity information according to an exemplary embodiment; Figure 4 is a structural diagram of a transmission apparatus provided according to an exemplary embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0011] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative efforts belong to the scope of protection of the present application.

[0012] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present application described here can be implemented in an order other than those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or server including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0013] Figure 1 It is a schematic flowchart of a live data transmission method provided according to an exemplary embodiment. The present specification provides method operation steps such as in the embodiment or flowchart, but based on routine or non-creative labor, it may include more or fewer operation steps. The order of steps listed in the embodiment is only one way among the execution orders of many steps and does not represent the only execution order. When the actual system or server product executes, it can be executed in the order shown in the embodiment or the drawing or in parallel (for example, in an environment of parallel processors or multi-threaded processing). Specifically, as Figure 1 shown, taking the live controller as the execution subject, an embodiment of a live data transmission method of the present application is introduced. The above method may include: S101: Obtain the current bandwidth corresponding to the current live broadcast, the preset transmission frame rate, the current real-time bit rate, the remaining live broadcast time, the data frame indication information corresponding to the live broadcast buffer data queue, the target maturity information corresponding to the cooking ingredients, and the reference maturity information corresponding to the reference image frame.

[0014] In a specific embodiment, the data frame indication information is used to indicate whether the last frame image in the live buffer data queue is located in the frame extraction area; the current image frame is an image frame collected after the last frame image in the live buffer data queue; the current bandwidth may be the transmission rate of the current live broadcast; the preset transmission frame rate may be the standard number of frames transmitted per second in the current live broadcast; the current real-time bit rate may be the current live broadcast encoding rate; the target maturity information may be the maturity characteristic information of the cooking ingredient when the cooking ingredient is mature. Optionally, the maturity characteristic information of the cooking ingredient may include the color information and contour information of the cooking ingredient; the reference maturity information may be the maturity characteristic information corresponding to the cooking ingredient in the reference image frame; the remaining live broadcast time may be the remaining live broadcast time of the current live broadcast. Specifically, before starting the live broadcast, the preset live broadcast duration of the current live broadcast is set in advance. Correspondingly, when the current image frame is obtained, the live broadcast duration corresponding to the current live broadcast that has been broadcast is also obtained. Correspondingly, the difference between the preset live broadcast duration and the live broadcast duration that has been broadcast is calculated to determine the remaining live broadcast time of the current live broadcast.

[0015] Specifically, during the live broadcast, the current bandwidth, preset transmission frame rate, current real-time bit rate, remaining live broadcast time, data frame indication information corresponding to the live buffer data queue, target maturity information corresponding to the cooking ingredient, and reference maturity information corresponding to the reference image frame corresponding to the current live broadcast are obtained. Optionally, the live buffer data queue may be a storage queue for storing live image frames. Specifically, the live buffer data queue may be divided into an adaptive area and a frame extraction area. Optionally, the live image frames are preferentially stored in the adaptive area. When the number of live image frames to be stored is greater than the number of stored image frames corresponding to the adaptive area, the subsequent live image frames are stored in the frame extraction area. Optionally, in this application, it is determined whether to extract frames from the live data frames according to the area position of the last frame image in the live buffer data queue.

[0016] S103: When the current real-time bit rate is greater than the current bandwidth and the data frame indication information indicates that the last frame image is located in the frame extraction area, obtain the cooking result of the cooking ingredient in the current image frame.

[0017] In a specific embodiment, the cooking result of the ingredient is used to indicate whether the color and contour of the cooking ingredient have changed; specifically, when the current real-time bit rate is greater than the current bandwidth and the data indication information indicates that the last frame image is located in the frame extraction area, it means that the current live broadcast is in a weak network environment and the live image frames of the current live broadcast have been stored in the frame extraction area. In this case, the current image frame is collected, and the cooking result of the cooking ingredient in the current image frame is obtained using an image processing algorithm. Optionally, the image processing algorithm may include algorithms with image processing functions such as a binarization algorithm and an RGB color extraction algorithm.

[0018] S105: Determine the current maturity information based on the cooking result of the food ingredient.

[0019] In a specific embodiment, the current maturity information may be the maturity feature information corresponding to the cooked food ingredient in the current image frame; specifically, based on the cooking result of the cooked food ingredient in the previously obtained current image frame, determine the current maturity information of the cooked food ingredient in the current image frame.

[0020] S107: Determine the maturity threshold based on the current maturity information, the target maturity information, the remaining live broadcast time, the current real-time bit rate, the current bandwidth, and the preset transmission frame rate.

[0021] In a specific embodiment, the maturity threshold may be the judgment threshold for determining whether to drop frames for the current image frame. Optionally, the maturity threshold may include a grayscale maturity threshold, an area ratio maturity threshold, and a weighted maturity threshold; calculate the maturity threshold based on the previously obtained information. Specifically, use the current maturity information, the target maturity information, the remaining live broadcast time, the current real-time bit rate, the current bandwidth, and the preset transmission frame rate to calculate and determine the maturity threshold.

[0022] S109: In the case where the first maturity difference between the current maturity information and the reference maturity information is greater than the maturity threshold, drop the current image frame, and determine the real-time transmission frame rate of the current live broadcast based on the current bandwidth, the current real-time bit rate, and the preset transmission frame rate.

[0023] In a specific embodiment, the real-time transmission frame rate may be the transmission frame rate of the live broadcast image frames in the current live broadcast when the current image frame is dropped; the first maturity difference may be the maturity difference of the cooked food ingredient between the current image frame and the reference image frame. Optionally, the first maturity difference may include a first grayscale difference, a first area ratio difference, and a first weighted difference; call the reference maturity information corresponding to the reference image frame, calculate the difference between the obtained current maturity information and the reference maturity information to determine the first maturity difference. Further, compare the calculated first maturity difference with the previously calculated maturity threshold. In the case where the first maturity difference is greater than the maturity threshold, drop the current image frame, and calculate the real-time transmission frame rate of the current live broadcast based on the current bandwidth, the current real-time bit rate, and the preset transmission frame rate. Specifically, perform a division operation based on the current bandwidth and the current real-time bit rate to determine the transmission ratio. Further, perform a multiplication operation on the transmission ratio and the preset transmission frame rate to obtain the real-time transmission frame rate of the current live broadcast.

[0024] In a specific embodiment, when extracting a frame from the current image frame, the current image frame will be updated as the reference image frame, and then the next frame of the current image frame will be continued, and the next frame of the current image frame will be regarded as the current frame image again, and it will be determined whether to extract a frame from the current image frame again to ensure the image quality and smoothness of the current live broadcast.

[0025] Figure 2 In an alternative embodiment, determining the maturity threshold based on the current maturity information, the target maturity information, the remaining live broadcast time, the current real-time bitrate, the current bandwidth, and the preset transmission frame rate includes: S201: Determine a second maturity difference based on the current maturity information and the target maturity information; S203: Determine the standard transmission amount based on the remaining live broadcast time and the preset transmission frame rate; S205: Determine the transmission ratio based on the current real-time bitrate and the current bandwidth; S207: Determine the maturity threshold based on the second maturity difference, the transmission ratio, and the standard transmission amount.

[0026] In a specific embodiment, the second maturity difference may be the difference between the maturity characteristic information corresponding to the cooked cooking ingredients and the maturity characteristic information corresponding to the cooking ingredients in the current image frame; the transmission ratio may be the transmission ratio of the live image frames in the current live broadcast; the standard transmission amount may be the number of live image frames that should be transmitted during the remaining live broadcast time of the current live broadcast.

[0027] Specifically, based on the current maturity information and the target maturity information, a difference calculation is performed to obtain the second maturity difference. Further, the current real-time bitrate and the current bandwidth are divided, that is, the current real-time bitrate / the current bandwidth, to obtain the transmission ratio. Further, a multiplication operation is performed based on the remaining live broadcast time and the preset transmission frame rate to obtain the standard transmission amount. Further, the second maturity difference, the transmission ratio, and the standard transmission amount are multiplied to determine the maturity threshold.

[0028] In the above embodiment, the maturity threshold is calculated and determined through the current maturity information, the target maturity information, the remaining live broadcast time, the preset transmission frame rate, the current real-time bitrate, and the current bandwidth, which improves the accuracy of determining the maturity threshold. Further, the accuracy of frame extraction for live image frames is improved.

[0029] In an alternative embodiment, the target maturity information includes the target gray-scale information corresponding to the cooking ingredients, the current maturity information includes the current gray-scale information corresponding to the cooking ingredients, and the second maturity difference includes the second gray-scale difference; determining the current maturity information based on the cooking result of the ingredients includes: When the cooking processing result indicates that neither the contour nor the color of the cooking ingredient has changed, obtain the current grayscale information; The determining of the second maturity difference based on the current maturity information and the target maturity information includes: Based on the current grayscale information and the target grayscale information, perform a difference process to determine the second grayscale difference.

[0030] In a specific embodiment, the current grayscale information is used to indicate the grayscale value corresponding to the cooking ingredient in the current image frame; the target grayscale information can be the grayscale value corresponding to the cooking ingredient when the cooking ingredient is mature; the second grayscale difference can be the difference between the grayscale value corresponding to the cooking ingredient when it is mature and the grayscale value corresponding to the cooking ingredient in the current image frame; Specifically, when the cooking processing result obtained after the image model processes the current frame image is that neither the contour nor the color of the cooking ingredient has changed, the current grayscale information corresponding to the current image frame is obtained. Optionally, the current grayscale information is obtained by using an image processing algorithm to extract the grayscale value. Further, after obtaining the current grayscale information of the current image frame, the stored target grayscale information is called, and then based on the current grayscale information and the target grayscale information, a difference calculation is performed to obtain the second grayscale difference.

[0031] Further, when the second grayscale difference is obtained, calculate the grayscale maturity threshold based on the second grayscale difference, and then calculate the first grayscale difference based on the current grayscale information and the reference grayscale information. Further, compare the first grayscale difference with the grayscale maturity threshold.

[0032] In the above embodiment, the second grayscale difference is calculated through the current grayscale information and the target grayscale information, and the calculated second grayscale difference is applied to calculate the grayscale maturity threshold, and the grayscale maturity threshold is compared with the first grayscale difference calculated through the current grayscale information and the reference grayscale information, which improves the rationality and rigor of the solution, and also improves the rigor of frame extraction of the live image frame.

[0033] In an optional embodiment, the target maturity information includes the target area change rate corresponding to the cooking ingredient, the current maturity information includes the current area change rate corresponding to the cooking ingredient, and the second maturity difference includes the second area rate difference; the determining of the current maturity information based on the cooking result of the ingredient includes: When the cooking processing result indicates that the contour of the cooking ingredient has changed and the color of the cooking ingredient has not changed, obtain the current ingredient area and the initial ingredient area of the cooking ingredient; Based on the current ingredient area and the initial ingredient area, perform area calculation to determine the current area change rate; The determining of the second maturity difference based on the current maturity information and the target maturity information further includes: Based on the current area change rate and the target area change rate, perform difference processing to determine the second area rate difference.

[0034] In a specific embodiment, the current area change rate is used to indicate the area change rate of the cooking ingredient in the current image frame; the target area change rate can be the area change rate of the cooking ingredient when the cooking ingredient is mature; the second area change rate can be the difference between the area change rate of the cooking ingredient when it is mature and the area change rate of the cooking ingredient in the current image frame; the current ingredient area can be the area of the cooking ingredient in the current image frame; the initial ingredient area can be the area of the cooking ingredient when cooking starts; Specifically, when the cooking processing result indicates that the contour of the cooking ingredient has changed and the color of the cooking ingredient has not changed, obtain the current ingredient area and the initial ingredient area of the cooking ingredient, then calculate the area difference between the current ingredient area and the initial ingredient area, and divide the area difference between the current ingredient area and the initial ingredient area by the initial ingredient area to obtain the current area change rate. Further, based on the current area change rate and the target area change rate, perform subtraction calculation to obtain the second area rate difference, and then calculate the area rate maturity threshold based on the second area rate difference.

[0035] In the above embodiment, the second area rate difference is calculated through the current area change rate and the target area change rate, and the calculated second area change rate is used to calculate the area rate maturity threshold. Then, the area rate maturity threshold is compared with the first area rate difference calculated through the current area change rate and the reference area change rate, which improves the rationality and rigor of the solution, and further improves the comparison accuracy of the first maturity difference and the maturity threshold.

[0036] Figure 3 In an alternative embodiment, the first maturity difference includes a first area rate difference; the determining of the first maturity difference based on the current maturity information and the reference maturity information includes: S301: Obtain the reference ingredient area of the reference image frame; S303: Based on the reference ingredient area and the initial ingredient area, perform area calculation to determine the reference area change rate; S305: Based on the reference area change rate and the current area change rate, perform difference processing to determine the first area rate difference.

[0037] In a specific embodiment, the reference ingredient area may be the area of the cooking ingredient in the reference image frame; when the reference ingredient area of the reference image frame is obtained, a subtraction operation is performed on the initial ingredient area and the reference ingredient area to obtain the area difference between the initial ingredient area and the reference ingredient area. Further, the area difference between the initial ingredient area and the reference ingredient area is divided by the initial ingredient area to determine the reference area change rate. Further, a subtraction calculation is performed based on the current area change rate and the reference area change rate to obtain the first area rate difference. Further, the first area rate difference is compared with the area rate maturity threshold.

[0038] In an alternative embodiment, the current maturity information includes the current grayscale information and the current area change rate corresponding to the cooking ingredient, the target maturity information includes the target grayscale information and the target area change rate corresponding to the cooking ingredient, the second maturity difference includes the second area rate difference and the second grayscale difference, and the maturity threshold includes a weighted maturity threshold; determining the second maturity difference based on the current maturity information and the target maturity information includes: When the cooking processing result indicates that both the contour and color of the cooking ingredient have changed, based on the current grayscale information, the current area change rate, the target grayscale information, and the target area change rate, determine the second area rate difference and the second grayscale difference; The method further includes: Obtain the frame extraction ratio value and the weighting factor; Based on the weighting factor, the second area rate difference, and the second grayscale difference, perform a weighting process to determine the current weighted change value; Based on the current weighted change value and the frame extraction ratio value, determine the weighted maturity threshold.

[0039] In a specific embodiment, when the cooking processing result indicates that both the contour and color of the cooking ingredient have changed, based on the current grayscale information, the current area change rate, the target grayscale information, and the target area change rate, determine the second area rate difference and the second grayscale difference. Optionally, the specific refinement of determining the second area rate difference and the second grayscale difference based on the current grayscale information, the current area change rate, the target grayscale information, and the target area change rate is similar to the specific refinement of determining the second grayscale difference based on the current grayscale information and the target grayscale information and determining the second area rate difference based on the current area change rate and the target area change rate, and will not be elaborated here.

[0040] Further, when the cooking processing result indicates that both the contour and color of the cooking ingredients have changed, the frame extraction ratio value and the weighting factor are obtained. Correspondingly, based on the weighting factor, a weighted average process is performed on the calculated second gray difference value and the second area rate difference value to obtain the current weighted change value. A multiplication operation is performed on the current weighted change value and the frame extraction ratio value to determine the weighted maturity threshold. Further, based on the reference gray information and the current gray information, the first gray difference is calculated. Based on the reference area change rate and the current area change rate, the first area rate difference is calculated. Based on the weighting factor, a weighted average of the first gray difference and the first area rate difference is performed to determine the first weighted difference. Further, the first weighted difference is compared with the weighted maturity threshold.

[0041] In the above embodiment, when the cooking result indicates that both the contour and color of the cooking ingredients have changed, the weighted maturity threshold is determined based on the first area rate difference, the first gray difference, the weighting factor, and the frame extraction ratio value, and the weighted maturity threshold is compared with the first weighted difference determined by the first gray difference, the first area rate difference, and the weighting factor, which improves the diversity of determining the maturity threshold, and further improves the diversity and rationality of the solution. Whether to extract a frame is determined by comparing the weighted maturity threshold and the first weighted threshold, which improves the rigor of frame extraction for live images.

[0042] S111: Based on the real-time transmission frame rate, the live image frames in the live buffer data queue are transmitted in sequence.

[0043] In a specific embodiment, when the real-time transmission frame rate is calculated, the live image frames in the live buffer data queue are transmitted in sequence according to the real-time transmission frame rate. Specifically, the live image frames in the live buffer data queue are arranged in time sequence according to the first frame image of the adaptive area, and then transmitted from the first frame image of the adaptive area according to the real-time transmission frame rate.

[0044] In the above embodiment, based on the comparison result between the first maturity difference and the maturity threshold, the real-time transmission frame rate is calculated and the current live broadcast is controlled to be transmitted according to the real-time transmission frame rate, which meets the picture quality requirements and resolution requirements of the current live broadcast, enables the live image frames to be transmitted, and improves the timeliness of live image frame transmission and the smoothness of the current live broadcast.

[0045] In an alternative embodiment, the method further includes: When the current real-time bitrate is less than the current bandwidth and the data frame indication information indicates that the last frame of the image is in the adaptive region, or when the current real-time bitrate is less than the current bandwidth and the data frame indication information indicates that the live buffer data queue is empty, control the live image frames in the current live broadcast to be transmitted at the preset transmission frame rate.

[0046] In a specific embodiment, when the current real-time bitrate is less than the current bandwidth and the data frame indication information indicates that the last frame of the image is in the adaptive region, or when the current real-time bitrate is less than the current bandwidth and the data frame indication information indicates that the live buffer data queue is empty, it indicates that the live network speed of the current live broadcast can meet the transmission requirements. Furthermore, control the live data frames in the current live broadcast to be transmitted at the preset transmission frame rate.

[0047] In an alternative embodiment, the method further includes: When the current real-time bitrate is less than the current bandwidth and the data frame indication information indicates that the last frame of the image is in the frame extraction region, based on the current bandwidth, the current real-time bitrate, and the preset transmission frame rate, determine the real-time transmission frame rate, and cause the live image frames in the live buffer data queue to be transmitted at the re-determined real-time transmission frame rate.

[0048] In a specific embodiment, when the current real-time bitrate is less than the current bandwidth and the data frame indication information indicates that the last frame of the image is in the frame extraction region, it indicates that there are more live image frames stored in the current live buffer data queue, but the live network speed can meet the requirements. The live image frames cannot be transmitted at the preset transmission frame rate. Perform a division operation based on the current bandwidth and the current real-time bitrate to determine the transmission ratio. Then, perform a multiplication operation on the transmission ratio and the preset transmission frame rate to determine the real-time transmission frame rate. Furthermore, control the live image frames in the live buffer data queue to be transmitted at the real-time transmission frame rate.

[0049] In an alternative embodiment, the method further includes: When the current real-time bitrate is greater than the current bandwidth and the data frame indication information indicates that the last frame of the image is in the adaptive region, or when the current real-time bitrate is greater than the current bandwidth and the data frame indication information indicates that the live buffer data queue is empty, control the live image frames in the current live broadcast to be transmitted at the preset transmission frame rate, and store the new live image frames generated by the current live broadcast in the live buffer data queue.

[0050] When the current real-time bitrate is greater than the current bandwidth and the data frame indication information indicates that the last frame of the image is in the adaptive region, or when the current real-time bitrate is greater than the current bandwidth and the data frame indication information indicates that the live buffer data queue is empty, it indicates that the network speed of the current live broadcast has deteriorated, and the current live broadcast starts in a weak network environment. In this case, control the live image frames in the current live broadcast to be transmitted at a preset transmission frame rate, and store the new live image frames generated by the current live broadcast in the live buffer data queue.

[0051] In the above embodiments, by listing the comparison results of the current real-time bitrate and the current bandwidth, as well as different situations of the data frame indication information, the rationality of the solution is improved. And through the discussion of different situations, the live image frames in the current live broadcast can be transmitted at an appropriate transmission frame rate, improving the timeliness and security of the transmission of the live image frames in the current live broadcast.

[0052] In a specific embodiment, in this application, the current bandwidth, preset transmission frame rate, current real-time bitrate, remaining live time, data frame indication information corresponding to the live buffer data queue, target maturity information corresponding to the cooking ingredients, and reference maturity information corresponding to the reference image frame corresponding to the current live broadcast are obtained. And when the current real-time bitrate is greater than the current bandwidth and the data frame indication information indicates that the last frame of the image in the live buffer data queue is in the frame extraction region, the cooking result of the cooking ingredients is obtained, and the current maturity information is determined based on the cooking result of the ingredients. Optionally, the cooking result of the ingredients is used to indicate whether the outline and color of the cooking ingredients have changed, that is, based on different indication information of the cooking result of the ingredients, the corresponding maturity information can be obtained, improving the rationality of determining the current maturity information. Further, based on the determined current maturity information, target maturity information, current real-time bitrate, current bandwidth, and preset transmission frame rate, a maturity threshold is calculated. Based on the current maturity information and target maturity information in different situations, the corresponding maturity threshold is calculated and determined. Then, the maturity threshold is compared with the first maturity difference calculated and determined by the current maturity information and reference maturity information. And when the first maturity difference is greater than the maturity threshold, the current image frame is extracted, improving the rigor of the frame extraction of the live image frames and meeting the requirements of the picture quality and resolution of the current live broadcast. Further, based on the current bandwidth, current real-time bitrate, and preset transmission frame rate, the real-time transmission frame rate is calculated and determined, and the live image frames in the live buffer data queue are transmitted in order, improving the timeliness of the transmission of the live image frames and ensuring the smoothness of the current live broadcast.

[0053] Figure 4 is a schematic structural diagram of a transmission device provided according to an exemplary embodiment, introducing an embodiment of a transmission device of the present application. Specifically, as Figure 4As shown, the device includes: A first acquisition module 401, configured to acquire the current bandwidth corresponding to the current live broadcast, a preset transmission frame rate, the current real-time bit rate, the remaining live broadcast time, data frame indication information corresponding to the live broadcast buffer data queue, target maturity information corresponding to cooking ingredients, and reference maturity information corresponding to a reference image frame; the data frame indication information is used to indicate whether the last frame of the image in the live broadcast buffer data queue is located in the frame extraction area; the current image frame is an image frame acquired after the last frame of the image in the live broadcast buffer data queue; A second acquisition module 403, configured to acquire the cooking result of the cooking ingredients in the current image frame when the current real-time bit rate is greater than the current bandwidth and the data frame indication information indicates that the last frame of the image is located in the frame extraction area; A first determination module 405, configured to determine the current maturity information based on the cooking result of the ingredients; A second determination module 407, configured to determine a maturity threshold based on the current maturity information, the target maturity information, the remaining live broadcast time, the current real-time bit rate, the current bandwidth, and the preset transmission frame rate; A third determination module 409, configured to extract the current image frame when the first maturity difference between the current maturity information and the reference maturity information is greater than the maturity threshold, and determine the real-time transmission frame rate of the current live broadcast based on the current bandwidth, the current real-time bit rate, and the preset transmission frame rate; A transmission module 411, configured to sequentially transmit the live image frames in the live broadcast buffer data queue based on the real-time transmission frame rate.

[0054] In an optional embodiment, the above-mentioned second determination module 407 includes: A second difference determination unit, configured to determine a second maturity difference based on the current maturity information and the target maturity information; A standard transmission determination unit, configured to determine a standard transmission amount based on the remaining live broadcast time and the preset transmission frame rate; A transmission ratio determination unit, configured to determine a transmission ratio based on the current real-time bit rate and the current bandwidth; A threshold determination unit, configured to determine the maturity threshold based on the second maturity difference, the transmission ratio, and the standard transmission amount.

[0055] In an optional embodiment, the target maturity information includes target gray information, the current maturity information includes current gray information, and the second maturity difference includes a second gray difference; the above-mentioned first determination module 405 includes a gray determination unit, configured to: When the cooking processing result indicates that neither the contour nor the color of the cooking ingredient has changed, obtain the current grayscale information; The second difference determination unit described above includes a grayscale difference subunit for: Based on the current grayscale information and the target grayscale information, perform a difference process to determine the second grayscale difference.

[0056] In an optional embodiment, the target maturity information includes the target area change rate corresponding to the cooking ingredient, the current maturity information includes the current area change rate, and the second maturity difference includes the second area rate difference; the first determination module 405 includes: An area acquisition unit for obtaining the current ingredient area and the initial ingredient area of the cooking ingredient when the cooking processing result indicates that the contour of the cooking ingredient has changed and the color of the cooking ingredient has not changed; A current area rate determination unit for performing area calculation based on the current ingredient area and the initial ingredient area to determine the current area change rate; The second difference determination unit described above further includes an area rate difference subunit for: Based on the current area change rate and the target area change rate, perform a difference process to determine the second area rate difference.

[0057] In an optional embodiment, the first maturity difference includes the first area rate difference; the device further includes: A reference area acquisition unit for obtaining the reference ingredient area of the reference image frame; A reference area rate unit for performing area calculation based on the reference ingredient area and the initial ingredient area to determine the reference area change rate; A first area rate unit for performing a difference process based on the reference area change rate and the current area change rate to determine the first area rate difference.

[0058] In an optional embodiment, the current maturity information includes the current grayscale information and the current area change rate corresponding to the cooking ingredient, the target maturity information includes the target grayscale information and the target area change rate corresponding to the cooking ingredient, the second maturity difference includes the second area rate difference and the second grayscale difference, and the maturity threshold includes a weighted maturity threshold; the second difference determination unit further includes a comprehensive determination subunit for: When the cooking processing result indicates that both the outline and color of the cooking ingredients have changed, based on the current grayscale information, the current area change rate, the target grayscale information, and the target area change rate, determine the second area rate difference and the second grayscale difference; The above device further includes: An acquisition ratio and weighting unit, configured to acquire a frame extraction ratio value and a weighting factor; A weighted change determination unit, configured to perform a weighting process based on the weighting factor, the second area rate difference, and the second grayscale difference to determine a current weighted change value; A weighted threshold determination unit, configured to determine the weighted maturity threshold based on the current weighted change value and the frame extraction ratio value.

[0059] In an alternative embodiment, the above device further includes a first transmission unit, configured to: When the current real-time bit rate is less than the current bandwidth and the data frame indication information indicates that the last frame image is located in the adaptive region, or when the current real-time bit rate is less than the current bandwidth and the data frame indication information indicates that the live buffer data queue is empty, control the live image frames in the current live broadcast to be transmitted at the preset transmission frame rate.

[0060] In an alternative embodiment, the above device further includes a second transmission unit, configured to: When the current real-time bit rate is less than the current bandwidth and the data frame indication information indicates that the last frame image is located in the frame extraction region, determine the real-time transmission frame rate based on the current bandwidth, the current real-time bit rate, and the preset transmission frame rate, and cause the live image frames in the live buffer data queue to be transmitted at the real-time transmission frame rate.

[0061] In a possible embodiment, the above device further includes a third transmission unit, configured to: When the current real-time bit rate is greater than the current bandwidth and the data frame indication information indicates that the last frame image is located in the adaptive region, or when the current real-time bit rate is greater than the current bandwidth and the data frame indication information indicates that the live buffer data queue is empty, control the live image frames in the current live broadcast to be transmitted at the preset transmission frame rate, and store the new live image frames generated by the current live broadcast in the live buffer data queue.

[0062] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0063] In an exemplary embodiment, an electronic device is further provided, including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the instructions to implement the live data transmission method as in the embodiments of the present disclosure.

[0064] In an exemplary embodiment, a computer-readable storage medium is further provided. When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the live data transmission method in the embodiments of the present disclosure. In an exemplary embodiment, a computer program product or a computer program is further provided. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the live data transmission method provided in the above various optional implementation manners.

[0065] It can be understood that in the specific implementation manner of the present application, user-related data is involved. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use, and processing of relevant data need to comply with relevant laws, regulations, and standards of relevant countries and regions.

[0066] Those of ordinary skill in the art can understand that all or part of the processes of implementing the methods in the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it may include the processes of the embodiments of the above various methods. Among them, any reference to a memory, storage, database, or other medium used in the embodiments provided in the present application may include non-volatile and / or volatile memories. The non-volatile memory may include a read-only memory (ROM), a programmable ROM (PROM), an electrically programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM) or an external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0067] Other embodiments of the present disclosure will be readily apparent to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common general knowledge or conventional technical means in the technical field not disclosed herein. The specification and examples are only to be considered as exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0068] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A live broadcast data transmission method, characterized in that, The method includes: Obtaining the current bandwidth, preset transmission frame rate, current real-time bit rate, remaining live broadcast time, data frame indication information corresponding to the live broadcast buffer data queue, target maturity information corresponding to the cooking ingredients, and reference maturity information corresponding to the reference image frame; the data frame indication information is used to indicate whether the last frame image in the live broadcast buffer data queue is located in the frame extraction area; the current image frame is an image frame collected after the last frame image in the live broadcast buffer data queue; When the current real-time bit rate is greater than the current bandwidth and the data frame indication information indicates that the last frame image is located in the frame extraction area, obtaining the ingredient cooking result of the cooking ingredients in the current image frame; Determining the current maturity information based on the ingredient cooking result; Determining a maturity threshold based on the current maturity information, the target maturity information, the remaining live broadcast time, the current real-time bit rate, the current bandwidth, and the preset transmission frame rate; When the first maturity difference between the current maturity information and the reference maturity information is greater than the maturity threshold, extracting the current image frame, and determining the real-time transmission frame rate of the current live broadcast based on the current bandwidth, the current real-time bit rate, and the preset transmission frame rate; Transmitting the live broadcast image frames in the live broadcast buffer data queue in sequence based on the real-time transmission frame rate.

2. The method according to claim 1, wherein The determining the maturity threshold based on the current maturity information, the target maturity information, the remaining live broadcast time, the current real-time bit rate, the current bandwidth, and the preset transmission frame rate includes: Determining a second maturity difference based on the current maturity information and the target maturity information; Determining a standard transmission amount based on the remaining live broadcast time and the preset transmission frame rate; Determining a transmission ratio based on the current real-time bit rate and the current bandwidth; Determining the maturity threshold based on the second maturity difference, the transmission ratio, and the standard transmission amount.

3. The method according to claim 2, wherein The target maturity information includes the target gray-scale information corresponding to the cooking ingredients, the current maturity information includes the current gray-scale information corresponding to the cooking ingredients, and the second maturity difference includes a second gray-scale difference; the determining the current maturity information based on the ingredient cooking result includes: When the cooking processing result indicates that neither the contour nor the color of the cooking ingredients has changed, obtaining the current gray-scale information; The determining the second maturity difference based on the current maturity information and the target maturity information includes: Performing a difference process based on the current gray-scale information and the target gray-scale information to determine the second gray-scale difference.

4. The method according to claim 2, wherein The target maturity information includes the target area change rate corresponding to the cooking ingredients, the current maturity information includes the current area change rate corresponding to the cooking ingredients, and the second maturity difference includes a second area rate difference; the determining the current maturity information based on the ingredient cooking result includes: When the cooking processing result indicates that the contour of the cooking ingredient has changed and the color of the cooking ingredient has not changed, obtain the current ingredient area and the initial ingredient area of the cooking ingredient; Based on the current ingredient area and the initial ingredient area, perform area calculation to determine the current area change rate; The determining the second maturity difference based on the current maturity information and the target maturity information further includes: Based on the current area change rate and the target area change rate, perform difference processing to determine the second area rate difference.

5. The method according to claim 4, wherein The first maturity difference includes a first area rate difference; the method further includes: Obtain the reference ingredient area of the reference image frame; Based on the reference ingredient area and the initial ingredient area, perform area calculation to determine the reference area change rate; Based on the reference area change rate and the current area change rate, perform difference processing to determine the first area rate difference.

6. The method according to claim 2, wherein The current maturity information includes the current gray level information and the current area change rate corresponding to the cooking ingredient, the target maturity information includes the target gray level information and the target area change rate corresponding to the cooking ingredient, the second maturity difference includes a second area rate difference and a second gray level difference, and the maturity threshold includes a weighted maturity threshold; the determining the second maturity difference based on the current maturity information and the target maturity information further includes: When the cooking processing result indicates that both the contour and color of the cooking ingredient have changed, based on the current gray level information, the current area change rate, the target gray level information, and the target area change rate, determine the second area rate difference and the second gray level difference; The method further includes: Obtain the frame extraction ratio value and the weighting factor; Based on the weighting factor, the second area rate difference, and the second gray level difference, perform weighting processing to determine the current weighted change value; Based on the current weighted change value and the frame extraction ratio value, determine the weighted maturity threshold.

7. The method according to claim 1, characterized in that, The method further includes: When the current real-time bit rate is less than the current bandwidth and the data frame indication information indicates that the last frame image is located in the adaptive region, or when the current real-time bit rate is less than the current bandwidth and the data frame indication information indicates that the live broadcast buffer data queue is empty, control the live broadcast image frames in the current live broadcast to be transmitted at the preset transmission frame rate.

8. The method according to claim 1, characterized in that The method further includes: When the current real-time bit rate is less than the current bandwidth and the data frame indication information indicates that the last frame image is located in the frame extraction region, based on the current bandwidth, the current real-time bit rate, and the preset transmission frame rate, determine the real-time transmission frame rate, and cause the live broadcast image frames in the live broadcast buffer data queue to be transmitted at the real-time transmission frame rate.

9. The method according to claim 1, wherein The method further includes: When the current real-time bitrate is greater than the current bandwidth and the data frame indication information indicates that the last frame of the image is located in the adaptive region, or when the current real-time bitrate is greater than the current bandwidth and the data frame indication information indicates that the live buffer data queue is empty, control the live image frames in the current live broadcast to be transmitted at the preset transmission frame rate, and store the new live image frames generated by the current live broadcast in the live buffer data queue.

10. A live broadcast data transmission device, characterized in that, The device includes: A first acquisition module, configured to acquire the current bandwidth corresponding to the current live broadcast, the preset transmission frame rate, the current real-time bitrate, the remaining live broadcast time, the data frame indication information corresponding to the live buffer data queue, the target maturity information corresponding to the cooking ingredients, and the reference maturity information corresponding to the reference image frame; the data frame indication information is used to indicate whether the last frame of the image in the live buffer data queue is located in the frame extraction region; the current image frame is the image frame acquired after the last frame of the image in the live buffer data queue; A second acquisition module, configured to acquire the cooking result of the cooking ingredients in the current image frame when the current real-time bitrate is greater than the current bandwidth and the data frame indication information indicates that the last frame of the image is located in the frame extraction region; A first determination module, configured to determine the current maturity information based on the cooking result of the ingredients; A second determination module, configured to determine the maturity threshold based on the current maturity information, the target maturity information, the remaining live broadcast time, the current real-time bitrate, the current bandwidth, and the preset transmission frame rate; A third determination module, configured to extract the current image frame when the first maturity difference between the current maturity information and the reference maturity information is greater than the maturity threshold, and determine the real-time transmission frame rate of the current live broadcast based on the current bandwidth, the current real-time bitrate, and the preset transmission frame rate; A transmission module, configured to transmit the live image frames in the live buffer data queue in sequence based on the real-time transmission frame rate.

Images