Livestream picture processing method, apparatus, device and storage medium

Abstract

Provided in the embodiments of the present disclosure are a livestream picture processing method, an apparatus, a device and a storage medium. The method comprises: acquiring a captured current livestream picture frame, and, on the basis of historical subject state information within a previous historical duration, determining current subject state information of a livestream subject contained in the current livestream picture frame; on the basis of the current subject state information combined with historical picture adjustment information within the previous historical duration, determining current picture adjustment information of the current livestream picture frame; and, on the basis of the current picture adjustment information, performing picture adjustment on the current livestream picture frame so as to form a displayable target livestream picture frame. Using the method can determine corresponding picture adjustment information for livestream picture frames captured in real time in a livestream scenario, so as to use the picture adjustment information to adjust a picture presentation state used when the livestream picture frames are played. Livestreaming pictures formed on the basis of the processed livestream picture frames can present the effect of picture following shooting, thus effectively improving the picture visual perception of livestream pictures.

Description

Live screen processing method, device, equipment and storage medium

[0001] This application claims priority to the Chinese invention patent application entitled “A method, device, equipment and storage medium for processing live broadcast images” and application number 202311091570.1, filed on August 28, 2023. The entire contents of that application are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present disclosure relate to the field of live broadcast technology, and in particular to a method, apparatus, device, and storage medium for processing live broadcast images. Background Art

[0003] In the application scenario of online live broadcast, due to the limitations of shooting equipment and environment, the anchor side often uses the camera on the mobile terminal as a shooting prop, and uses the mobile terminal as the live broadcast execution end to realize online live broadcast.

[0004] In order to facilitate the anchor to better display their talents, mobile terminals are often processed in a fixed state, so that the live broadcast images during the live broadcast process are often presented from a fixed perspective. Compared with professional follow-up shooting, the existing live broadcast images have a weaker visual experience.

[0005] Summary of the Invention

[0006] The present disclosure provides a live image processing method, device, equipment and storage medium to effectively improve the image viewing experience during the live streaming image playback process.

[0007] In a first aspect, an embodiment of the present disclosure provides a method for processing a live broadcast image, the method comprising:

[0008] Acquire the captured current live picture frame, and determine the current object state information of the live objects contained in the current live picture frame based on the historical object state information within the previous historical duration;

[0009] Determining current picture adjustment information of the current live picture frame according to the current object state information and the historical picture adjustment information within the previous historical duration;

[0010] The current live picture frame is adjusted according to the current picture adjustment information to form a displayable target live picture frame.

[0011] In a second aspect, an embodiment of the present disclosure further provides a live image processing device, the device comprising:

[0012] A first determining module is configured to obtain a captured current live picture frame and determine current object state information of a live object contained in the current live picture frame based on historical object state information within a previous historical duration;

[0013] A second determining module is configured to determine the current picture adjustment information of the current live picture frame according to the current object state information and the historical picture adjustment information within the previous historical duration;

[0014] The target determination module is used to adjust the current live picture frame according to the current picture adjustment information to form a displayable target live picture frame.

[0015] In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising:

[0016] one or more processors;

[0017] a storage device for storing one or more programs,

[0018] When the one or more programs are executed by the one or more processors, the one or more processors implement the live screen processing method provided by the first aspect of the embodiment of the present disclosure.

[0019] In a fourth aspect, the embodiments of the present disclosure further provide a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to implement the live image processing method provided in the first aspect of the embodiments of the present disclosure.

[0020] In a fifth aspect, the embodiments of the present disclosure further provide a computer program product. The computer program product includes computer-executable instructions, which, when executed by a processor, implement the live image processing method provided in the first aspect of the embodiments of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0022] FIG1 is a flow chart of a method for processing live images according to an embodiment of the present disclosure;

[0023] FIG2 is a diagram illustrating an example of picture adjustment in a live picture processing method provided by an embodiment of the present disclosure;

[0024] FIG3 is a flow chart of a method for broadcasting live images provided by an embodiment of the present disclosure;

[0025] FIG4 is a schematic structural diagram of a live image processing device provided by an embodiment of the present disclosure;

[0026] FIG5 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0028] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0029] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0030] It should be noted that the concepts such as "first" and "target" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0031] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0032] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0033] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0034] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0035] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0036] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0037] Figure 1 is a flow chart of a live picture processing method provided by an embodiment of the present disclosure. The embodiment of the present disclosure is applicable to the situation of processing live pictures. The method can be executed by a live picture processing device, which can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device as an execution terminal, which can be a mobile terminal, a PC or a server, etc.

[0038] It should be noted that one application scenario of the live broadcast image processing method provided in this embodiment can be described as follows: For the host side in a live broadcast scene, a mobile terminal is usually used as a shooting tool to shoot the live broadcast image, and the mobile terminal usually processes a fixed shooting state, so that the captured live broadcast image is often played to the audience side with a fixed perspective. The live broadcast image formed by this shooting method affects the visual experience of the image during playback.

[0039] As shown in FIG1 , a live broadcast image processing method provided by an embodiment of the present disclosure may specifically include the following steps:

[0040] S110: Acquire the captured current live picture frame, and determine the current object state information of the live objects included in the current live picture frame based on the historical object state information within the previous historical duration.

[0041] In this embodiment, the execution subject of the method provided in this embodiment can be the execution subject of the live broadcast function on the anchor side. The execution subject can directly serve as a shooting tool to capture the live broadcast picture and obtain the captured live broadcast picture frame through this step; or the execution subject can be an external shooting tool, and receive the live broadcast picture frame captured by the external shooting tool through this step.

[0042] It can be known that each captured live picture frame can be processed through this embodiment, and the processing of the live picture frame in this embodiment can be continuous. Considering that each live picture frame is processed in the same way, the live processing process is described from the perspective of the current execution moment, and this embodiment can record the live picture frame to be processed at the current execution moment as the current live picture frame.

[0043] Generally, in live broadcast scenarios, the live screen often displays objects such as the host or other participants, or their virtual avatars. These objects displayed in the live screen can be considered the objects that the audience will focus on. These objects need to be kept within the viewing angle of the camera to ensure that they can be captured in the live screen. In this embodiment, objects that are captured in the live screen can be recorded as live objects.

[0044] To process the current live video frame, this embodiment first needs to determine the data related to the live object contained in the current live video frame. Live objects do not remain stationary throughout the live scene; they may move freely within the permitted viewing angle. When a live object is in motion, its state may vary across live video frames captured at different times. Therefore, in this embodiment, the data related to the live object can be embodied as object state information.

[0045] Continuing with the above description, the current object status information can be understood as an information set that records the position of the live object in the current live picture frame, the size of the picture it occupies, the posture and motion state it possesses, the number of objects displayed, the overall composition formed, and the scene to which it belongs. This step can determine the current object status information based on the historical object status information determined within the previous historical duration. The previous historical duration can be understood as an execution time period of a certain length of time before the current execution moment, such as the duration consisting of 3 seconds from the current execution moment forward, or the duration consisting of 5 seconds from the current execution moment forward.

[0046] In this embodiment, the object status information corresponding to each historical live picture frame contained in the previous historical time length can be collectively referred to as historical object status information. That is, it can be considered that the previous historical time length can contain multiple historical live picture frames, and each historical live picture frame corresponds to object status information. After the object status information is summarized, it can be collectively recorded as historical object status information.

[0047] In this embodiment, the process of determining the current object state information of the live broadcast object in the current live broadcast picture frame can be described as follows: first, the current key point information of the live broadcast object can be detected from the current live broadcast picture frame, and the position of the live broadcast object in the current picture frame, the size of the picture occupied, the scene type (such as close-up, long shot or close-up, etc.), and the composition of the live broadcast object can be determined through the current key point information; then, the object state information of the live broadcast object in each historical live broadcast picture frame included in the previous historical duration can be combined to determine what kind of motion trajectory the live broadcast object exhibits from the previous historical duration to the current execution moment, what kind of posture it presents, and whether the number of live broadcast objects increases or decreases; finally, the above determined information can be summarized to obtain the current object state information associated with the live broadcast object in the current live broadcast picture frame.

[0048] S120: Determine current picture adjustment information of the current live picture frame according to the current object state information and the historical picture adjustment information within the previous historical duration.

[0049] In this embodiment, the current live picture frame is processed for presentation status, and it is relatively critical to first determine the picture adjustment information used for presentation status processing. In this embodiment, the picture adjustment information corresponding to the current live picture frame can be recorded as the current picture adjustment information, and the current picture adjustment information can include the position information to be adjusted, the size information to be adjusted, and the adjustment action information of what kind of adjustment action to use when adjusting the current live picture frame. Among them, the adjustment action can include translation and rotation of the screenshot, and can also include zooming and capturing relative to the current live picture frame. It can also include a combination of at least two operations such as translation, rotation, and zooming, such as translation plus rotation, translation plus zooming, etc.

[0050] In this embodiment, this step can be used to determine the current picture adjustment information. Specifically, the current picture adjustment information can be determined by combining the current object state information determined above with the historical picture adjustment information relied upon when adjusting the historical live pictures contained in the previous historical duration. It is understandable that because the live object may process the running state and posture change state at different execution times, when determining the current picture adjustment information in this embodiment, it is necessary to consider the impact of the posture change of the live object within the previous historical duration on the picture adjustment of the current live picture frame.

[0051] For example, the posture information of the live broadcast object in each historical live broadcast picture frame corresponding to the previous historical duration can be analyzed to determine whether the posture of the live broadcast object at the current execution moment is still changing along the posture change trend of the historical execution moment in the previous historical duration. If it is still changing along the original posture change trend, the current picture adjustment information of the current live broadcast picture frame can be directly determined through the historical picture adjustment information. For example, the previous picture adjustment information corresponding to the previous live broadcast picture frame of the current live broadcast picture frame can be found from the historical picture adjustment information to determine the current picture adjustment information of the current live broadcast picture frame. How to directly use the previous picture adjustment information as the current picture adjustment information; or the previous adjustment offset information involved in the previous picture adjustment information can be used as the current adjustment offset information, and then the previous adjustment offset information is combined with the current object state information to determine the final current picture adjustment information.

[0052] Continuing with the above description, if it is determined that the posture of the live broadcast object at the current execution moment has not changed along the posture change trend of the historical execution moment within the previous historical duration, it can be considered that the posture of the live broadcast object at the current execution moment is not closely related to the historical posture. In this case, the current object state information determined above can be used to first determine the initial picture adjustment information for adjusting the current live broadcast picture frame. The initial picture adjustment information may include the initial adjustment position, initial adjustment size and adjustment action for adjusting the current live broadcast picture; then the picture adjustment action, adjustment strength and adjustment duration that match the current live broadcast picture frame can be further determined through the current object state information, thereby determining the adjustment offset information, which is recorded as the current adjustment offset information in this embodiment. The current adjustment offset information is equivalent to further optimization of the initial picture adjustment information, and the current picture adjustment information is formed by integrating the current adjustment offset information and the initial picture adjustment information.

[0053] S130: Adjust the current live picture frame according to the current picture adjustment information to form a displayable target live picture frame.

[0054] In this embodiment, the purpose of determining the current picture adjustment information is mainly to adjust the current live picture frame. This step can implement the adjustment logic of the current live picture frame. In this embodiment, the adjustment of the current live picture frame can be understood as performing image conversion and rendering operations on the current live picture frame.

[0055] In one of the implementation methods, adjustment position information and adjustment size information related to picture adjustment can be obtained from the current picture adjustment information. The specific adjustment position and adjustment size can be determined from the current live picture frame through the adjustment position information and adjustment size, so that the picture area to be captured can be found in the current live picture frame. After taking a screenshot of the picture area, the display form can be adjusted according to the current picture adjustment information to obtain the adjusted target picture in the current live picture frame. In order to ensure the consistency of the live picture display size, this step can restore the picture size of the obtained target picture to the picture size of the current live picture frame, and finally form the target live picture frame that can be output to the audience live end.

[0056] The live picture processing method provided by this embodiment can determine the picture adjustment information of the live picture frames captured in real time in the live scene, and adjust the corresponding presentation state of the live picture frames through the picture adjustment information. The technical solution provided by this embodiment is equivalent to making up for the shortcomings of using fixed shooting tools for fixed-angle shooting in live scenes. It can realize the diversification of the live picture frame playback angle from the algorithm level by processing the live picture frames captured under a fixed angle. That is, when the live stream picture formed by the adjusted live picture frames is displayed to the audience side, the live picture can present the display effect of the angle corresponding to the moving follow-up shooting, thereby effectively improving the picture perception of the live pictures involved in the live scene without increasing the shooting cost.

[0057] Based on the live image processing method provided in the above embodiment, the method may be further optimized as follows. Specifically, the method may further include the following steps:

[0058] According to the set adjustment information optimization strategy, the current picture adjustment information is corrected to obtain the corrected current picture adjustment information.

[0059] It's important to note that, from a global perspective in a live broadcast scenario, if the current live frame's presentation is adjusted using the current image adjustment information determined above, errors in the current image adjustment information might cause the adjusted target live frame to appear abruptly within the overall live stream. In other words, the target live frame might appear isolated from the style changes of the already played live frames.

[0060] Based on this, this embodiment can optimize and correct the current picture adjustment information through the above-mentioned optimized and added steps, and then adjust the current live picture frame through the corrected current picture adjustment information. The target live picture frame obtained by this processing method can present a smoother and more stable display effect in the entire live stream picture, thereby better improving the picture viewing experience.

[0061] Specifically, an implementation method for optimizing and correcting the current picture adjustment information can be described as: summarizing the historical target live picture frames formed after adjustment processing through historical picture adjustment information within the previous historical time length; analyzing the change trend information of the live broadcast object in the historical target live broadcast picture frames, and performing curve fitting through the change trend information to obtain the theoretical change trend curve of the changes of the live broadcast object in the target live broadcast picture frame.

[0062] Continuing with the above description, the theoretical change trend curve can be used to predict the presentation status information that the live broadcast object should have in the target live broadcast picture frame corresponding to the current live broadcast picture frame. This embodiment can reversely infer the theoretical picture adjustment information corresponding to the current live broadcast picture frame based on the presentation status information. Thereafter, the current picture adjustment information can be corrected based on the theoretical picture adjustment information, and the error information in the current picture adjustment information can be filtered out. The corresponding theoretical information in the theoretical picture adjustment information is used to replace the error information, and finally the corrected current picture adjustment information is obtained.

[0063] The additional picture adjustment information correction logic in this embodiment can ensure that the target live picture frame determined after adjustment can present a smoother and more stable display effect in the entire live stream picture.

[0064] For example, in order to facilitate understanding of the adjustment of the live broadcast picture in the present method, the live broadcast object is a person as an example for demonstration. FIG2 is an example diagram of picture adjustment in a live broadcast picture processing method provided by an embodiment of the present disclosure. As shown in FIG2, S21 is a current live broadcast picture frame that has not been adjusted, wherein the rectangular frame corresponding to S211 is the outer bounding box of the live broadcast object determined according to the position of the live broadcast object, S22 is the current live broadcast picture frame determined by the current picture adjustment information, and the rectangular frame S221 in the middle position is the initial picture determined according to the determined current picture adjustment information. The current picture The adjustment information may include a push-in action, a push-in distance, and a cropping ratio of 0.8. S23 is the current live picture frame determined after correction by the current adjustment offset information. S231 is the rectangular box corresponding to the current picture adjustment information after the current offset information is offset. S24 is the current live picture frame obtained after the current picture adjustment information is corrected according to the set adjustment information optimization strategy. S241 is the rectangular box corresponding to the corrected current picture adjustment information. The cropping ratio is 0.7, and the adjustment action includes a rotation of y degrees. After adjustment, the target live picture frame shown in S26 can be obtained.

[0065] FIG3 is a flow chart of a method for broadcasting live video provided by an embodiment of the present disclosure. The embodiment of the present disclosure further illustrates the above embodiment. As shown in FIG3 , the video playback method can be specifically optimized to include the following steps:

[0066] S301: Acquire the captured current live picture frame and determine the current key point information of the live object contained in the current live picture frame.

[0067] In this embodiment, the current key point information can be understood as the set of all key points used to reflect the current motion posture of the live broadcast subject. By marking and connecting all the current key points of the live broadcast subject, the motion posture of the live broadcast subject can be determined. If the live broadcast subject is a real person or an avatar, the current key points can be joints with rotational characteristics, such as 18 key points, including: nose, eyes, ears, neck, shoulders, elbows, wrists, hips, knees, and ankles.

[0068] Specifically, the processor can obtain the captured current live picture frame, first identify the live object in the current live picture frame, and since the processing of the live picture is aimed at the movement changes of the live object to determine the adjustment information, in order to accurately reflect the movement changes of the live object, the processor can detect all current key points reflecting the current movement posture of the live object to determine the current key point information of the live object in the current live picture frame.

[0069] S302. Determine the current scale information and current composition scene information of the live object in the current live frame based on the current key point information, and determine the current motion state information, current posture information, and current object quantity information of the live object in combination with historical object state information.

[0070] It is important to know that when adjusting the live screen, the adjustment is usually made by referring to the subject's movements and the magnitude of the movement change compared to the previous live screen frame. For example, when the subject is a person, when the person stretches their arms, the camera movement usually used is to gradually pan the live screen along the direction and magnitude of the arm stretch. When the number of people increases or decreases, the corresponding live screen needs to be zoomed in or out along with the position of the person, and the zooming in or out needs to be combined with the corresponding shooting composition method and shot type to fit the actual shooting effect. That is, when adjusting the live screen, the motion state, posture and number of the subject, the shooting composition method and shot type, etc. need to be used as reference standards for adjusting the live screen.

[0071] In this embodiment, the current scale information can be understood as information reflecting the size of the live object in the current live frame, as well as the size of the key parts of the live object in the current live frame. The current composition and shot information can be understood as information reflecting the composition method and shot size for the live object during shooting. For example, the composition method can be a center composition that places the live object in the center of the shooting screen, a thirds composition that places the live object in one-third of the shooting screen, etc., and the shot size can be a close-up, a close-up, or a panoramic view. The current motion state information can be understood as information representing the motion trajectory of the live object. The current posture information can be understood as posture information representing the action of the live object, such as sideways, sideways, with the back turned, and with arms extended. The current object quantity information can be understood as information about the number of live objects included in the current live frame.

[0072] Specifically, since the current scale information is used to represent the live object in the live picture, the processor can determine the outer bounding box of the live object based on the current key point information, so as to select the live object by fitting the outer edge of the live object and determine the object scale corresponding to the live object. Since the face of the live object is the most critical point during live broadcast, the center point corresponding to the face and the center point of the live object as a whole can be determined from the current key point information, and this information can be used as the current scale information of the live object in the current live picture frame. The processor can determine the shot type based on the current key point information of the live object in the current live picture frame relative to the picture space area, that is, the proportion of the live object to the live picture space, and determine the composition method based on the current key point information to determine the position of the live object in the live picture, thereby determining the current composition shot information.

[0073] Specifically, the processor can determine the motion trajectory of the live object in the form of a coordinate sequence using the outer bounding box position, facial center point, and overall center point in the current scale information and historical scale information. To address the accuracy error caused by key point detection, the predicted position of the live object in the previous frame can be fused and filtered with the actual position of the live object in the current frame to obtain the filtered current position of the live object, thereby determining the change distance, change direction, and range occupied by the live object in the live frame over a period of time as the current motion state information. The processor can combine the historical key point information in the historical object state information for a certain period of time with the current key point information. For example, each key point can be represented in the form of coordinates. By comparing the coordinates of the historical key points with the current key points, the change amplitude of each key point of the live object can be obtained. The processor can then determine the corresponding posture based on the movement amplitude of each key point combined with a set change threshold to obtain the current posture information of the live object. For example, if the change distance of the key point corresponding to the elbow exceeds a certain range, it can be determined that the live object is performing an arm extension action, and its corresponding posture is an arm extension posture.

[0074] Specifically, the processor can also determine the current number of objects in the current live picture frame through the number of current key point information and the distribution of each current key point, and determine whether the current live picture frame is in a steady state of people (i.e., whether the number of people has changed) in combination with the number of historical objects in the previous live picture frame, and determine the steady state type (i.e., the number of people in the current live picture frame: no one, one person, or many people), and use the steady state of people and the steady state type as the current object number information. Among them, when the current number of objects changes for the first time relative to the number of historical objects, the steady state type and the steady state of people of this live picture frame are not adjusted. It is necessary to refer to the changes in subsequent frames. If changes occur for several consecutive frames and the changes are consistent, the steady state type and the steady state of people can be adjusted.

[0075] S303: Summarize the current size information, the current composition scene information, the current motion state information, the current posture information, and the current object quantity information to form the current object state information of the live broadcast object.

[0076] Specifically, the processor may aggregate current size information, current composition scene information, current motion state information, current posture information, and current object quantity information to form current object state information of the live broadcast object.

[0077] S304: Determine whether the current live picture frame meets the conditions for continuing to use the previous picture adjustment based on the historical posture information in the historical object state information.

[0078] In this embodiment, the adoption condition can be understood as a condition for determining whether to adopt the historical screen adjustment information.

[0079] What needs to be known is that since each live broadcast picture frame is a decomposition of a continuous action, a continuous action can be determined through several live broadcast picture frames. The picture adjustment corresponding to a continuous action should be consistent and gradual to ensure the effect of the picture adjustment. Therefore, it is necessary to first identify the continuous action. The historical posture information can be used to determine the historical posture of the live broadcast object in the historical frames. The current posture information can be used to determine the current posture of the live broadcast object. By judging whether the two postures are consistent, it can be determined whether the live broadcast object is still in the process of a continuous action, or starting to do the next action, and then the picture adjustment strategy can be determined based on the judgment situation.

[0080] Specifically, the processor can determine whether the live object in the current live screen frame is still in the same posture change process as the previous live screen frame by comparing the consistency of historical posture information and current posture information in the historical object status information, that is, determine whether the current live screen frame meets the conditions for continuing to use the previous screen adjustment through the consistency.

[0081] S305: If the conditions are met, determine the current adjustment offset information of the current live picture frame according to the historical picture adjustment information.

[0082] In this embodiment, the current adjustment offset information can be understood as information for determining the degree of picture adjustment for the current live picture frame.

[0083] Specifically, if the current live picture frame meets the conditions for continuing to use the previous picture adjustment, that is, the current posture of the live object is still in the previous posture process, it is necessary to ensure that the picture adjustment made to the current live picture frame is similar to the picture adjustment made to the previous live picture frame. The processor can adjust the historical picture adjustment information to determine the current adjustment offset information of the current live picture frame.

[0084] Furthermore, based on the above embodiment, the step of determining the current adjustment offset information of the current live picture frame according to the historical picture adjustment information may be optimized, including:

[0085] a1. Obtain previous adjustment offset information corresponding to a previous live picture frame of the current live picture frame from historical picture adjustment information.

[0086] In this embodiment, the previous live picture frame can be understood as the live picture frame corresponding to the moment before the current live picture frame according to the time sequence. The previous adjustment offset information can be understood as information for performing offset adjustment on the previous live picture frame.

[0087] Specifically, the processor may search the historical picture adjustment information in chronological order to obtain the previous adjustment offset information corresponding to the previous live picture frame of the current live picture frame.

[0088] b1. Determine the current adjustment offset information of the current live picture frame according to the previous adjustment offset information.

[0089] Specifically, the processor can determine the duration corresponding to the current live picture frame based on the information representing the duration in the previous adjustment offset information, and then combine other information corresponding to the previous adjustment offset information with the duration to obtain the current adjustment offset information of the current live picture frame.

[0090] S306: If not satisfied, determine the current adjustment offset information of the current live picture frame according to the current object state information.

[0091] Specifically, if the current live picture frame does not meet the conditions for continuing to use the previous picture adjustment, that is, the current posture of the live object is not in the previous posture process, the processor can determine the adjustment method and adjustment intensity of the current live picture frame based on the current posture information and current scale information of the live object in the current object status information, and obtain the current adjustment offset information.

[0092] Furthermore, based on the above embodiment, the current adjustment offset information of the current live picture frame can be determined according to the current object state information and optimized as follows:

[0093] a2. According to the current posture information and the current composition scene information in the current object state information, a current picture adjustment action that matches the current live picture frame is determined from a preset picture adjustment information library.

[0094] In this embodiment, the image adjustment information library can be understood as a pre-defined information library used to determine how to perform image adjustments. Image adjustment actions can be mapped to required judgment conditions. Image adjustments can include actions such as zooming, panning, zooming, lens vibration, and fixed lens. When broken down into individual image frames, these actions correspond to translation, rotation, and zooming. The current image adjustment action can be understood as the method of adjusting the current image.

[0095] Specifically, the picture adjustment information library records the mapping relationship between different picture adjustment actions and the corresponding posture information and composition scene information. The processor can search the current posture information and current composition scene information in the current object state information in the preset picture adjustment information library, determine the mapped current picture adjustment action, and obtain the current picture adjustment action that matches the current live picture frame.

[0096] b2. Determine the current adjustment intensity and the current adjustment duration of the current live picture frame according to the current motion state information in the current object state information.

[0097] In this embodiment, the current adjustment intensity can be understood as indicating the degree of adjustment during the image adjustment, such as the rotation angle, translation distance, and zoom ratio. The current adjustment duration can be understood as the duration of the current image adjustment action. For example, if the gesture is a head turn, the duration required for the head turn is 3 seconds. If the image adjustment action under the same gesture needs to be consistent, the adjustment duration of the image adjustment action can also be 3 seconds.

[0098] What you need to know is that the live broadcast screen needs to be adjusted along with the live broadcast object. As the live broadcast object's movements and position change, the live broadcast screen should also change accordingly. Since the live broadcast object changes in different amplitudes, the corresponding live broadcast screen change amplitudes should also change accordingly. That is, the adjustment corresponding to the screen adjustment action is intensity-limited. However, for the continuous movement of the live broadcast object, it has a predictable duration, such as stretching movements, and the corresponding screen adjustment action should also have an adjustment duration.

[0099] Specifically, the processor can determine the changes in the motion trajectory of the live broadcast object, the relative position of the live broadcast object in the picture, the cropping ratio for displaying the live broadcast object in a certain proportion, and the intensity of preventing lens shake and other parameter information based on the current motion state information in the current object state information, thereby determining the current adjustment intensity and the current adjustment duration based on the relevant parameter information.

[0100] For example, based on the current motion state information, it is determined that the live broadcast object has moved x distance to the left, and the corresponding current adjustment intensity is also to adjust x distance to the left. Since it is predicted that the overall movement of the live broadcast object may not be a continuous action, the current adjustment duration is 0.

[0101] c2. Determine current adjustment offset information of the current live picture frame relative to the current picture adjustment action based on the current adjustment strength and the current adjustment duration.

[0102] Specifically, the processor may use the current adjustment intensity and the current adjustment duration as the current adjustment offset information of the current live picture frame relative to the current picture adjustment action.

[0103] Through such a setting, based on the current adjustment strength and the current adjustment duration, the current adjustment offset information of the current live picture frame relative to the current picture adjustment action is determined, thereby ensuring the smoothness and stability of the adjustment action.

[0104] S307: Determine initial picture adjustment information of the current live picture frame, and determine current picture adjustment information based on the initial picture adjustment information and the current adjustment offset information.

[0105] In this embodiment, the initial picture adjustment information can be understood as adjustment information for protecting the composition and information integrity of the live object, and the current picture adjustment information can be understood as picture adjustment information obtained after adjustment according to the current adjustment offset information.

[0106] Specifically, the processor can determine the parameter information required for picture trajectory calculation based on the current motion state information and current object quantity information of the live broadcast object, such as the relative position of the live broadcast object in the picture, the cropping ratio, the anti-shake strength and other information, calculate the initial picture adjustment information based on the object trajectory of the live broadcast object and the calculated parameter information, and correct the corresponding items in the initial picture adjustment information based on the current adjustment offset information to determine the current picture adjustment information.

[0107] For example, if the cropping ratio of the current live picture frame is 0.8, the current adjustment offset information includes reducing the cropping ratio by 0.1 and shifting by a distance x, then the corresponding current picture adjustment information is shifting by a distance x and a cropping ratio of 0.7.

[0108] Based on the above embodiment, the step of determining the current picture adjustment information according to the initial picture adjustment information and the current adjustment offset information may be further optimized, including:

[0109] a3. Determine initial picture adjustment information of the current live picture frame according to the current motion state information and the current object quantity information in the current object state information.

[0110] Specifically, the processor can determine the parameter information required for picture trajectory calculation based on the current motion state information and the current object quantity information in the current object state information, such as the relative position of the live object in the picture, the cropping ratio, the anti-shake strength and other information, thereby determining the initial picture adjustment information of the current live picture frame.

[0111] b3. Superimpose the current adjustment offset information on the initial picture adjustment information, and determine the superimposed picture adjustment information as the current picture adjustment information.

[0112] Specifically, the processor may search the initial picture adjustment information for the same adjustment item as that in the current adjustment offset information, superimpose the same adjustment items, and determine the superimposed picture adjustment information as the current picture adjustment information.

[0113] Through such a setting, after determining the initial picture adjustment information, the current adjustment offset information is superimposed to ensure the integrity of the character composition and image, ensure the smoothness of the adjustment action, and avoid sudden changes in the position of the live broadcast object.

[0114] S308: Extracting the adjustment size information, the adjustment position information, and the adjustment action information from the current picture adjustment information.

[0115] In this embodiment, the size adjustment information can be understood as the size information used to crop the image. The position adjustment information can be understood as the position information corresponding to the center of the image. The action adjustment information can be understood as information including the action, amplitude, and duration of the action.

[0116] Specifically, the processor may extract the adjustment size information, the adjustment position information, and the adjustment action information included in the current picture adjustment information.

[0117] S309: Capture a target image from the current live image frame according to the size adjustment information and the position adjustment information.

[0118] In this embodiment, the target picture can be understood as the live picture obtained after being intercepted.

[0119] Specifically, the processor can determine the target picture corresponding to the current live screen according to the adjustment size information and the adjustment position information, determine and capture the target picture from the current live screen frame, such as determining the center position in the adjustment position information, and determining a selection box in the current live screen according to the adjustment size information and the center position. The picture selected by the selection box is the target picture.

[0120] S310: Adjust the presentation state of the target picture according to the adjustment action information, and restore the adjusted target picture to the picture size of the current live picture frame to form a target live picture frame.

[0121] In this embodiment, the screen size can be understood as the screen size displayed during the live broadcast.

[0122] Specifically, the processor can adjust the presentation status of the target picture according to the adjustment action, adjustment intensity and duration in the adjustment action information, and stretch or reduce the adjusted target picture to the picture size of the current live picture frame to form a target live picture frame for live broadcast.

[0123] The configuration of this embodiment determines the current key point information of the live object in the current live screen frame, and based on the current key point information, determines the current motion state information, current posture information, and current object quantity information of the live object, thereby obtaining the current object state information. Before determining the current adjustment offset information, it determines whether to use the previous adjustment offset information based on historical posture information. This avoids the problems of frequent determination of the current live screen adjustment information and insufficient effect diversity. The current screen adjustment action is determined through a preset screen adjustment information library, and the current adjustment intensity and current adjustment duration determined through the current motion state information are combined to form the current screen adjustment information. This realizes the automatic recognition of live screen adjustment, enhances the smoothness and stability of live screen adjustment, achieves the follow-up effect of the live screen, and improves the viewing experience of the live screen.

[0124] The disclosed embodiments provide a live picture processing method, apparatus, device and storage medium, which first obtains the captured current live picture frame, and determines the current object status information of the live object contained in the current live picture frame based on the historical object status information within the previous historical duration; then determines the current picture adjustment information of the current live picture frame based on the current object status information combined with the historical picture adjustment information within the previous historical duration, and finally performs picture adjustment on the current live picture frame based on the current picture adjustment information to form a displayable target live picture frame. The above technical solution can realize the determination of the picture adjustment information of the live picture frame captured in real time in the live scene, and realize the corresponding presentation state adjustment of the live picture frame through the picture adjustment information. The technical solution provided by this embodiment is equivalent to making up for the deficiency of using fixed shooting tools for fixed-angle shooting in live broadcast scenes. It can realize the diversification of live broadcast frame playback angles by processing the live broadcast picture frames captured at a fixed angle from the algorithm level. That is, it is equivalent to adjusting the live stream picture formed by the processed live broadcast picture frames and displaying it to the audience side, so that the live broadcast picture can present the display effect of the angle corresponding to the moving follow-up shooting, thereby effectively improving the picture perception of the live broadcast pictures involved in the live broadcast scene without increasing the shooting cost.

[0125] FIG4 is a schematic diagram of the structure of a live image processing device provided by an embodiment of the present disclosure. As shown in FIG4 , the device includes:

[0126] The first determining module 41 is configured to obtain the captured current live picture frame and determine the current object state information of the live objects contained in the current live picture frame based on the historical object state information within the previous historical duration;

[0127] A second determining module 42 is configured to determine current picture adjustment information of the current live picture frame based on the current object state information and the historical picture adjustment information within the previous historical duration;

[0128] The target determination module 43 is configured to perform picture adjustment on the current live picture frame according to the current picture adjustment information to form a displayable target live picture frame.

[0129] It is possible to determine the picture adjustment information of the live picture frames captured in real time in the live scene, and adjust the corresponding presentation state of the live picture frames through the picture adjustment information. The technical solution provided by this embodiment is equivalent to making up for the shortcomings of using fixed shooting tools for fixed-angle shooting in live scenes. It can realize the diversification of the live picture frame playback angle by processing the live picture frames captured under a fixed angle from the algorithm level. That is, when the live stream picture formed by adjusting the processed live picture frames is displayed to the audience side, the live picture can present the display effect of the angle corresponding to the moving follow-up shooting, thereby effectively improving the picture perception of the live pictures involved in the live scene without increasing the shooting cost.

[0130] Furthermore, the first determining module 41 is specifically configured to:

[0131] Determining current key point information of the live broadcast object contained in the current live broadcast picture frame;

[0132] Determine, based on the current key point information, current scale information and current composition scene information of the live object in the current live picture frame, and determine, in combination with the historical object state information, current motion state information, current posture information, and current object quantity information of the live object;

[0133] The current size information, current composition scene information, current motion state information, current posture information and current object quantity information are summarized to constitute the current object state information of the live broadcast object.

[0134] Furthermore, the second determining module 42 includes:

[0135] A first determining unit is configured to determine whether the current live picture frame satisfies a condition for continuing to adjust the previous picture based on the historical posture information in the historical object state information;

[0136] A second determining unit is configured to determine, if satisfied, current adjustment offset information of the current live picture frame according to the historical picture adjustment information;

[0137] a third determining unit, configured to determine, if the condition is not satisfied, current adjustment offset information of the current live picture frame according to the current object state information;

[0138] The fourth determining unit is configured to determine initial picture adjustment information of the current live picture frame, and determine the current picture adjustment information according to the initial picture adjustment information and the current adjustment offset information.

[0139] The second determining unit is specifically configured to:

[0140] Acquire the previous adjustment offset information corresponding to the previous live picture frame of the current live picture frame from the historical picture adjustment information;

[0141] The current adjustment offset information of the current live picture frame is determined according to the previous adjustment offset information.

[0142] The third determining unit is specifically configured to:

[0143] Determining, from a preset picture adjustment information library, a current picture adjustment action that matches the current live picture frame according to the current posture information and the current composition scene information in the current object state information;

[0144] Determining a current adjustment intensity and a current adjustment duration of the current live picture frame according to the current motion state information in the current object state information;

[0145] According to the current adjustment strength and the current adjustment duration, current adjustment offset information of the current live picture frame relative to the current picture adjustment action is determined.

[0146] The fourth determining unit is specifically configured to:

[0147] Determining initial picture adjustment information of the current live picture frame according to current motion state information and current object quantity information in the current object state information;

[0148] The current adjustment offset information is superimposed on the initial picture adjustment information, and the superimposed picture adjustment information is determined as the current picture adjustment information.

[0149] Furthermore, the target determination module 43 is specifically configured to:

[0150] Extracting adjustment size information, adjustment position information, and adjustment action information from the current picture adjustment information;

[0151] intercepting a target picture from the current live picture frame according to the adjustment size information and the adjustment position information;

[0152] The presentation state of the target picture is adjusted according to the adjustment action information, and the adjusted target picture is restored to the picture size of the current live picture frame to form the target live picture frame.

[0153] Optionally, the device further includes: a third determining module.

[0154] The third determining module is configured to perform adjustment information correction on the current picture adjustment information according to a set adjustment information optimization strategy to obtain corrected current picture adjustment information.

[0155] The live screen processing device provided in the embodiments of the present disclosure can execute the live screen processing method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0156] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present disclosure.

[0157] FIG5 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Referring to FIG5 , a schematic diagram of the structure of an electronic device (e.g., a terminal device or server in FIG5 ) 500 suitable for implementing an embodiment of the present disclosure is shown below. The terminal device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. The electronic device shown in FIG5 is merely an example and should not impose any limitations on the functions and scope of use of the embodiments of the present disclosure.

[0158] As shown in FIG5 , the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 505. Various programs and data required for the operation of the electronic device 500 are also stored in the RAM 503. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An edit / output (I / O) interface 505 is also connected to the bus 504.

[0159] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although FIG5 shows the electronic device 500 with various devices, it should be understood that not all of the devices shown are required to be implemented or present. More or fewer devices may alternatively be implemented or present.

[0160] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0161] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0162] The electronic device provided by the embodiment of the present disclosure and the live image processing method provided by the above embodiment belong to the same inventive concept. For technical details not fully described in this embodiment, please refer to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0163] An embodiment of the present disclosure provides a computer storage medium on which a computer program is stored. When the program is executed by a processor, the live image processing method provided by the above embodiment is implemented.

[0164] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0165] In some embodiments, the client and server can communicate using any currently known or later developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or later developed network.

[0166] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0167] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains a virtual model corresponding to the physical object to which the target object belongs, and a model map corresponding to the target object, wherein the model map is pre-determined by the physical object color presented by the target object in the physical object; according to the position correspondence between the virtual model and the reference model, performs map color extraction from the model map to obtain map color information, and there is a mapping relationship between the vertex position on the reference model and the pixel point of the model map; renders and displays the virtual model according to the map color information, and the virtual object color presented by the target object in the virtual model is correlated with the physical object color.

[0168] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0169] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0170] The units involved in the embodiments described in this disclosure may be implemented in software or hardware. In some cases, the name of a unit does not limit the unit itself. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses."

[0171] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0172] In the context of this disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing.

[0173] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0174] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0175] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A method for processing live broadcast images, comprising: Acquire the captured current live picture frame, and determine the current object state information of the live object contained in the current live picture frame according to the historical object state information in the previous historical duration; Determine the current picture adjustment information of the current live picture frame according to the current object state information and the historical picture adjustment information within the previous historical duration; The current live picture frame is adjusted according to the current picture adjustment information to form a displayable target live picture frame.

2. The method according to claim 1, wherein determining the current object state information of the live object contained in the current live picture frame according to the historical object state information in the previous historical duration comprises: Determine current key point information of the live object contained in the current live picture frame; Determine the current scale information and current composition scene information of the live object in the current live picture frame according to the current key point information, and determine the current motion state information, current posture information and current object quantity information of the live object in combination with the historical object state information; The current size information, the current composition scene information, the current motion state information, the current posture information and the current object quantity information are summarized to constitute the current object state information of the live broadcast object.

3. The method according to claim 1, wherein determining the current picture adjustment information of the current live picture frame according to the current object state information combined with the historical picture adjustment information within the previous historical duration comprises: Determining whether the current live picture frame satisfies a condition for continuing to use the previous picture adjustment according to the historical posture information in the historical object state information; If satisfied, determining the current adjustment offset information of the current live picture frame according to the historical picture adjustment information; If not satisfied, determining the current adjustment offset information of the current live picture frame according to the current object state information; Determine the initial picture adjustment information of the current live picture frame, and determine the current picture adjustment information according to the initial picture adjustment information and the current adjustment offset information.

4. The method according to claim 3, wherein determining the current adjustment offset information of the current live picture frame according to the historical picture adjustment information comprises: Acquire the previous adjustment offset information corresponding to the previous live picture frame of the current live picture frame from the historical picture adjustment information; The current adjustment offset information of the current live picture frame is determined according to the previous adjustment offset information.

5. The method according to claim 3, wherein determining the current adjustment offset information of the current live picture frame according to the current object state information comprises: According to the current posture information and the current composition scene information in the current object state information, determine the current picture adjustment action that matches the current live picture frame from a preset picture adjustment information library; Determining a current adjustment intensity and a current adjustment duration of the current live picture frame according to the current motion state information in the current object state information; According to the current adjustment strength and the current adjustment duration, current adjustment offset information of the current live picture frame relative to the current picture adjustment action is determined.

6. The method according to claim 3, wherein the determining the initial picture adjustment information of the current live picture frame, and determining the current picture adjustment information according to the initial picture adjustment information and the current adjustment offset information, comprises: Determining initial picture adjustment information of the current live picture frame according to current motion state information and current object quantity information in the current object state information; The current adjustment offset information is superimposed on the initial picture adjustment information, and the superimposed picture The adjustment information is determined to be the current picture adjustment information.

7. The method according to claim 1, wherein the step of adjusting the current live picture frame according to the current picture adjustment information to form a displayable target live picture frame comprises: Extracting adjustment size information, adjustment position information and adjustment action information from the current picture adjustment information; Capturing a target picture from the current live picture frame according to the adjustment size information and the adjustment position information; The presentation state of the target picture is adjusted according to the adjustment action information, and the adjusted target picture is restored to the picture size of the current live picture frame to form the target live picture frame.

8. The method according to any one of claims 1 to 7, further comprising: According to the set adjustment information optimization strategy, the current picture adjustment information is corrected to obtain the corrected current picture adjustment information.

9. A live picture processing device, comprising: A first determination module is used to obtain the captured current live picture frame, and determine the current object state information of the live object contained in the current live picture frame according to the historical object state information in the previous historical duration; A second determination module is used to determine the current picture adjustment information of the current live picture frame according to the current object state information combined with the historical picture adjustment information within the previous historical duration; The target determination module is used to adjust the current live picture frame according to the current picture adjustment information to form a displayable target live picture frame.

10. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the live screen processing method as described in any one of claims 1-8.

11. A storage medium comprising computer executable instructions, wherein the computer executable instructions are used to execute the live picture processing method as described in any one of claims 1 to 8 when executed by a computer processor.

12. A computer program product comprising computer executable instructions, wherein the computer executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 8.

Images