Picture display method and apparatus, device, and storage medium

Abstract

Disclosed in the embodiments of the present disclosure are a picture display method and apparatus, a device, and a storage medium. The method comprises: in response to a video playback instruction, displaying a video stream picture on a video playback page; and in response to a wake-up instruction of an application panel, determining a first display area and a second display area on the video playback page, the first display area being used for displaying the application panel, the second display area being used for displaying a key picture area in the video stream picture, and the key picture area being part of image areas in the video stream picture. Using the method can better prevent the problem of wakened application panels blocking the content of key pictures of video streams on video playback pages, and can perform other operations in application panels without affecting the playback of the content of the key pictures of displayed objects in video stream pictures, thereby better improving the user experience for video stream playback.

Description

A method, apparatus, device, and storage medium for displaying images

[0001] This application claims priority to Chinese Patent Application No. 202311049185.0, filed on August 18, 2023, entitled "A screen display method, apparatus, device and storage medium", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This disclosure relates to the field of computer technology, and in particular to a screen display method, apparatus, device, and storage medium. Background Technology

[0003] In the current live streaming industry, the live streaming panel that displays the live stream often includes other triggerable functional controls, such as information input controls, live gift-related controls, shopping list space, and live stream duration display controls.

[0004] In the current implementation of live streaming display, triggering a function control on the live streaming panel directly displays the corresponding function panel on the live streaming panel layer. In this display method, the function panel obscures the original live streaming content on the live streaming panel, thus affecting the live streaming viewing experience.

[0005] Summary of the Invention

[0006] This disclosure provides a screen display method, apparatus, computer device, and storage medium to effectively display a portion of the image area of ​​the object displayed in the video screen when the video playback page is obstructed.

[0007] In a first aspect, embodiments of this disclosure provide a screen display method, the method comprising: displaying a video stream screen on a video playback page in response to a video playback command; and determining a first display area and a second display area on the video playback page in response to an application panel activation command; wherein the first display area is used to display the application panel, and the second display area is used to display a key screen area within the video stream screen, the key screen area being a portion of the image area in the video stream screen.

[0008] Secondly, embodiments of this disclosure also provide a screen display device, the device comprising: a first display module, configured to display a video stream image on a video playback page in response to a video playback command; and a second display module, configured to determine a first display area and a second display area on the video playback page in response to an application panel activation command; wherein the first display area is used to display the application panel, and the second display area is used to display a key image area within the video stream image, the key image area being a portion of the image area in the video stream image.

[0009] Thirdly, this disclosure also provides a computer device, which includes: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the screen display method provided in any embodiment of this disclosure.

[0010] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the screen display method provided in any embodiment of this disclosure. Attached Figure Description

[0011] To more clearly illustrate the technical solutions of the exemplary embodiments of this disclosure, the accompanying drawings used in describing the embodiments are briefly introduced below. Obviously, the accompanying drawings described are only a portion of the embodiments to be described in this disclosure, and not all of them. For those skilled in the art, other drawings can be obtained from these drawings without any creative effort.

[0012] Figure 1 is a flowchart illustrating a screen display method provided in an embodiment of this disclosure;

[0013] Figure 2a shows the effect of displaying the video stream only after the video playback page is invoked;

[0014] Figure 2b shows the presentation effect of the video stream when the application panel is invoked and displayed after the video playback page in the prior art;

[0015] Figure 2c shows the presentation effect of the video stream when the application panel is invoked again after the video playback page using the method provided in this embodiment;

[0016] Figure 2d shows an effect diagram of the key screen area determined in the video stream of the method provided in this embodiment;

[0017] Figure 2e shows the effect when the proportion of the object in the screen is greater than a set threshold in the method provided in this embodiment;

[0018] Figure 2f shows the effect on the video playback page when the proportion of the object in the screen is greater than a set threshold in the method provided in this embodiment;

[0019] Figure 3 is a schematic diagram of the structure of a screen display device provided in an embodiment of this disclosure;

[0020] Figure 4 is a schematic diagram of the structure of a computer device provided in an embodiment of this disclosure. Detailed Implementation

[0021] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0022] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0023] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0024] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules, or units, and are not used to limit the order of functions performed by these devices, modules, or units or their interdependencies. It should also be noted that the modifications of "a" and "a plurality of" mentioned in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0025] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0026] It is understood that before using the technical solutions disclosed in the embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained. Furthermore, in the collection and use of object processing information, users' personal information will not be collected or processed in a manner that does not reveal the user's identity.

[0027] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0028] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0029] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0030] Figure 1 is a flowchart illustrating a screen display method according to an embodiment of this disclosure. This embodiment is applicable to situations where there is obstruction in the live broadcast screen, and the method can be executed by a screen display device. This device can be implemented by software and / or hardware and can be configured in a terminal and / or server to implement the screen display method in this embodiment of the disclosure.

[0031] It should be noted that one application scenario of this embodiment can be described as follows: In live streaming applications that support live viewing, the live stream on the broadcaster's side can be displayed through the live video playback page on the viewer's side. As the functions in live streaming applications become increasingly diverse, many function activation controls are presented on the video playback page for viewers to activate other functions during the live stream. Under the current display mechanism, the application panel of the activated function is overlaid on the already presented video playback page, obscuring the live stream on the video playback page. In some cases, this overlay display can obscure important live content in the live stream, thus affecting the viewer's viewing experience.

[0032] The screen display method provided in this embodiment can effectively avoid the aforementioned problems in existing screen display methods. Specifically, as shown in Figure 1, the screen display method provided in this embodiment may include steps S101 and S102.

[0033] S101. In response to a video playback command, display the video stream on the video playback page.

[0034] In this embodiment, the video playback page can be considered as a playback interface for playing video streams. When the application associated with the video playback page is a live streaming application, the video stream displayed on the video playback page can be a live stream forwarded from the broadcaster's side. It should be noted that the scenario in which the video playback page plays video streams in this embodiment can be any video playback scenario. This embodiment uses a live streaming scenario as an example, but it is not limited to the implementation in a live streaming scenario.

[0035] In this embodiment, the video playback command can be generated by triggering a playback component or playback link entry in the associated playback application software. This step can respond to the video playback command. For example, taking a live streaming application software installed on the viewer's side as an example, the live streaming application software can contain multiple live streaming link entries to different anchors' live streaming rooms. By triggering any live streaming link entry, a video playback command corresponding to that anchor's live streaming room can be generated, and this step can respond to the video playback command and display the video playback page corresponding to that anchor's live streaming room, so that the relevant live stream footage can be played on the video playback page.

[0036] S102. In response to the application panel wake-up command, a first display area and a second display area are determined on the video playback page, wherein the first display area is used to display the application panel, and the second display area is used to display key image areas within the video stream, wherein the key image areas are partial image areas within the video stream.

[0037] In this embodiment, the video playback page invoked after responding to the video playback command through the above steps may also include other triggerable application functions. Taking the live streaming application scenario as an example, the invoked video playback page may include application functions of other applications involved in the live stream. For example, it may include a function to trigger a gift revenue panel, which can be used to display gifts earned during the live stream; or a function to trigger a live shopping list panel, which can be used to provide a purchase channel for products recommended by the streamer during the live stream. In addition, there may also be functions that trigger other application panels, such as information input functions and online user display functions.

[0038] In this embodiment, after triggering any application function item on the video playback page, this step can be used to receive the wake-up command of the application panel associated with that application function item. By responding to the wake-up command, the video playback page can be divided into a first display area and a second display area. The wake-up application panel can be displayed in the first display area and the key screen area in the second display area.

[0039] It is known that the page size of the invoked video playback page and the panel size of the application panel can be predetermined. Normally, the panel size of the application panel will be smaller than the page size of the video playback page. Therefore, this embodiment can divide the video playback page into two display areas. The first display area can be the same size as the application panel and is used to display the invoked application panel. The area outside the first display area on the video playback page can serve as the second display area.

[0040] It should be noted that this embodiment does not specifically limit the location of the first and second display areas on the video playback page. However, it is preferable to divide the bottom border of the video playback page into a display area whose size is the same as the application panel size as the first display area. The area on the video playback page that is not obscured by the first display area is then designated as the second display area.

[0041] It is also understood that the first display area used to display the application panel will obscure the original video stream on the video playback page, leaving only the second display area to display the video stream content normally. When displaying the video stream in the second display area, the method provided in this embodiment can be used to determine a portion of the image area from the video stream as a key image area, and display the key image area in the second display area, thereby reducing the impact of the application panel obscuring the video stream.

[0042] In this embodiment, the key image area can be considered as a portion of the image area determined within the video stream. The size of this portion of the image area can be the same as or smaller than the displayable size of the second display area, wherein the displayable size is smaller than the original image size of the video stream. The selection of this portion of the image area can be achieved by performing feature recognition on the video stream, identifying objects of interest that viewers would focus on, determining the image area occupied by these objects within the video stream, and then selecting a portion of the image area from these occupied areas.

[0043] It is known that not all content in the video stream displayed on the video playback page is crucial. Viewers often pay more attention to dynamic objects in the video stream, such as people or pets, or products being promoted and sold by the host. This embodiment can select the objects that viewers are more interested in as the focus areas, thereby preferentially selecting a portion of the image area from the image region containing the focus areas, and designating the selected portion of the image area as the key screen area.

[0044] Meanwhile, the central area of ​​the video stream is often the area that viewers will focus on. When feature recognition is performed on the video stream and no object of focus for the viewer is identified, an image area no larger than the second display area can be taken from the central area of ​​the video stream as a partial image area.

[0045] In this embodiment, to display the key image area in the second display area, it is necessary to determine the key image area. In this embodiment, the identification results of the object of interest in the video stream are used. If the identification results confirm that the video stream contains the object of interest, the position of the object of interest in the video stream is determined. Then, based on the position information of the object of interest and the area size of the second display area, the corresponding portion of the image area of ​​interest is selected from the video stream and designated as the key image area. This embodiment allows the key image area to be displayed in the second display area. Since the selected key image area contains part of the image content of the object of interest, this embodiment effectively ensures unobstructed playback of the portion of the image area containing the object of interest.

[0046] As described above, when the identification results determine that the video stream does not contain the object of interest, it can be assumed that the importance of the content presented in the video stream is basically the same. Therefore, a portion of the image area in the video stream that matches the displayable size of the second display area can be selected as the key image area. Generally, in this embodiment, when determining the key image area in this way, any portion of the image area in the video stream that matches the displayable size can be selected as the key image area. Considering that important content is often displayed in the middle area of ​​the video playback page, this embodiment preferably selects a portion of the image area in the middle of the video stream that matches the displayable size as the key image area. In some embodiments, the middle image area can be selected using the horizontal centerline of the video stream.

[0047] In this embodiment, the identified key screen area can be displayed in the second display area by moving the playback position of the video stream or by directly taking a screenshot of the key screen area in the video stream.

[0048] For example, to better understand the effect of the method provided in this embodiment in a video streaming playback scenario, Figure 2a shows the effect of the video stream displayed only after the video playback page is invoked. As shown in Figure 2a, the display screen shows the video playback page 21 invoked after launching a playback application. The page size of the video playback page 21 can be the same as the screen size of the display screen, or it can be slightly smaller than the screen size. The video stream 210 is displayed in the video playback page 21, wherein the circled area 211 in the video stream 210 can be considered as the location of the object of the viewer's attention.

[0049] Figure 2b shows the presentation effect of the video stream when the application panel is displayed after the video playback page in the prior art. As shown in Figure 2b, the application panel 22 is overlaid on the video playback page 21. The application panel 22 obscures the content of the video stream 210, especially the content of the circled area 211 shown in Figure 2a.

[0050] Figure 2c shows the presentation effect of the video stream when the application panel is invoked and displayed after the video playback page using the method provided in this embodiment. As shown in Figure 2c, the video playback page 21 is divided into a first display area 212 and a second display area 213 (the part with bold lines). The first display area 212 is used to display the invoked application panel 22. Through the method provided in this embodiment, a portion of the image area can be determined from the circled area 211 shown in Figure 2a as the key image area of ​​the video stream, and the key image area can be adjusted and displayed in the second display area 213, effectively alleviating the problem of the invoked application panel obscuring the key image content in the video stream in the prior art.

[0051] This embodiment provides a screen display method that, in response to an invoked application panel during video stream playback, divides the video playback page into two display areas. One area displays the application content of the application panel, while the other area displays key areas of the image region of the object shown in the video stream. Compared to existing technologies, this method better alleviates the problem of the invoked application panel obscuring key content of the video stream on the video playback page. It allows viewers to perform other operations in the application panel without affecting their viewing of key content in the video stream, thus significantly improving the user viewing experience.

[0052] Based on the above embodiments, the step of determining the key screen area within the video stream can be further optimized into steps a1) and b1).

[0053] a1) Determine the playback scene associated with the video stream, and determine the target object based on the playback scene.

[0054] In this embodiment, as described above, one implementation for determining keyframe regions primarily considers locating the viewer's target object within the video stream, thereby defining the portion of the image containing the target object as the keyframe region. In this embodiment, the viewer's target object within the video stream can be designated as the target object.

[0055] It's important to understand that the playback scenarios associated with the video stream displayed on a video playback page are often diverse, and the objects of focus for viewers within the video stream will differ depending on the playback scenario. Therefore, this embodiment can determine different target objects based on the different playback scenarios associated with the video stream. This embodiment can first determine the playback scenario associated with the video stream through this step, and then determine the specific target object based on the playback scenario.

[0056] This embodiment can predetermine the playback scene associated with different video stream frames, and determine the target object corresponding to the playback scene associated with the video stream frame through this step.

[0057] It is known that viewers focus on different aspects of the video content presented in different playback scenarios. For example, in playback scenarios involving the sale of goods or product explanations, viewers are more focused on the items being sold or explained. In contrast, in performance-based playback scenarios where the host is performing, viewers are more focused on the objects participating in the performance or creating entertainment, such as people, pets, or virtual characters. This embodiment can determine the target object matching the playback scenario by combining the actual content of the video stream after determining the playback scenario.

[0058] As one implementation, determining the playback scene associated with the video stream frame and determining the target object based on the playback scene can be specified as steps a11), a12), and a13).

[0059] a11) Determine the playback scene associated with the video stream based on the video attribute information of the video to which the video stream belongs.

[0060] In this embodiment, the playback scene associated with the video stream can be determined based on the video attribute information of the video being played, such as tag information, title information, or other video attributes. For example, when the video being played is tagged as a shopping live stream, the playback scene associated with the video stream can be determined as a live shopping scene. Similarly, when the video being played is tagged as a live streamer performance or live streamer show, the playback scene associated with the video stream can be determined as a live performance scene.

[0061] a12) If the playback scenario belongs to the first type of playback scenario, then the first object of interest set relative to the first type of playback scenario is determined as the target object.

[0062] In this embodiment, the playback scenarios associated with the video stream can be broadly divided into a first type of playback scenario and a second type of playback scenario. The first type of playback scenario may include one or more active participants performing, rapping, showcasing skills, or engaging in other activities. In this first type of playback scenario, the participants in the activity demonstration can be designated as the primary focus. It is understood that the participants involved in different videos may differ. For example, in the given live performance scenario, the main focus is on the activities of people or pets. In this embodiment, the people or objects participating in the live performance can be designated as the primary focus in this live performance scenario, preferably as the target object in this first type of playback scenario.

[0063] In this embodiment, the specific participant corresponding to a video stream frame can be determined by analyzing the attribute information involved in the video stream. In some embodiments, the attribute information may include key content of the video stream, video title, and video tags.

[0064] a13) If the playback scenario belongs to the second type of playback scenario, then the second object of interest relative to the device in the second type of playback scenario is determined as the target object, wherein the first type of playback scenario is different from the second type of playback scenario.

[0065] In this embodiment, the second type of playback scenario can be a scenario where items to be sold, products to be explained, or content to be explained are being presented, demonstrated, or sold. This second type of playback scenario allows the objects being explained, demonstrated, or sold to be designated as the second focus object. Similarly, the objects being displayed differ in different videos. For example, in an e-commerce shopping live stream, the displayed object might be a product being explained or demonstrated by the host; while in an educational live stream, the displayed object might be teaching materials being presented by the instructor. This embodiment can also determine the specific displayed object corresponding to the video stream image by analyzing the attribute information involved in the video stream.

[0066] The above technical solution in this embodiment provides a specific implementation for determining the target object, and the determined target object can be used for subsequent determination of key screen areas.

[0067] b1) Based on the feature recognition results of the target object in the video stream, determine a portion of the image region from the video stream as the key image region.

[0068] In this embodiment, after identifying the target object through the above steps, it can be determined whether the target object exists in the video stream by recognizing its features. It is understandable that, regardless of the playback scenario, there are situations where the target object does not appear in the video stream. For example, in the first type of playback scenario, the participants in the activity may only appear as sound, while the person or other performers themselves do not appear on screen, and no related visuals are presented. Similarly, in the second type of playback scenario, the object being sold or the content being explained may not be specifically presented in the frame due to issues such as the shooting angle.

[0069] Based on this, this embodiment can perform feature recognition of the target object in the video stream to determine whether the video stream currently contains the target object, and can provide different methods for determining the key screen area based on the recognition results of whether the target object features are contained in the video stream.

[0070] Following the above description, as one implementation, determining a portion of the image region as the key image region from the video stream based on the feature recognition results of the target object in the video stream can be specified as steps b11), b12), and b13).

[0071] b11) Perform feature recognition on the target object in the video stream.

[0072] In this embodiment, the feature recognition logic of the target object can be executed through this step, and the corresponding recognition result can be determined. For example, an object feature matching the target object can be identified in the video stream; or no object feature matching the target object can be identified in the video stream. Depending on the different recognition results, the following different key screen area determination steps can be executed.

[0073] b12) If the feature recognition result is that no object feature matching the target object is identified, then according to the displayable size of the second display area, a first part of the image area is selected from the video stream and the first part of the image area is used as the key screen area.

[0074] In this embodiment, as long as the target object is not identified from the video stream, the critical image area can be determined through this step. It is understood that when the target object does not exist in the video stream, the criticality of different image areas in the video stream can be considered to be basically the same. In this case, this embodiment only needs to ensure that the selected image area can be displayed in the second display area.

[0075] Therefore, this step can select an image region in the video stream whose size is no larger than the displayable size of the second display area as the first part of the image region, and use this first part of the image region as the key frame area. It should be noted that, to ensure maximum utilization of the display space of the second display area, this embodiment can select an image region whose size is the same as the displayable size as the first part of the image region.

[0076] Furthermore, considering that the main content is often played in the center of the video playback page, that is, considering that viewers pay more attention to the content in the middle of the video stream than to the content at the ends, this embodiment can also select an image area with the same displayable size from the middle of the video stream as the first part of the image area.

[0077] Specifically, in this embodiment, selecting a first portion of the image area from the video stream based on the displayable size of the second display area, and optimizing the first portion of the image area as the key screen area includes the following steps b121) and b122).

[0078] b121) Determine the horizontal centerline of the video stream image along the vertical direction of the video playback page.

[0079] In this embodiment, the vertical direction can be relative to the bottom and top of the video playback page, and can be considered as the extension direction of the straight line formed by connecting the bottom and top vertices of the video playback page. This step can determine a horizontal central axis along this vertical direction that bisects the video stream frame. The height of this horizontal central axis in the vertical direction can be considered as half the height of the video stream frame.

[0080] b122) In the video stream, a first part of the image area is selected with the horizontal central axis as the symmetry line of the selected area and the displayable size as the size of the selected area, and the selected first part of the image area is used as the key screen area.

[0081] In this embodiment, the symmetry line of the selected area can be understood as the axis of symmetry of the image area to be selected in the video stream. This step uses the horizontal central axis as the symmetry line of the selected area. Additionally, this step can limit the size of the selected image area to be the same as the displayable size of the second display area. The image area selected by the above two limiting conditions can be used as the first part of the image area in the video stream, and this step can record this first part of the image area as the key frame area.

[0082] For example, Figure 2d shows an effect demonstration of the key image area determined in the video stream frame by the method provided in this embodiment. As shown in Figure 2d, the straight line 23 (which is not actually displayed in the video stream frame) is the horizontal central axis in the video stream frame area 24. Using this straight line 23 as the starting position, image areas can be delineated in the video stream frame along the arrow directions of the two dashed lines 25, and the size of the selected area is limited to be the same as the displayable size of the second display area. This forms the first part of the image area, which is denoted as the key image area 26 in the video stream frame.

[0083] The execution of steps b121) and b122) in this embodiment provides an implementation method for determining key screen areas when no target object is identified. By using the key screen areas determined through the steps provided in this embodiment, it can be ensured that the content of the video stream located in the middle of the video playback page is presented unobstructed in the second display area.

[0084] b13) If the feature recognition result is that an object feature matching the target object is identified, then according to the proportion of the target object in the video stream, the second part of the image region of the target object is selected, and the second part of the image region is used as the key image region.

[0085] In this embodiment, this step can be used as a step to determine the key screen region under another recognition result of feature recognition of the target object. In this embodiment, after determining that the target object is in the video stream, this step can be used to further determine the screen proportion of the target object in the video stream, and then the key screen region can be specifically determined by the ratio of the screen proportion.

[0086] For example, when the ratio of screen proportions is greater than or equal to a predetermined proportion threshold, the importance of the screen content in the entire video stream is considered to be substantially the same. Therefore, the original screen area already displayed in the video stream and presented in the second display area can be directly determined as the second part of the image area of ​​the target object, and this second part of the image area can be used as the key screen area. Similarly, when the screen proportion is less than the proportion threshold, the image area occupied by the target object in the video stream can be considered as a candidate area for selecting the key screen area.

[0087] In one implementation, a second portion of the image region of the target object can be further selected based on the coordinates of the target object displayed in the candidate area (such as the position of the top of the object), and this can be used as the key frame region. For example, if the top position of the target object falls within the sub-region defined in the second display area, the original image region already displayed in the second display area can be directly determined as the second portion of the image region of the target object; alternatively, if the top position of the target object does not fall within the sub-region defined in the second display area, the second portion of the image region of the target object can be defined by combining the center point of the target object with the displayable size of the second display area, thus forming the key frame region within the video stream.

[0088] The execution of steps b11) to b13) in this embodiment provides different implementation methods for determining key screen areas under different recognition results of the target object. The key screen areas determined by the steps provided in this embodiment can better match the actual playback scenario of the video stream and can better ensure that the content that the viewer wants to see is effectively presented in the second display area.

[0089] Following the above description, as one implementation, this embodiment can select a second part of the image region of the target object based on the proportion of the target object in the video stream frame, and use the second part of the image region as the key frame region, which is specifically defined as the following steps b131), b132), and b133).

[0090] b131) Determine the proportion of the target object in the video stream frame.

[0091] In this embodiment, after identifying the target object in the video stream, its position information within the video stream can be determined. This position information allows us to determine the area of ​​the screen occupied by the target object. This step involves calculating the ratio between this area and the total area of ​​the video stream, and the determined ratio can be used as the object's percentage of the screen.

[0092] b132) If the proportion of the object screen is greater than or equal to the first set threshold, then the screen area that has been displayed in the second display area in the video stream screen is determined as the second part of the image area of ​​the target object, and the second part of the image area is used as the key screen area.

[0093] In this embodiment, the first set threshold can be understood as an empirical value used to compare the ratio of the image screen proportions. In this step, when the object screen proportion is greater than or equal to the first set threshold, the screen area already displayed in the second display area of ​​the video stream can be directly used as the second part of the image area of ​​the target object, and this second part of the image area can be determined as the key screen area.

[0094] In this case, the target object can be considered to occupy a large proportion of the entire video stream. This means that most of the content presented in the video stream is relatively critical. Therefore, when the conditions of this step are met, it is equivalent to not needing to adjust the content already presented in the second display area of ​​the video stream. The original image area in the second display area of ​​the video stream can be considered the second part of the target object's image area, and the content presented in this second part of the image area can be considered the content of the critical image area.

[0095] Figure 2e shows the effect when the object's screen proportion exceeds a set threshold in the method provided in this embodiment. As shown in Figure 2e, the video stream frame 27 contains the target object 271. The target object 271 almost fills the entire video stream frame 27, which can be considered as the screen proportion of the target object 271 to the video stream frame 27 being too large, and the actual value of the screen proportion may have exceeded the set threshold.

[0096] For example, in the scenario shown in Figure 2e, Figure 2f shows the effect on the video playback page when the proportion of the object in the screen exceeds a set threshold in the method provided in this embodiment. As shown in Figure 2f, the video playback page 28 includes a first display area 281 and a second display area 282, wherein the application panel 29 is displayed in the first display area 281. Regarding the second display area 282, in the scenario where the proportion of the target object in the screen exceeds the set threshold as shown in Figure 2e, the second display area 282 still displays the image area of ​​the target object that was originally displayed in the second display area (this originally displayed image area is shown in Figure 2e, equivalent to the target object partial screen area 272 shown in Figure 2e). That is, it is equivalent to directly using the originally displayed image area as the second part of the image area determined when the proportion of the screen exceeds the set threshold.

[0097] b133) If the screen proportion is less than the first set threshold, then based on the second baseline set in the video stream screen and combined with the object position information of the target object, the key screen area in the video stream screen is determined.

[0098] In this embodiment, if the screen area is less than the first preset threshold, it can be considered that the video stream contains not only the target object but also a significant amount of non-critical screen content, such as static background images. In this case, this step can be used to determine the critical screen area.

[0099] For example, in the implementation of determining the key screen area in this step, the object position information of the target object can be obtained. This object position information may include the center position information of the target object's center position, or the top position information of the target object's top position. Furthermore, the required baseline can be determined within the video stream. Specifically, this baseline can be determined in the second display area, and its position can be selected at half the height of the second display area, or at a predetermined distance from the top of the second display area. In some embodiments, the baseline can also be considered as a straight line parallel to the horizontal direction of the video playback page.

[0100] Following the above description, after determining the baseline and obtaining the object's position information, a closed sub-display area can be formed by the baseline, the top border of the second display area, and the left and right borders of the video stream. In this embodiment, the positional relationship between the target object and the sub-display area can be compared based on the object's position information, and the key image area can be specifically determined based on the comparison results. For example, when the top of the target object falls into the sub-display area, the original image area in the second display area of ​​the video stream can be directly used as the second part of the target object's image area, and the second part of the image area can be determined as the key image area. Alternatively, when the top of the target object does not fall into the sub-display area, the key image area can be further determined by combining the object's center position and the displayable size of the second display area.

[0101] The execution of steps b131) to b133) in this embodiment provides an implementation method for determining key screen regions when a target object is identified from a video stream. By using the key screen regions determined through the steps provided in this embodiment, it can be ensured that the screen content corresponding to the target object in the video stream is presented unobstructed in the second display area.

[0102] Following the above description, as one implementation method, this embodiment can specify the selection of the second part of the image region of the target object based on the baseline set in the video stream and combined with the object position information of the target object as steps a2), b2), c2), and d2).

[0103] a2) Along the vertical direction of the video playback page, determine a horizontal line with a second set threshold distance from the top border of the video stream image, and use the horizontal line as the baseline.

[0104] In this embodiment, the baseline can still be determined through the video playback page. For example, a horizontal line can be found along the vertical direction of the video playback page, with a distance of a second set threshold from the top edge of the video stream. In some embodiments, the second set threshold can also be regarded as an empirical distance threshold, or as a height ratio threshold formed by combining the height of the video stream with an empirical constant value.

[0105] The horizontal line formed in this step can be recorded as the baseline of this embodiment. This embodiment can cache information related to this baseline in the video stream, but it will not be directly displayed in the video stream.

[0106] b2) A closed sub-display area is formed by the baseline and the top border and the video stream image, and the top position and center position of the object in the object position information are extracted.

[0107] This step involves determining the sub-display area and extracting positional information related to the top and center positions of the object. In this embodiment, the sub-display area can be defined by defining the closed area formed by the baseline, the top border, and the left and right borders of the video stream.

[0108] It can be understood that the sub-display area is the area closer to the top of the second display area. This embodiment can determine whether the top position of the target object falls into the sub-display area, and provide different steps for determining key screen areas based on the determination result.

[0109] c2) If the top position of the object is within the sub-display area, the screen area is determined as the second part of the image area of ​​the target object.

[0110] In this embodiment, this step can be seen as a step of determining the key screen area when the top position of the object is within the sub-display area. Specifically, when the top position of the target object meets this determination result, it can be considered that most of the target object's area has basically fallen into the second display area. Therefore, there is no need to adjust the content currently displayed in the second display area, and the area of ​​the video stream image already displayed in the second display area can be directly determined as the second part of the image area of ​​the target object.

[0111] d2) If the top position of the object is not within the sub-display area in the video stream, then in the video stream, the second part of the image area of ​​the target object is selected with the center position of the object as the center point of the selected area and the displayable size as the size of the selected area.

[0112] In this embodiment, this step can be viewed as a step to determine the key image area when the top position of the object is not within the sub-display area. Specifically, when the top position of the target object meets this determination result, it can be considered that the image content already displayed in the second display area in the video stream does not contain the key content of the target object. Therefore, it is necessary to adjust the image area to be displayed in the second display area according to the actual position of the target object. The area to be adjusted to be displayed in the second display area can be the second part of the image area of ​​the target object. The second part of the image area is preferably formed by taking the center point of the target object as the center point of the selected area and extending outwards around the center point of the selected area in the video stream to a displayable size.

[0113] The above technical solution of this embodiment provides a step for determining the key screen area corresponding to a video stream frame containing a target object and the target object's screen proportion being less than a first set threshold. The key screen area determined by the steps provided in this embodiment better matches the actual playback scenario of the video stream and better ensures that the content the viewer wants to see is effectively presented in the second display area.

[0114] Figure 3 is a schematic diagram of a screen display device provided in an embodiment of this disclosure. This embodiment is applicable to situations where live streaming screens need adaptive adjustment. The device can be implemented through software and / or hardware and can be configured in a terminal and / or server to implement the screen display method in this embodiment. Specifically, the device may include: a first display module 31 and a second display module 32. The first display module 31 is used to display a video stream on a video playback page in response to a video playback command. The second display module 32 is used to determine a first display area and a second display area on the video playback page in response to an application panel activation command; wherein the first display area is used to display the application panel, and the second display area is used to display key image areas within the video stream, where the key image areas are partial image areas of target objects displayed in the video stream.

[0115] This embodiment provides a screen display device that, in response to an invoked application panel during video stream playback, divides the video playback page into two display areas. One area displays the application content of the application panel, while the other area displays key areas of the image region of the object shown in the video stream. Compared with existing technologies, this better alleviates the problem of the invoked application panel obscuring key content of the video stream on the video playback page. It allows viewers to perform other operations on the application panel without affecting their viewing of key content in the video stream, thus significantly improving the user viewing experience.

[0116] Furthermore, the second display module 32 may specifically include: an object determination unit, used to determine the playback scene associated with the video stream image, and to determine the target object according to the playback scene; and a key image area determination unit, used to determine a portion of the image area from the video stream image as the key image area based on the feature recognition result of the target object in the video stream image.

[0117] Furthermore, the object determination unit can be specifically used to: determine the playback scene associated with the video stream frame based on the video attribute information of the video to which the video stream frame belongs; if the playback scene belongs to a first type of playback scene, then determine the first attention object set relative to the first type of playback scene as the target object; if the playback scene belongs to a second type of playback scene, then determine the second attention object relative to the device in the second type of playback scene as the target object; wherein, the first type of playback scene is different from the second type of playback scene.

[0118] Furthermore, the key image area determination unit may specifically include: an identification subunit, used to perform feature recognition on the target object in the video stream; a first determination subunit, used to select a first portion of the image area from the video stream based on the displayable size of the second display area when the feature recognition result is that no object feature matching the target object is identified, and to use the first portion of the image area as the key image area; and a second determination subunit, used to select a second portion of the image area of ​​the target object based on the screen proportion of the target object in the video stream when the feature recognition result is that an object feature matching the target object is identified, and to use the second portion of the image area as the key image area.

[0119] Furthermore, the first determining subunit can be specifically used to: determine the horizontal central axis of the video stream image along the vertical direction of the video playback page; in the video stream image, select a first part of the image area with the horizontal central axis as the symmetry line of the selected area and the displayable size as the size of the selected area, and use the selected first part of the image area as the key image area.

[0120] Furthermore, the second determining subunit can be specifically used to: determine the proportion of the target object in the video stream frame; if the proportion of the target object is greater than or equal to a first set threshold, then determine the frame area already displayed in the second display area in the video stream frame as the second part image area of ​​the target object, and use the second part image area as the key frame area; if the proportion of the frame is less than the first set threshold, then select the second part image area of ​​the target object based on the baseline set in the video stream frame and in combination with the object position information of the target object, and use the second part image area as the key frame area.

[0121] Further, the second determining subunit's step of selecting a second portion of the image region of the target object based on a baseline set in the video stream and in conjunction with the object position information of the target object includes: determining a horizontal line along the vertical direction of the video playback page, with a distance of a second set threshold from the top border of the video stream, and using the horizontal line as the baseline; forming a closed sub-display area with the video stream through the baseline and the top border, and extracting the top position and center position of the object from the object position information; if the top position of the object is within the sub-display area, determining the image region as the second portion of the image region of the target object; otherwise, in the video stream, selecting the second portion of the image region of the target object with the object center position as the center point of the selected area and the displayable size as the size of the selected area.

[0122] The above-described apparatus can execute the methods provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects for executing the methods.

[0123] It is worth noting that the various units and modules included in the above-mentioned device are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the embodiments of this disclosure.

[0124] Figure 4 is a schematic diagram of the structure of a computer device provided in an embodiment of this disclosure. Referring to Figure 4 below, a schematic diagram of the structure of a computer device (e.g., the terminal device or server in Figure 4) 40 suitable for implementing embodiments of this disclosure is shown. The terminal device in the embodiments of this disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The computer device shown in Figure 4 is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this disclosure.

[0125] As shown in Figure 4, the computer device 40 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 41, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 42 or a program loaded from a storage device 48 into a random access memory (RAM) 43. The RAM 43 also stores various programs and data required for the operation of the computer device 40. The processing unit 41, ROM 42, and RAM 43 are interconnected via a bus 45. An edit / output (I / O) interface 44 is also connected to the bus 45.

[0126] Typically, the following devices can be connected to I / O interface 44: input devices 46 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 47 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 48 including, for example, magnetic tapes, hard disks, etc.; and communication devices 49. Communication device 49 allows computer device 40 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 shows a computer device 40 with various devices, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0127] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 49, or installed from a storage device 48, or installed from a ROM 42. When the computer program is executed by the processing device 41, it performs the functions defined in the methods of embodiments of this disclosure.

[0128] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0129] The computer device provided in this embodiment and the screen display method provided in the above embodiments belong to the same inventive concept. Technical details not described in detail in this embodiment can be found in the above embodiments, and this embodiment has the same beneficial effects as the above embodiments.

[0130] This disclosure provides a computer storage medium storing a computer program that, when executed by a processor, implements the screen display method provided in the above embodiments.

[0131] It should be noted that the computer-readable medium described above in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0132] In this disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0133] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol), and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0134] The aforementioned computer-readable medium may be included in the aforementioned computer device; or it may exist independently and not assembled into the computer device.

[0135] The aforementioned computer-readable medium carries one or more programs, which, when executed by the computer device, cause the computer device to perform the screen display method provided in the above embodiments.

[0136] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0137] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0138] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0139] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0140] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0141] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0142] Furthermore, although the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while some specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0143] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A method for displaying a screen, comprising: In response to a video playback command, display the video stream on the video playback page; In response to the application panel activation command, a first display area and a second display area are determined on the video playback page; wherein, the first display area is used to display the application panel, and the second display area is used to display key image areas within the video stream, the key image areas being partial image areas within the video stream.

2. The method according to claim 1, wherein, The steps for determining key areas within the video stream include: Determine the playback scene associated with the video stream frame, and determine the target object based on the playback scene; Based on the feature recognition results of the target object in the video stream, a portion of the image region is determined from the video stream as the key image region.

3. The method according to claim 2, wherein, The steps of determining the playback scene associated with the video stream frame and determining the target object based on the playback scene include: Based on the video attribute information of the video to which the video stream belongs, determine the playback scene associated with the video stream; If the playback scenario belongs to the first type of playback scenario, then the first object of interest set relative to the first type of playback scenario is determined as the target object; If the playback scenario belongs to the second type of playback scenario, then the second object of interest relative to the device in the second type of playback scenario is determined as the target object; The first type of playback scenario is different from the second type of playback scenario.

4. The method according to claim 2, wherein, The step of determining a portion of the image region as the key image region from the video stream frame based on the feature recognition result of the target object in the video stream frame includes: Feature recognition is performed on the target object in the video stream; If the feature recognition result is that no object feature matching the target object is identified, then according to the displayable size of the second display area, a first part of the image area is selected from the video stream and the first part of the image area is used as the key image area; If the feature recognition result indicates that an object feature matching the target object has been identified, then based on the proportion of the target object in the video stream, a second part of the image region of the target object is selected, and the second part of the image region is used as the key image region.

5. The method according to claim 4, wherein, The step of selecting a first portion of the image area from the video stream based on the displayable size of the second display area, and using the first portion of the image area as the key image area, includes: Determine the horizontal centerline of the video stream image along the vertical direction of the video playback page; In the video stream, a first part of the image region is selected using the horizontal central axis as the symmetry line of the selected area and the displayable size as the size of the selected area. The selected first part of the image region is then used as the key image region.

6. The method according to claim 4, wherein, The step of selecting a second portion of the image region of the target object based on its proportion of the video stream frame, and using this second portion of the image region as the key frame region, includes: Determine the percentage of the target object in the video stream frame; If the proportion of the object in the frame is greater than or equal to a first preset threshold, then the video stream frame will be... The image area already displayed in the second display area is determined as the second part of the image area of ​​the target object, and the second part of the image area is used as the key image area; If the percentage of the image area is less than the first set threshold, then based on the baseline set in the video stream and combined with the object position information of the target object, the second part of the image area of ​​the target object is selected and the second part of the image area is used as the key image area.

7. The method according to claim 6, wherein, The step of selecting a second portion of the image region of the target object based on a baseline set in the video stream and in conjunction with the object position information of the target object includes: Along the vertical direction of the video playback page, a horizontal line is determined with a second preset threshold distance from the top edge of the video stream image, and the horizontal line is used as the baseline. The baseline and the top border together form a closed sub-display area with the video stream, and the top position and center position of the object are extracted from the object position information. If the top of the object is within the sub-display area, the screen area is defined as the second portion of the image area of ​​the target object; otherwise... In the video stream, the second part of the image region of the target object is selected, with the center position of the object as the center point of the selected region and the displayable size as the size of the selected region.

8. A display device, comprising: The first display module is used to display the video stream on the video playback page in response to video playback commands; The second display module is used to respond to the application panel's activation command and determine a first display area and a second display area on the video playback page; wherein, the first display area is used to display the application panel, and the second display area is used to display key areas within the video stream. The key image area within the video stream is a portion of the image area of ​​the target object displayed in the video stream.

9. A computer device, the computer device comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the screen display method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the screen display method as described in any one of claims 1-7.

Images