A video playing method, device, equipment, storage medium and program product

Abstract

The present disclosure provides a video playing method and device, equipment, storage medium and program product, relates to the technical field of data processing, and particularly relates to the technical field of video processing. The specific implementation scheme is as follows: obtaining a region feature of a target region in a video frame in a video; taking the width-height ratio of a player in a screen as a region size reference, and taking the maximization of the number of contained target regions as a region selection criterion, determining a to-be-displayed region in the video frame according to the obtained region feature; and playing the to-be-displayed region in the video frame in the player. The video playing scheme provided in the embodiment of the present disclosure can solve the problem that the width-height ratio of the video is inconsistent with the width-height ratio of the screen of the terminal.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, particularly to the field of video processing technology, and especially to a video playback method, apparatus, device, storage medium, and program product. Background Technology

[0002] Common video aspect ratios are often greater than 1, such as 4:3 and 16:9, while the aspect ratios of most mobile phones, tablets and other terminals are often less than 1. It can be seen that the aspect ratios of videos and terminals are inconsistent. Summary of the Invention

[0003] This disclosure provides a video playback method, apparatus, device, storage medium, and program product.

[0004] According to one aspect of this disclosure, a video playback method is provided, comprising:

[0005] Obtain the regional features of a target region within a video frame, wherein the target region is: a region that is representative of the content in the video frame;

[0006] Using the aspect ratio of the player on the screen as the basis for the area size, and maximizing the number of target areas contained as the criterion for area selection, the area to be displayed in the video frame is determined based on the obtained area characteristics.

[0007] The area to be displayed within a video frame is played in the player.

[0008] According to another aspect of this disclosure, a video playback device is provided, comprising:

[0009] The feature acquisition module is used to acquire the regional features of a target region within a video frame in the video, wherein the target region is: a region that is representative of the content in the video frame;

[0010] The region determination module is used to determine the region to be displayed in the video frame based on the aspect ratio of the player on the screen as the region size reference and the maximization of the number of target regions contained as the region selection criterion.

[0011] A video playback module is used to play the area to be displayed within a video frame in the player.

[0012] According to another aspect of this disclosure, an electronic device is provided, comprising:

[0013] At least one processor; and

[0014] A memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the video playback method described above.

[0016] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the video playback method described above.

[0017] According to another aspect of this disclosure, a computer program product is provided, including a computer program that is executed by a processor using the video playback method described above.

[0018] As can be seen from the above, when playing video using the solution provided in this embodiment, the area to be displayed in the video frame is determined based on the aspect ratio of the player on the screen and the regional characteristics of the target area within the video frame, thereby playing the area to be displayed within the video frame in the player. Since the area to be displayed is determined based on the aspect ratio of the player, the aspect ratio of the area to be displayed is consistent with the aspect ratio of the player. This allows the video to be played across the entire playback area of ​​the player, thereby improving the user's viewing experience and increasing user engagement.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0020] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0021] Figure 1a A flowchart illustrating the first video playback method provided in this embodiment of the disclosure;

[0022] Figure 1b A schematic diagram of the interface of a first type of player provided in an embodiment of this disclosure;

[0023] Figure 1c A schematic diagram of the interface of a second type of player provided in an embodiment of this disclosure;

[0024] Figure 1d A heatmap of regional features of a visually salient region provided in an embodiment of this disclosure;

[0025] Figure 2 A flowchart illustrating a second video playback method provided in this embodiment of the present disclosure;

[0026] Figure 3 A flowchart illustrating the third video playback method provided in this embodiment of the disclosure;

[0027] Figure 4 A flowchart illustrating the fourth video playback method provided in this embodiment of the disclosure;

[0028] Figure 5 A flowchart illustrating the fifth video playback method provided in this embodiment of the disclosure;

[0029] Figure 6 A flowchart illustrating the sixth video playback method provided in this embodiment of the disclosure;

[0030] Figure 7 A flowchart illustrating the seventh video playback method provided in this embodiment of the disclosure;

[0031] Figure 8 This is a schematic diagram of the structure of a first video playback device provided in an embodiment of the present disclosure;

[0032] Figure 9 This is a block diagram of an electronic device used to implement the video playback method of the embodiments of this disclosure. Detailed Implementation

[0033] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0034] First, the implementing entity and application scenarios of the solutions provided in the embodiments of this disclosure will be explained.

[0035] The implementation entity of the solution provided in this embodiment is a client, which integrates a player, and the server stores the video. When a user watches a video through the client, the client can retrieve the video from the server and call the integrated player to play the video for the user.

[0036] The video playback method, apparatus, electronic device, and storage medium provided in this disclosure will be described in detail below through specific embodiments.

[0037] See Figure 1a , Figure 1a This is a flowchart illustrating a first video playback method provided in an embodiment of the present disclosure. In this embodiment, the method includes the following steps S101-S103.

[0038] Step S101: Obtain the regional features of the target region within the video frame.

[0039] The aforementioned video frame can be a single keyframe in the video, multiple consecutive keyframes, or one or more randomly selected non-keyframes from the video.

[0040] The target region is the region that is representative of the content in the video frame.

[0041] The regional features of the target region described above can be used to characterize the regional location of the target region.

[0042] Specifically, the regional features of the target region can be obtained through either of the following two methods.

[0043] In the first implementation, the client can obtain the video to be played from the server, select one or more video frames from the obtained video, extract features from each selected video frame, and thus obtain the regional features of the target area in the video frame.

[0044] In the second implementation, the server can pre-extract the regional features of the target area within the video frame. This way, when the client obtains the video to be played from the server, it can obtain the regional features of the target area within the video frame.

[0045] Regional features of the target region can be extracted using existing feature extraction methods, which will not be detailed here.

[0046] Step S102: Using the aspect ratio of the player on the screen as the area size reference and maximizing the number of target areas contained as the area selection criterion, determine the area to be displayed in the video frame based on the obtained area characteristics.

[0047] The aspect ratio of the aforementioned player can be preset or adjusted by the user according to their own preferences.

[0048] For example, Figure 1b , 1c These are screenshots of the player interface before and after user adjustments. Users can drag the border of the black playback area to adjust the player's aspect ratio while viewing the interface.

[0049] Specifically, when determining the area to be displayed, the size and position of the area to be displayed in the video frame can be determined. This can be done by first determining the size of the area to be displayed, and then determining its position based on the size and characteristics of the target area. Alternatively, the size and position of the area to be displayed can be determined comprehensively based on the aforementioned area size benchmark, area selection criteria, and the characteristics of the target area.

[0050] For details on how to determine the area to be displayed, please refer to the following sections. Figure 2 , Figure 5 , Figure 6 , Figure 7 The illustrated embodiment will not be described in detail here.

[0051] Step S103: Play the area to be displayed within the video frame in the video player.

[0052] Specifically, when the aforementioned video frame is a single video frame in the aforementioned video, after determining the area to be displayed in that video frame, an area in other video frames at the same position as the area to be displayed in that video frame can be determined as the area to be displayed in other video frames, so that the area to be displayed in each video frame is displayed when the video is played.

[0053] When the above video frames are multi-frame video frames, the final area to be displayed can be determined according to the position of the area to be displayed in the multi-frame video frames, so that the final area to be displayed in each video frame can be displayed when playing the video.

[0054] In one embodiment of this disclosure, the final area to be displayed can be determined by either of the following two implementation methods.

[0055] In the first implementation, the average position of the area to be displayed in multiple video frames can be calculated as the final position of the area to be displayed.

[0056] In the second implementation method, multiple areas to be displayed that are close in location can be identified, and one area can be selected from these multiple areas to be displayed or the average value can be calculated to determine the final area to be displayed.

[0057] As can be seen from the above, when playing video using the solution provided in this embodiment, the area to be displayed in the video frame is determined based on the aspect ratio of the player on the screen and the regional characteristics of the target area within the video frame, thereby playing the area to be displayed within the video frame in the player. Since the area to be displayed is determined based on the aspect ratio of the player, the aspect ratio of the area to be displayed is consistent with the aspect ratio of the player. This allows the video to be played across the entire playback area of ​​the player, thereby improving the user's viewing experience and increasing user engagement.

[0058] Furthermore, in this solution, for players with different aspect ratios, the display area can be determined based on the different aspect ratios to suit different players. Therefore, the video playback solution provided by this disclosure can adapt to players with various aspect ratios. Compared with the prior art of using a completely black or white area to fill the screen, this solution can expand the playback area of ​​the video in the terminal, thereby improving the user experience.

[0059] In one embodiment of this disclosure, the regional features of the target area within the video frame in the video can be stored in the server. When the client obtains the video, it can obtain the regional features corresponding to the video frame in the video and, in combination with the aspect ratio of the client's own player, determine the area to be displayed in the video frame, thereby playing the area to be displayed within the video frame in the video. In this way, the server does not need to store multiple videos with different aspect ratios, thereby saving storage space in the server.

[0060] In one embodiment of this disclosure, the video frame includes: the current keyframe in the video and a plurality of keyframes before and / or after the current keyframe.

[0061] The video received by the client from the server is usually encoded. After receiving the encoded video, the client needs to decode it. During the encoding process on the server and the decoding process on the client, each video frame typically suffers from encoding loss. However, the encoding loss of keyframes is usually less than that of non-keyframes. Therefore, by utilizing the regional features of target areas within multiple keyframes in the video, the accuracy of the determined display area can be improved as much as possible. This, when playing the video in the player, can maximize the user's viewing experience and increase user engagement.

[0062] In one embodiment of this disclosure, the target region includes at least one of the following regions:

[0063] Visually salient area, object area, text area.

[0064] The visually salient regions mentioned above are those with a high degree of visual salience.

[0065] When obtaining the regional features of visually salient regions within a video frame, the video frame can be divided into multiple sub-regions, and a visual saliency parameter can be calculated to characterize the visual saliency of each sub-region. For each sub-region, if the visual saliency parameter of the sub-region is greater than a preset threshold, the sub-region is determined to be a visually salient region, and the visual saliency parameter of the sub-region can be used as the regional feature of the sub-region.

[0066] like Figure 1d As shown, Figure 1d A heatmap representing the regional characteristics of a visually salient region. Figure 1d Each pixel in the above video frame corresponds to a sub-region. When determining the visual saliency parameter of the sub-region, if the visual saliency parameter of the sub-region is greater than a preset threshold, the pixel value of the pixel corresponding to the sub-region in the heat map is determined to be 1; otherwise, it is determined to be 0.

[0067] Figure 1dThere are 32*16 pixels in the video, and the pixel value of each pixel is 1 or 0. Thus, only a 512-bit bitset is needed to represent the visual salient features of the entire video frame. Compared with the above video, the visual salient features occupy a very small space.

[0068] The aforementioned object region can include regions containing various objects, such as face regions, body regions, or regions containing other objects.

[0069] The regional features of the aforementioned object region can be represented by two sets of two-dimensional coordinates in the video frame. These two sets of two-dimensional coordinates can be the coordinates of the two diagonally opposite endpoints of the object region, or the center coordinates and the width and height of the object region.

[0070] In addition, the size of each coordinate value in each of the above two-dimensional coordinates can be set to 2 bytes. In this way, a set of two-dimensional coordinates can represent any point in a video frame with a size of 65535*65535, that is, it can represent any point in a video frame in 8K quality video. Therefore, the space where the regional features of the object area are located is also extremely small.

[0071] The aforementioned text area may include the area where the subtitles are located and the area where the title is located, as well as the area where other text appears in the video frame.

[0072] Similar to the object region mentioned above, the regional features of the text region mentioned above can also be represented by two sets of two-dimensional coordinates in the video frame.

[0073] In this solution, the three regions mentioned above are all highly representative of the content in the video frame. Therefore, by obtaining at least one of the three regions and using the regional characteristics of at least one of these regions to determine the video frame, the displayable region in the video frame can be accurately determined. This allows the displayable region within the video frame to be played in the player, providing users with rich video content, thereby improving the user's viewing experience and increasing user engagement.

[0074] As can be seen from the above embodiments, there can be multiple types of target regions in the video frame. Therefore, the regional features of multiple types of target regions can be obtained. Thus, when determining the displayable region in the video frame based on the obtained regional features, the displayable region can be determined by using the regional features of any one of the obtained target regions, or by using the regional features of multiple types of target regions.

[0075] The following explains the specific implementation method of determining the area to be displayed using the regional features of a target region of a certain category.

[0076] In one embodiment of this disclosure, see [link to embodiment]. Figure 2The present invention provides a flowchart of a second video playback method. In this embodiment, the above method includes the following steps S201-S205.

[0077] Step S201: Obtain the regional features of the target region within the video frame.

[0078] This step is the same as step S101 above, and will not be repeated here.

[0079] Step S202: Determine the maximum area size that satisfies the aspect ratio of the player on the screen based on the size of the video frame.

[0080] Specifically, the maximum region size can be determined using either of the following two methods.

[0081] In the first implementation, given the aspect ratio of the player and the size of the video frame, one dimension can be chosen as the baseline dimension and the other as the calibration dimension. The size of the baseline dimension in the video frame is used as the baseline dimension size of the aforementioned maximum area size. Based on the size of the baseline dimension in the video frame and the aspect ratio of the player, the size of the calibration dimension of the aforementioned maximum area size is calculated, and the calculated first size size is compared with the second size of the calibration dimension in the video frame. If the first size size is greater than the second size size, it means that the calculated maximum area size is greater than the size of the video frame. In this case, the baseline dimension and the calibration dimension are swapped, and the maximum area size is recalculated. If the first size size is less than or equal to the second size size, then the size of the two dimensions calculated is the maximum area size.

[0082] For example, if the aspect ratio of the aforementioned player is 3:4 and the video frame size is 36*24, choosing the width dimension as the base dimension and the height dimension as the calibration dimension, the width in the maximum area size can be determined to be 36. Based on the width of the video frame and the aspect ratio of the player, the height of the maximum area size can be calculated to be 48, i.e., the calculated maximum area size is 36*48. Since the height of the maximum area size is greater than the height of the video frame, the base dimension and calibration dimension need to be swapped, and the maximum area size needs to be recalculated. That is, the height dimension is used as the base dimension and the width dimension as the calibration dimension. In this case, the calculated maximum area size is 18*24.

[0083] In the second implementation, a display frame with an aspect ratio equal to that of the player can be randomly generated in the aforementioned video frame, and the display frame can be enlarged until the two widths of the display frame coincide with the width of the video frame, or the two heights of the display frame coincide with the height of the video frame. At this point, the size of the display frame is the aforementioned maximum area size.

[0084] Step S203: Based on the regional characteristics of the target region of this category, determine the candidate region in the video frame that contains the target region of this category and has the largest region size.

[0085] Specifically, based on the regional characteristics of the target region of this category, the regional location of each target region of this category can be determined, and thus multiple candidate regions can be determined based on the regional location of each target region and the aforementioned maximum region size.

[0086] Step S204: Determine the region with the highest number of complete target regions of this category from the candidate regions, and obtain the region to be displayed in the video frame based on the determined region.

[0087] Specifically, after identifying multiple candidate regions, the number of complete target regions to be categorized included in each candidate region can be detected, thereby identifying the candidate region corresponding to the maximum number of regions in each candidate region.

[0088] After identifying the candidate region corresponding to the maximum number, this region can be directly used as the area to be displayed in the video frame. Alternatively, it can be further processed through subsequent steps. Figure 3 or Figure 4 In the illustrated embodiment, each step adjusts the position of the determined area to obtain the adjusted area to be displayed.

[0089] Step S205: Play the area to be displayed within the video frame in the video player.

[0090] This step is the same as step S103 above, and will not be repeated here.

[0091] As can be seen from the above, when playing video using the solution provided in this embodiment, the maximum region size is determined based on the size of the video frame, thus maximizing the amount of content displayed in the video frame to the user. After determining the maximum region size, candidate regions containing the target region and having a size equal to the maximum region size are identified within the video frame. From these candidate regions, the region containing the largest number of complete target regions is selected. Based on the determined region, the region to be displayed is obtained. This ensures that the region to be displayed contains as many target regions as possible, thereby displaying a more content-rich area of ​​the video frame to the user.

[0092] After obtaining the area to be displayed in the video frame, the position of the area to be displayed can be adjusted according to the regional characteristics of various target areas in the video frame to improve the viewing experience of the area to be displayed.

[0093] In one embodiment of this disclosure, see [link to embodiment]. Figure 3 The present invention provides a flowchart of a third video playback method. In this embodiment, the above method includes the following steps S301-S307.

[0094] Step S301: Obtain the regional features of the target region within the video frame.

[0095] Step S302: Determine the maximum area size that satisfies the aspect ratio of the player on the screen based on the size of the video frame.

[0096] Step S303: Based on the regional characteristics of the target region of this category, determine the candidate region in the video frame that contains the target region of this category and has the largest region size.

[0097] Step S304: Determine the region with the highest number of complete target regions of this category from the candidate regions, and obtain the region to be displayed in the video frame based on the determined region.

[0098] The steps S301-S304 described above are the same as those S201-S204 described above, and will not be repeated here.

[0099] Step S305: Based on the region location represented by the first feature, detect whether the target region of this category in the region to be displayed is complete. If it is incomplete, proceed to step S306.

[0100] The first feature is the characteristic of the target region of this category.

[0101] Specifically, based on the first feature, the location of the target area can be determined, and then the completeness of the target area in the area to be displayed can be detected based on the location of the target area.

[0102] In one embodiment of this disclosure, the completeness of the target area in the area to be displayed can be detected through the following two methods.

[0103] In the first implementation, for each target area, it can be detected whether all pixels in the target area belong to the area to be displayed or not. If there is a target area where some pixels belong to the area to be displayed and some pixels do not belong to the area to be displayed, it means that the target area is incomplete.

[0104] In the second implementation, it is also possible to determine whether the boundary pixels of the area to be displayed belong to the target area. If they do, it means that the area to be displayed intersects with the target area, and the intersecting target area is incomplete.

[0105] Step S306: Adjust the position and / or size of the area to be displayed according to the location of the incomplete area, so as to obtain a display area in which all target areas of the same category are complete.

[0106] Adjusting the area to be displayed based on the location of the incomplete area can be achieved using existing technologies, which will not be detailed here.

[0107] Step S307: Play the area to be displayed within the video frame in the video player.

[0108] As can be seen from the above, when playing a video using the solution provided in this embodiment, the position and / or size of the area to be displayed are adjusted so that all target areas of that category included in the area to be displayed are complete target areas. This avoids showing incomplete target areas to the user when playing the video, thereby improving the user's viewing experience.

[0109] When adjusting the location of the area to be displayed, adjustments can also be made based on the regional characteristics of target areas of other categories.

[0110] In one embodiment of this disclosure, see [link to embodiment]. Figure 4 The present invention provides a flowchart of a fourth video playback method. In this embodiment, the above method includes the following steps S401-S407.

[0111] Step S401: Obtain the regional features of the target region within the video frame.

[0112] Step S402: Determine the maximum area size that satisfies the aspect ratio of the player on the screen based on the size of the video frame.

[0113] Step S403: Based on the regional characteristics of the target region of this category, determine the candidate region in the video frame that contains the target region of this category and has the largest region size.

[0114] Step S404: Determine the region with the highest number of complete target regions of this category from the candidate regions, and obtain the region to be displayed in the video frame based on the determined region.

[0115] The steps S401-S404 described above are the same as those S201-S204 described above, and will not be repeated here.

[0116] Step S405: Based on the region location represented by the second feature, detect whether other categories of target regions in the region to be displayed are complete. If not, proceed to step S406, where the second feature is: the feature of other categories of target regions.

[0117] Step S406: Adjust the position and / or size of the area to be displayed according to the location of the incomplete area, so as to obtain a display area in which all target areas of each category are complete.

[0118] Steps S405 and S406 are the same as steps S305 and S306, respectively, and will not be repeated here.

[0119] Step S407: Play the area to be displayed within the video frame in the video player.

[0120] As can be seen from the above, when playing a video using the solution provided in this embodiment, the position and / or size of the area to be displayed are adjusted so that all target areas of each category included in the area to be displayed are complete target areas. This avoids showing incomplete target areas to the user when playing the video, thereby improving the user's viewing experience.

[0121] If there are multiple candidate regions corresponding to the maximum number mentioned above, multiple regions to be displayed can be obtained. In this case, the final region to be displayed can be randomly selected from these multiple regions, or it can be done through the following... Figure 5 In the illustrated embodiment, step S505 selects the final area to be displayed.

[0122] In one embodiment of this disclosure, see [link to embodiment]. Figure 5 The present invention provides a flowchart of the fifth video playback method. In this embodiment, the above method includes the following steps S501-S506.

[0123] Step S501: Obtain the regional features of the target region within the video frame.

[0124] Step S502: Determine the maximum area size that satisfies the aspect ratio of the player on the screen based on the size of the video frame.

[0125] Step S503: Based on the regional characteristics of the target region of this category, determine the candidate region in the video frame that contains the target region of this category and has the largest region size.

[0126] Step S504: Determine the region with the highest number of complete target regions of this category from the candidate regions, and obtain the region to be displayed in the video frame based on the determined region.

[0127] The steps S501-S504 described above are the same as those S201-S204 described above, and will not be repeated here.

[0128] Step S505: When the number of areas to be displayed is greater than 1, select the final area to be displayed from the areas to be displayed based on the visual salience of each area.

[0129] Specifically, visual saliency parameters that characterize the visual saliency of each region to be displayed can be obtained, and based on the visual saliency parameters of each region to be displayed, the region with the largest visual saliency parameter value is selected as the final region to be displayed.

[0130] In one embodiment of this disclosure, the visual saliency parameters of the area to be displayed can be obtained through either of the following two implementation methods.

[0131] In the first implementation method, the visual saliency parameters of the entire area to be displayed can be calculated using existing technologies.

[0132] In the second implementation, the regional features of visually salient regions within a video frame can be obtained. Based on these regional features, the visually salient regions within the video frame can be determined. This allows us to determine the number or area of ​​visually salient regions within the area to be displayed. Thus, the number or area of ​​visually salient regions within the area to be displayed can be used as the visual salient parameter of that area.

[0133] Step S506: Play the area to be displayed within the video frame in the video player.

[0134] This step is the same as step S103 above, and will not be repeated here.

[0135] As can be seen from the above, when playing a video using the solution provided in this embodiment, if the number of areas to be displayed is greater than 1, the final area to be displayed can be accurately selected from each area to be displayed based on the visual salience of each area to be displayed, thereby playing the area to be displayed within the video frame in the player, which can improve the user's viewing experience.

[0136] The number of target regions within a video frame may be large. For example, the target regions mentioned above could be face regions, and the video frame may contain multiple face regions. In this case, before determining the region to be displayed, some regions can be pre-filtered out, and the region to be displayed can be determined only based on the regional characteristics of the remaining target regions.

[0137] In one embodiment of this disclosure, see [link to embodiment]. Figure 6 The present invention provides a flowchart of a sixth video playback method. In this embodiment, the above method includes the following steps S601-S606.

[0138] Step S601: Obtain the regional features of the target region within the video frame.

[0139] Step S602: Determine the maximum area size that satisfies the aspect ratio of the player on the screen based on the size of the video frame.

[0140] The steps S601 and S602 described above are the same as steps S201 and S202 described above, and will not be repeated here.

[0141] Step S603: Determine the significant regions in the target regions of this category based on the area represented by the regional characteristics of the target regions of this category.

[0142] Specifically, based on the regional characteristics of the target region of this category, the target region of this category can be determined in the video frame, and the regional area of ​​each target region of this category can be calculated. Based on the regional area of ​​each target region of this category, the salient region among the target regions of this category can be determined.

[0143] In one embodiment of this disclosure, when determining the above-mentioned significant regions, a target region with an area greater than a preset area threshold can be determined as a significant region, and a preset number of target regions with the largest area can also be determined as significant regions.

[0144] In addition, after determining the target region based on the regional characteristics of the target region of this category, the area ratio of each target region of this category in the video frame can be calculated, and the aforementioned significant regions can be determined based on the area ratio of each target region of this category.

[0145] Step S604: Based on the regional characteristics of the salient regions of the category, determine the candidate regions in the video frame that contain the salient regions of the category and have the largest region size.

[0146] Step S605: Determine the region with the highest number of complete salient regions of this category from the candidate regions, and obtain the region to be displayed in the video frame based on the selected region.

[0147] Step S606: Play the area to be displayed within the video frame in the video player.

[0148] Steps S604-S606 are similar to steps S203-S205, and will not be repeated here.

[0149] As can be seen from the above, when playing video using the solution provided in this embodiment, salient regions are selected from multiple target regions based on the area of ​​each target region, and the region to be displayed is determined based on the regional characteristics of the salient regions. The aforementioned salient regions can be understood as regions with strong representational value of the content in the video frame, while other regions in the target regions can be understood as regions with weaker representational value of the content in the video frame. By using only the regional characteristics of the salient regions to determine the region to be displayed, the accuracy of the region to be displayed is ensured while also improving the efficiency of determining the region to be displayed, thereby improving the efficiency of video playback.

[0150] The following explains the specific implementation method of determining the area to be displayed using the regional features of multiple categories of target areas.

[0151] In one embodiment of this disclosure, see [link to embodiment]. Figure 7 The document provides a flowchart of the seventh video playback method. In this embodiment, the method includes the following steps S701-S705.

[0152] Step S701: Obtain the regional features of the target region within the video frame.

[0153] This step is the same as step S101 above, and will not be repeated here.

[0154] Step S702: According to the priority of different regions in descending order, based on the obtained region features, detect whether there is a target region of the current priority target category in the video frame. If it exists, proceed to step S703; if it does not exist, proceed to step S704.

[0155] In one embodiment of this disclosure, the priority of the different categories is determined based on the content category of the video.

[0156] For example, the target area mentioned above can include the area where a face is located, the area where a human body is located, and the area where a vehicle is located. If the content category of the video is dance, then the area where a human body is located can be determined to have the highest priority. If the content category of the video is automobiles, then the area where a vehicle is located can be determined to have the highest priority.

[0157] In this solution, the priority of different categories of regions is determined based on the content category of the video. This allows for prioritizing the regional characteristics of regions closely related to the content category of the video, thereby accurately determining the regions to be displayed.

[0158] In addition, the priorities of the above-mentioned different categories can also be preset by humans.

[0159] Specifically, the initial value of the current priority is the highest priority. After obtaining the regional features of the target region within the video frame, according to the priority of the region category from high to low, we can first detect whether there is a target region of the highest priority target category in the video frame. If it exists, we execute step S703; if it does not exist, we execute step S704.

[0160] For example, if the priorities of the above-mentioned region categories are from high to low as follows: face region, human body region, and vehicle region, and the obtained region features are the region features of the human body region and the vehicle region, then it can be detected that there is no highest priority face region in the video frame. In this case, step S704 is executed.

[0161] If the obtained regional features are the regional features of the face region and the human body region, then the highest priority face region can be detected in the video frame. In this case, step S703 is executed.

[0162] Step S703: Using the aspect ratio of the player on the screen as the area size reference and maximizing the number of target areas of the included target categories as the area selection criterion, determine the area to be displayed in the video frame according to the area characteristics of the target areas of the target categories.

[0163] This step is similar to step S102 above, and will not be repeated here.

[0164] Step S704: Adjust the current priority to the next priority and return to step S702.

[0165] Step S705: Play the area to be displayed within the video frame in the video player.

[0166] This step is the same as step S103 above, and will not be repeated here.

[0167] As can be seen from the above, when playing a video using the solution provided in this embodiment, the priority of the different categories of the above-mentioned regions can be understood as the order in which the user expects to focus on the regions. By utilizing the priority of the different categories of regions, the target region of the highest priority target category can be determined within the target region corresponding to the obtained region features. Based on the region features of the target region of the determined higher priority target category, the region to be displayed in the video frame can be determined. In this way, when playing a video in the player, the region to be displayed within the video frame can show the user the region in the video frame that the user most expects to focus on, thereby improving the user's viewing experience.

[0168] Corresponding to the video playback method described above, this disclosure also provides a video playback device.

[0169] In one embodiment of this disclosure, see [link to embodiment]. Figure 8 A schematic diagram of the structure of a first type of video playback device is provided. In this embodiment, the device includes:

[0170] The feature acquisition module 801 is used to acquire the regional features of a target region within a video frame in the video, wherein the target region is: a region that is representative of the content in the video frame;

[0171] The region determination module 802 is used to determine the display region in the video frame based on the obtained region characteristics, with the aspect ratio of the player on the screen as the region size reference and the maximization of the number of target regions contained as the region selection criterion.

[0172] The video playback module 803 is used to play the area to be displayed within a video frame in the video in the player.

[0173] As can be seen from the above, when playing video using the solution provided in this embodiment, the area to be displayed in the video frame is determined based on the aspect ratio of the player on the screen and the regional characteristics of the target area within the video frame, thereby playing the area to be displayed within the video frame in the player. Since the area to be displayed is determined based on the aspect ratio of the player, the aspect ratio of the area to be displayed is consistent with the aspect ratio of the player. This allows the video to be played across the entire playback area of ​​the player, thereby improving the user's viewing experience and increasing user engagement.

[0174] Furthermore, in this solution, for players with different aspect ratios, the display area can be determined based on the different aspect ratios to suit different players. Therefore, the video playback solution provided by this disclosure can adapt to players with various aspect ratios. Compared with the prior art of using a completely black or white area to fill the screen, this solution can expand the playback area of ​​the video in the terminal, thereby improving the user experience.

[0175] In one embodiment of this disclosure, the regional features of the target area within the video frame in the video can be stored in the server. When the client obtains the video, it can obtain the regional features corresponding to the video frame in the video and, in combination with the aspect ratio of the client's own player, determine the area to be displayed in the video frame, thereby playing the area to be displayed within the video frame in the video. In this way, the server does not need to store multiple videos with different aspect ratios, thereby saving storage space in the server.

[0176] In one embodiment of this disclosure, the video frame contains multiple categories of target regions;

[0177] The feature acquisition module 801 is specifically used for:

[0178] For each category, the area to be displayed in the video frame is determined as follows:

[0179] Based on the size of the video frames, determine the maximum area size that satisfies the aspect ratio of the player on the screen;

[0180] Based on the regional characteristics of the target region of this category, candidate regions in the video frame that contain the target region of this category and have a size equal to the maximum region size are determined;

[0181] From the candidate regions, identify the region that contains the most complete target regions of that category, and obtain the region to be displayed in the video frame based on the identified region.

[0182] As can be seen from the above, when playing video using the solution provided in this embodiment, the maximum region size is determined based on the size of the video frame, thus maximizing the amount of content displayed in the video frame to the user. After determining the maximum region size, candidate regions containing the target region and having a size equal to the maximum region size are identified within the video frame. From these candidate regions, the region containing the largest number of complete target regions is selected. Based on the determined region, the region to be displayed is obtained. This ensures that the region to be displayed contains as many target regions as possible, thereby displaying a more content-rich area of ​​the video frame to the user.

[0183] In one embodiment of this disclosure, the feature acquisition module 801 further includes:

[0184] Based on the location of the region represented by the first feature, it is detected whether the target region of this category in the region to be displayed is complete, wherein the first feature is: the feature of the target region of this category;

[0185] If incomplete, adjust the position and / or size of the area to be displayed based on the location of the incomplete area to obtain a display area where all target areas of the included category are complete.

[0186] As can be seen from the above, when playing a video using the solution provided in this embodiment, the position and / or size of the area to be displayed are adjusted so that all target areas of that category included in the area to be displayed are complete target areas. This avoids showing incomplete target areas to the user when playing the video, thereby improving the user's viewing experience.

[0187] In one embodiment of this disclosure, the feature acquisition module 801 further includes:

[0188] Based on the region location represented by the second feature, it is determined whether other categories of target regions in the region to be displayed are complete, wherein the second feature is: the feature of other categories of target regions;

[0189] If incomplete, adjust the position and / or size of the area to be displayed based on the location of the incomplete area to obtain a display area where all target areas of each category are complete.

[0190] As can be seen from the above, when playing a video using the solution provided in this embodiment, the position and / or size of the area to be displayed are adjusted so that all target areas of each category included in the area to be displayed are complete target areas. This avoids showing incomplete target areas to the user when playing the video, thereby improving the user's viewing experience.

[0191] In one embodiment of this disclosure, the feature acquisition module 801 further includes:

[0192] If the number of areas to be displayed is greater than 1, the final area to be displayed is selected from the areas to be displayed based on the visual salience of each area, or the final area to be displayed is randomly selected from the areas to be displayed.

[0193] As can be seen from the above, when playing a video using the solution provided in this embodiment, if the number of areas to be displayed is greater than 1, the final area to be displayed can be accurately selected from each area to be displayed based on the visual salience of each area to be displayed, thereby playing the area to be displayed within the video frame in the player, which can improve the user's viewing experience.

[0194] In one embodiment of this disclosure, the feature acquisition module 801 is specifically used for:

[0195] Based on the area represented by the regional characteristics of the target region of this category, determine the salient region in the target region of this category;

[0196] Based on the regional characteristics of the salient regions of this category, candidate regions in the video frame that contain the salient regions of this category and have a size equal to the maximum region size are determined;

[0197] From the candidate regions, identify the region that contains the most complete salient regions of that category, and obtain the region to be displayed in the video frame based on the selected region.

[0198] As can be seen from the above, when playing video using the solution provided in this embodiment, salient regions are selected from multiple target regions based on the area of ​​each target region, and the region to be displayed is determined based on the regional characteristics of the salient regions. The aforementioned salient regions can be understood as regions with strong representational value of the content in the video frame, while other regions in the target regions can be understood as regions with weaker representational value of the content in the video frame. By using only the regional characteristics of the salient regions to determine the region to be displayed, the accuracy of the region to be displayed is ensured while also improving the efficiency of determining the region to be displayed, thereby improving the efficiency of video playback.

[0199] In one embodiment of this disclosure, the region determination module 802 is specifically used for:

[0200] Based on the obtained regional features, the system detects whether there is a target region of the current priority category in the video frame, according to the priority of different regional categories from high to low.

[0201] If present, the area to be displayed in the video frame is determined based on the aspect ratio of the player on the screen as the area size reference, and the area selection criterion is to maximize the number of target areas of the target category contained therein.

[0202] If it does not exist, the current priority is adjusted to the next priority, and the step of detecting whether there is a target region of the target category of the current priority in the video frame based on the obtained regional features is executed.

[0203] As can be seen from the above, when playing a video using the solution provided in this embodiment, the priority of the different categories of the above-mentioned regions can be understood as the order in which the user expects to focus on the regions. By utilizing the priority of the different categories of regions, the target region of the highest priority target category can be determined within the target region corresponding to the obtained region features. Based on the region features of the target region of the determined higher priority target category, the region to be displayed in the video frame can be determined. In this way, when playing a video in the player, the region to be displayed within the video frame can show the user the region in the video frame that the user most expects to focus on, thereby improving the user's viewing experience.

[0204] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0205] In one embodiment of this disclosure, an electronic device is provided, comprising:

[0206] At least one processor; and

[0207] A memory communicatively connected to the at least one processor; wherein,

[0208] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform any of the video playback methods described in the foregoing method embodiments.

[0209] In one embodiment of this disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to perform any of the video playback methods described in the foregoing method embodiments.

[0210] In one embodiment of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the video playback methods described in the foregoing method embodiments.

[0211] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0212] It should be noted that the head model in this embodiment is not a head model for a specific user and cannot reflect the personal information of a specific user.

[0213] It should be noted that the two-dimensional face images in this embodiment are from a publicly available dataset.

[0214] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0215] Figure 9 A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0216] like Figure 9 As shown, device 900 includes a computing unit 901, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 902 or a computer program loaded from storage unit 908 into random access memory (RAM) 903. RAM 903 may also store various programs and data required for the operation of device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via bus 904. Input / output (I / O) interface 905 is also connected to bus 904.

[0217] Multiple components in device 900 are connected to I / O interface 905, including: input unit 906, such as keyboard, mouse, etc.; output unit 907, such as various types of monitors, speakers, etc.; storage unit 908, such as disk, optical disk, etc.; and communication unit 909, such as network card, modem, wireless transceiver, etc. Communication unit 909 allows device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0218] The computing unit 901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as the video playback method. For example, in some embodiments, the video playback method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program may be loaded and / or installed on device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the video playback method described above may be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the video playback method by any other suitable means (e.g., by means of firmware).

[0219] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0220] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0221] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0222] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0223] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0224] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0225] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0226] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A video playback method, comprising: Obtain the regional features of a target region within a video frame, wherein the target region is: a region that is representative of the content in the video frame; Using the aspect ratio of the player on the screen as the basis for the area size, and maximizing the number of target areas contained as the criterion for area selection, the area to be displayed in the video frame is determined based on the obtained area characteristics. The area to be displayed within a video frame of the video is played in the player; The video frame contains multiple categories of target regions, including visually salient regions, object regions, and text regions. A visually salient region is defined as a sub-region whose visual salience parameter, used to characterize the visual salience of the sub-region, is greater than a preset threshold among multiple sub-regions divided into the video frame. The region to be displayed is determined in the following way: Based on the obtained regional features, the system detects whether there is a target region of the current priority category in the video frame, according to the priority of different categories in descending order of regional priority. The priority of different categories is determined according to the content category of the video. If present, the area to be displayed in the video frame is determined based on the aspect ratio of the player on the screen as the area size reference, and the area selection criterion is to maximize the number of target areas of the target category contained therein. If it does not exist, the current priority is adjusted to the next priority, and the step of detecting whether there is a target region of the target category of the current priority in the video frame based on the obtained regional features is executed.

2. The method according to claim 1, wherein, The process of determining the display area in a video frame based on the aspect ratio of the player on the screen as the area size reference and maximizing the number of target areas included as the area selection criterion, and based on the obtained area features, includes: For each category, the area to be displayed in the video frame is determined as follows: Based on the size of the video frames, determine the maximum area size that satisfies the aspect ratio of the player on the screen; Based on the regional characteristics of the target region of this category, candidate regions in the video frame that contain the target region of this category and have a size equal to the maximum region size are determined; From the candidate regions, identify the region that contains the most complete target regions of that category, and obtain the region to be displayed in the video frame based on the identified region.

3. The method according to claim 2, further comprising: Based on the location of the region represented by the first feature, it is detected whether the target region of this category in the region to be displayed is complete, wherein the first feature is: the feature of the target region of this category; If incomplete, adjust the position and / or size of the area to be displayed based on the location of the incomplete area to obtain a display area where all target areas of the included category are complete.

4. The method according to claim 2 or 3, further comprising: Based on the region location represented by the second feature, it is determined whether other categories of target regions in the region to be displayed are complete, wherein the second feature is: the feature of other categories of target regions; If incomplete, adjust the position and / or size of the area to be displayed based on the location of the incomplete area to obtain a display area where all target areas of each category are complete.

5. The method according to claim 2 or 3, further comprising: If the number of areas to be displayed is greater than 1, the final area to be displayed is selected from the areas to be displayed based on the visual salience of each area, or the final area to be displayed is randomly selected from the areas to be displayed.

6. The method according to claim 2 or 3, wherein, The step of determining candidate regions in a video frame that contain target regions of that category and have a size equal to the maximum region size, based on the regional characteristics of the target region of that category, includes: Based on the area represented by the regional characteristics of the target region of this category, determine the salient region in the target region of this category; Based on the regional characteristics of the salient regions of this category, candidate regions in the video frame that contain the salient regions of this category and have a size equal to the maximum region size are determined; The process of determining the region that contains the most complete target regions of that category from the candidate regions, and obtaining the region to be displayed in the video frame based on the selected region, includes: From the candidate regions, identify the region that contains the most complete salient regions of that category, and obtain the region to be displayed in the video frame based on the selected region.

7. The method according to any one of claims 1-3, wherein, The video frame includes: the current keyframe in the video and multiple keyframes before and / or after the current keyframe.

8. A video playback device, comprising: The feature acquisition module is used to acquire the regional features of a target region within a video frame in the video, wherein the target region is: a region that is representative of the content in the video frame; The region determination module is used to determine the region to be displayed in the video frame based on the aspect ratio of the player on the screen as the region size reference and the maximization of the number of target regions contained as the region selection criterion. The video playback module is used to play the area to be displayed within a video frame in the video in the player; The video frame contains multiple categories of target regions, including visually salient regions, object regions, and text regions. A visually salient region is defined as a sub-region in which the visual saliency parameter, used to characterize the visual saliency of the sub-region, is greater than a preset threshold among multiple sub-regions divided into the video frame. The region determination module is specifically used for: Based on the obtained regional features, the system detects whether there is a target region of the current priority category in the video frame, according to the priority of different categories in descending order of regional priority. The priority of different categories is determined according to the content category of the video. If present, the area to be displayed in the video frame is determined based on the aspect ratio of the player on the screen as the area size reference, and the area selection criterion is to maximize the number of target areas of the target category contained therein. If it does not exist, the current priority is adjusted to the next priority, and the step of detecting whether there is a target region of the target category of the current priority in the video frame based on the obtained regional features is executed.

9. The apparatus according to claim 8, wherein, The feature acquisition module is specifically used for: For each category, the area to be displayed in the video frame is determined as follows: Based on the size of the video frames, determine the maximum area size that satisfies the aspect ratio of the player on the screen; Based on the regional characteristics of the target region of this category, candidate regions in the video frame that contain the target region of this category and have a size equal to the maximum region size are determined; From the candidate regions, identify the region that contains the most complete target regions of that category, and obtain the region to be displayed in the video frame based on the identified region.

10. The apparatus according to claim 9, wherein the feature acquisition module further comprises: Based on the location of the region represented by the first feature, it is detected whether the target region of this category in the region to be displayed is complete, wherein the first feature is: the feature of the target region of this category; If incomplete, adjust the position and / or size of the area to be displayed based on the location of the incomplete area to obtain a display area where all target areas of the included category are complete.

11. The apparatus according to claim 9 or 10, wherein the feature obtaining module further comprises: Based on the region location represented by the second feature, it is determined whether other categories of target regions in the region to be displayed are complete, wherein the second feature is: the feature of other categories of target regions; If incomplete, adjust the position and / or size of the area to be displayed based on the location of the incomplete area to obtain a display area where all target areas of each category are complete.

12. The apparatus according to claim 9 or 10, wherein the feature acquisition module further comprises: If the number of areas to be displayed is greater than 1, the final area to be displayed is selected from the areas to be displayed based on the visual salience of each area, or the final area to be displayed is randomly selected from the areas to be displayed.

13. The apparatus according to claim 9 or 10, wherein, The feature acquisition module is specifically used for: Based on the area represented by the regional characteristics of the target region of this category, determine the salient region in the target region of this category; Based on the regional characteristics of the salient regions of this category, candidate regions in the video frame that contain the salient regions of this category and have a size equal to the maximum region size are determined; From the candidate regions, identify the region that contains the most complete salient regions of that category, and obtain the region to be displayed in the video frame based on the selected region.

14. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-7.

15. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-7.

16. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-7.

Images