Display device and display area adjustment method

Abstract

Embodiments of the present application provide a display device and a display area adjustment method, which comprises: a display device parses a target video and determines a region of interest in the target video; adjusts the display size of the region of interest to determine a target display size of the region of interest in a to-be-switched window; and then controls the display to display the target video in the to-be-switched window according to the target display size. The region of interest is an image region in which a target object exists in the target video. In this way, the display size of the region of interest in the target video is adjusted in the to-be-switched window, which can maximize the display of the content of the region of interest and improve the user experience.

Description

Technical Field

[0001] This application relates to the field of terminal technology, and in particular to a display device and a method for adjusting the display area. Background Technology

[0002] With people placing greater emphasis on healthy lifestyles and the development of display device technology, users can play fitness videos on display devices and follow along to maintain their health.

[0003] In related technologies, if a user switches the display device from landscape to portrait mode, the display device proportionally determines the target display size of the fitness video in portrait mode based on the display window size corresponding to the portrait mode. That is, the width of the display area in landscape mode is the same as the width of the display window in portrait mode. Simultaneously, according to the aspect ratio of the fitness video, the height of the display area in portrait mode is compressed, allowing the fitness video to continue playing within that display area.

[0004] However, during the switching between portrait and landscape modes to play fitness videos, the size of the video display area after switching does not match the size of the display window, resulting in the figures in the fitness videos appearing too small and the fitness movements not being clearly displayed, leading to a poor user experience when following along. Summary of the Invention

[0005] This application provides a display device and a display area adjustment method, which can adjust the display size of the area of ​​interest in a video during the switching between portrait and landscape modes, so as to display the content in the area of ​​interest to the greatest extent.

[0006] In a first aspect, this application provides a display device, including a display and a controller; wherein the display is configured to display a target video; and the controller is configured to perform the following steps:

[0007] The target video is analyzed to determine the region of interest (ROI); the ROI is the image region in the target video where the object of interest is located.

[0008] Adjust the display size of the region of interest to determine the target display size of the region of interest in the window to be switched;

[0009] Control the monitor to display the target video in the window to be switched, according to the target display size.

[0010] In some embodiments of this application, the controller is further configured as follows:

[0011] Obtain the original size information of the region of interest in the target video;

[0012] Get the window size information of the window to be switched;

[0013] Based on the original size information and window size information, the original size information is adjusted to determine the target display size of the region of interest in the window to be switched.

[0014] The original size information includes the region width and region height values, and the window size information includes the window width and window height values.

[0015] In some embodiments of this application, if the region of interest is displayed in portrait mode and the window to be switched is displayed in landscape mode, the controller is further configured as follows:

[0016] Determine the horizontal expansion size based on the original size information and window size information;

[0017] Based on the horizontal expansion size, the display width value of the region of interest is expanded and adjusted to determine the target display size of the region of interest in the window to be switched.

[0018] In some embodiments of this application, if the region of interest is displayed in landscape mode and the window to be switched is displayed in portrait mode, the controller is further configured to:

[0019] Determine the vertical expansion dimensions based on the original size information and window size information;

[0020] Based on the vertical expansion dimension, the display height value of the region of interest is expanded and adjusted to determine the target display size of the region of interest in the window to be switched.

[0021] In some embodiments of this application, the controller is further configured as follows:

[0022] Based on the center point of the region of interest, the region of interest is expanded uniformly along the horizontal or vertical direction according to the target display size to determine the display area of ​​the target video in the window to be switched.

[0023] In response to a window switching command, control the monitor to display the target video in the display area according to the target display size.

[0024] In some embodiments of this application, if the target video is a fitness video, the region of interest of the target video includes the action safety regions corresponding to multiple fitness movements in the fitness video.

[0025] The controller is further configured as follows:

[0026] Based on the video tracking log of the fitness video, obtain the set of motion trajectories corresponding to each fitness movement in the fitness video; the video tracking log includes the motion identifiers and start and end times of multiple fitness movements.

[0027] Based on the set of motion trajectories of the target fitness movement, determine the movement safety zone corresponding to the target fitness movement; the target fitness movement is any fitness movement included in the video tracking log.

[0028] In some embodiments of this application, the motion trajectory set includes a set of human body key points corresponding to multiple sub-movements in a fitness exercise; the controller is further configured to:

[0029] Based on the set of human key points corresponding to multiple sub-movements in the target fitness movement, determine the sub-movement area corresponding to each sub-movement in the target fitness movement;

[0030] Based on the sub-movement regions corresponding to each sub-movement in the target fitness movement, the movement safety region corresponding to the target fitness movement is determined; the movement safety region includes the sub-movement regions corresponding to all sub-movements in the target fitness movement.

[0031] In some embodiments of this application, the motion trajectory set includes a set of human body key points corresponding to multiple sub-movements in a fitness exercise; the controller is further configured to:

[0032] Based on the set of human key points corresponding to multiple sub-movements in the target fitness movement, determine the movement edge information corresponding to the target fitness movement; the movement edge information includes the maximum and minimum values ​​of the horizontal coordinates of the human key points determined based on each set of human key points, as well as the maximum and minimum values ​​of the vertical coordinates of the human key points.

[0033] Based on the motion edge information, determine the motion safety zone corresponding to the target fitness motion.

[0034] In some embodiments of this application, the controller is further configured as follows:

[0035] Based on the video tracking logs of fitness videos, obtain the action icons for each fitness movement;

[0036] Based on the motion identifiers of each fitness movement, the set of motion trajectories corresponding to each fitness movement is obtained from a preset motion database.

[0037] Secondly, this application provides a method for adjusting a display area, the method comprising:

[0038] The target video is analyzed to determine the region of interest (ROI); the ROI is the image region in the target video where the object of interest is located.

[0039] Adjust the display size of the region of interest to determine the target display size of the region of interest in the window to be switched;

[0040] The target video is displayed in the window to be switched according to the target display size.

[0041] Thirdly, this application also provides a computer-readable storage medium storing a computer program that, when executed by a controller in a display device, can implement some or all of the steps of the display area adjustment method provided in this application.

[0042] Fourthly, this application also provides a computer program product, which includes a computer program that, when executed by a controller in a display device, can implement some or all of the steps of the display area adjustment method provided in this application.

[0043] The technical solution provided in this application can achieve at least the following beneficial effects:

[0044] In the display device and display area adjustment method provided in this application embodiment, the display device analyzes the target video and determines the region of interest (ROI) within the target video; it adjusts the display size of the ROI to determine the target display size of the ROI in the window to be switched; and then controls the display to display the target video in the window to be switched according to the target display size. The ROI is an image area in the target video containing an object of interest. Thus, by adjusting the display size of the ROI in the target video within the window to be switched, this application can maximize the display of the ROI's content, improving the user's video viewing experience. Attached Figure Description

[0045] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is a schematic diagram illustrating an operational scenario between a display device and a control device, as shown in an exemplary embodiment of this application.

[0047] Figure 2 This is a hardware configuration block diagram of a display device illustrated in an exemplary embodiment of this application;

[0048] Figure 3a An exemplary embodiment of this application illustrates a schematic diagram of displaying a video image in landscape mode;

[0049] Figure 3b This application provides an exemplary embodiment illustrating a diagram of displaying video footage in portrait mode.

[0050] Figure 4 This is a flowchart illustrating an exemplary embodiment of the present application of a method for adjusting the display area using a display device;

[0051] Figure 5 This is a flowchart illustrating an exemplary embodiment of this application for obtaining a region of interest in a fitness video;

[0052] Figure 6 This is a schematic diagram illustrating an exemplary embodiment of this application for obtaining a set of motion trajectories corresponding to fitness movements;

[0053] Figure 7 This is a schematic diagram illustrating a method for determining a safe area for movement, as shown in an exemplary embodiment of this application.

[0054] Figure 8 This is a schematic diagram illustrating another way of determining a safe zone for movement, as shown in an exemplary embodiment of this application;

[0055] Figure 9 This is a schematic diagram illustrating an exemplary embodiment of this application for determining a target display size;

[0056] Figure 10 This is a schematic diagram illustrating another method for determining the target display size, as shown in an exemplary embodiment of this application.

[0057] Figure 11 This is a schematic diagram illustrating an extended display size of the region of interest, as shown in an exemplary embodiment of this application. Detailed Implementation

[0058] To make the objectives and implementation methods of this application clearer, the exemplary implementation methods of this application will be clearly and completely described below with reference to the accompanying drawings of the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only some embodiments of this application, and not all embodiments.

[0059] Before explaining the display device and display area adjustment method provided in this application, the implementation environment of the embodiments of this application will be introduced first.

[0060] See Figure 1 In the interaction scenario between the display device and the control device, the user can operate the display device 200 through the terminal device 300 or the control device 100.

[0061] The display device 200 can take many forms, such as a television, a smart television, a computer, a laser projection device, a monitor, an electronic bulletin board, an electronic table, etc.

[0062] In some embodiments, the control device 100 may be a remote control. Communication between the remote control and the display device includes infrared protocol communication, Bluetooth protocol communication, and other short-range communication methods, controlling the display device 200 wirelessly or via wired means. Users can input user commands through buttons on the remote control, voice input, control panel input, etc., to control the display device 200.

[0063] In some embodiments, a terminal device 300 (such as a mobile terminal, tablet computer, computer, laptop computer, etc.) may be used to control the display device 200. For example, an application running on the terminal device 300 may be used to control the display device 200.

[0064] In some embodiments, the display device 200 may also be controlled by the user via touch or gestures.

[0065] In some embodiments, the display device 200 can also be controlled in ways other than the control device 100 and the terminal device 300. For example, it can be controlled by directly receiving the user's voice commands through a module configured inside the display device 200 for acquiring voice commands, or it can be controlled by receiving the user's voice commands through a voice control device set outside the display device 200.

[0066] In some embodiments, the display device 200 can also communicate with the server 400. The display device 200 can communicate via a local area network (LAN), wireless local area network (WLAN), or other networks. The server 400 can provide the display device 200 with various display and interactive content.

[0067] As an example, server 400 can be a standalone server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0068] Based on the above embodiments, Figure 2 A hardware configuration block diagram of the display device 200 described above is shown. The display device 200 includes at least one of the following: a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory 280, a power supply, and a user interface.

[0069] In some embodiments, the tuner 210 receives broadcast television signals via wired or wireless reception and demodulates audio and video signals from multiple wireless or wired broadcast television signals. For example, Electronic Program Guide (EPG) data signals.

[0070] In some embodiments, the communicator 220 is a component for communicating with external devices or the server 400 according to various communication protocol types. For example, the communicator may include at least one of the following: a Wireless Fidelity (Wi-Fi) module, a Bluetooth module, a wired Ethernet module, other network communication protocol chips or near-field communication protocol chips, and an infrared receiver.

[0071] In this embodiment of the application, the display device 200 can establish a connection with the terminal device 300 through the communicator 220 to send and receive control signals, data signals, etc.

[0072] In some embodiments, detector 230 is used to acquire signals from the external environment or to interact with the outside world. For example, detector 230 includes a light receiver, a sensor for acquiring ambient light intensity; or, detector 230 includes an image acquisition device, such as a camera, which can be used to acquire external environmental scenes, user attributes, or user interaction gestures; or, detector 230 includes a sound acquisition device, such as a microphone, for receiving external sounds.

[0073] In some embodiments, the external device interface 240 may include, but is not limited to, one or more of the following: High Definition Multimedia Interface (HDMI), analog or high-definition component input interface (component), Composite Video Broadcast Signal (CVBS), Universal Serial Bus (USB) input interface, RGB port, etc. It may also be a composite input / output interface formed by multiple of the above interfaces.

[0074] In some embodiments, the controller 250 includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), a RAM (RandomAccess Memory), a ROM (Read-Only Memory), a first to an nth interface for input / output, a communication bus, etc.

[0075] In practice, the controller 250 controls the operation of the display device 200 and responds to user operations through various software control programs stored in the memory 280. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command to select a UI object to display on the monitor 260, the controller 250 can execute operations related to the object selected by the user command.

[0076] In this embodiment, the controller 250 is used to parse the target video to determine the region of interest in the target video; then, based on the display size information of the region of interest and the display size information of the window to be switched, it determines the target display size of the region of interest in the window to be switched, and then controls the display to display the target video in the window to be switched according to the target display size.

[0077] In addition, in this embodiment, the memory 280 is also used to store video resources, video tracking logs, and motion databases, so that when the controller 250 implements the display area adjustment method provided in this application, it can obtain the target video resources from the memory 280 to play the target video.

[0078] It should be noted that the controller 250 and the tuner 210 can be located in different separate devices. That is, the tuner 210 can also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0079] In some embodiments, the display 260 includes a display screen component for presenting images, a driving component for driving image display, a component for receiving image signals output by the controller 250, and a user control UI interface for displaying video content, image content, menu control interface, and user control UI interface.

[0080] As an example, the display 260 may be a liquid crystal display, an OLED display, or a projection display, and may also be a projection device and a projection screen.

[0081] In this embodiment, the display 260 is used to display the target video.

[0082] Furthermore, the user can input user commands through the graphical user interface (GUI) displayed on the monitor 260, and the user input interface receives the user input commands through the GUI. Alternatively, the user can input user commands by entering specific sounds or gestures, and the user interface receives the user input commands by recognizing the sounds or gestures through sensors.

[0083] The "user interface" is the medium through which an application or operating system interacts and exchanges information with the user. It converts the internal form of information into a form that the user can accept. For example, a user interface can be an icon, window, control, or other interface element displayed on the screen of a display device 200. Controls can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0084] It should be understood that, Figure 2 The display device 200 shown is merely an example, and the display device 200 may have more than... Figure 2 The more or fewer components shown can be combined into two or more components, or they can have different component configurations. Figure 2 The various components shown can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0085] After introducing the control and interaction scenarios and specific hardware structure of the display device, the application scenarios of the display area adjustment method provided in this application will be explained based on the aforementioned display device 200.

[0086] The videos currently produced are not well adapted to large-screen displays. The size of the video's display area does not match the size of the display window on the display device, which can easily cause the display size of the area of ​​interest in the video to be too small when playing the video. At the same time, black borders may appear in the display window when the display device is in different display states.

[0087] As an example, if the video was recorded in its original 16:9 aspect ratio, see [link to video]. Figure 3a When the display device is in landscape mode, since the maximum aspect ratio of the display area in landscape mode can be 16:9, the video can be proportionally enlarged to fill the entire display window, thus clearly showing the fitness movements of the person in the video. However, when switching the display device from landscape to portrait mode, see [link to relevant documentation]. Figure 3b Because the maximum aspect ratio of the display window in portrait mode is 9:16, the width of the display area of ​​the video in portrait mode can be the same as the width of the display window. However, if the original aspect ratio of the video is kept unchanged, the height of the display area cannot be the same as the height of the display window. This results in the characters in the video appearing too small, and users may not be able to clearly see the characters' fitness movements. In addition, there are black borders at the top and bottom of the display window in portrait mode, which makes the user's video viewing experience poor.

[0088] Similarly, if the video is recorded in its original 9:16 aspect ratio, when the display device is in portrait mode, the maximum aspect ratio of the display area in portrait mode is 9:16. Therefore, the video can be proportionally enlarged to fill the entire display window, clearly showing the fitness movements of the person in the video. However, when the display device is switched from portrait to landscape mode, the maximum aspect ratio of the display window in landscape mode is 16:9. In landscape mode, the height of the video's display area can be the same as the height of the display window, but while maintaining the original aspect ratio of the video, the width of the display area cannot be the same as the width of the display window. This results in the person in the video appearing too small, and the user may not be able to clearly see the person's fitness movements. Moreover, in landscape mode, there are black bars on the left and right sides of the display window, leading to a poor viewing experience for the user.

[0089] Based on this, this application provides a display area adjustment method, which enables the display device to identify the region of interest in the video and adjust the display size of the region of interest to determine the target display size of the region of interest in the window to be switched, thereby displaying the target video in the window to be switched at the maximum ratio to clearly display the content of the region of interest; at the same time, it reduces the black borders when the target video is displayed in the window to be switched to a certain extent, improving the user's viewing experience.

[0090] Next, the technical solutions of the embodiments of this application, and how the technical solutions of the embodiments of this application solve the above-mentioned technical problems, will be described in detail with reference to the accompanying drawings. The embodiments shown below can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0091] In one exemplary embodiment, such as Figure 4 As shown, when the display device in this application performs the display area adjustment method, the controller in the display device is configured to perform the following steps:

[0092] Step 410: Analyze the target video and determine the region of interest in the target video; the region of interest is the image region in the target video where the object of interest exists.

[0093] Among them, the objects of interest can be people, items, target content, or other objects that users want to focus on in the video.

[0094] It should be noted that the region of interest (ROI) can be the image region containing a single object of interest, or it can be the image region containing multiple objects of interest. If the ROI contains multiple objects of interest, then the ROI covers the image regions corresponding to each of the multiple objects of interest; in other words, the ROI can be understood as the sum of the image regions corresponding to each of the multiple objects of interest.

[0095] In some embodiments, the target video may be, but is not limited to, any video that can be used by users for learning or practice, such as professional course videos, fitness videos, dance videos, musical instrument performance videos, item assembly videos, equipment repair videos, etc.

[0096] Among them, professional course videos can be, but are not limited to, online English courses, online math courses, video editing tutorials, classical poetry and prose explanation courses, etc.

[0097] If the target video is a professional course video, the display device may perform step 410 above in the following ways, including but not limited to:

[0098] (1) Analyze the target video frame by frame to determine the region of interest corresponding to each frame, and obtain multiple regions of interest.

[0099] Furthermore, steps 420 and 430 are performed on the region of interest corresponding to each frame of the image to determine the target display size of the region of interest in each frame of the image in the window to be switched, so that the target video is played sequentially in the window to be switched at the target display size corresponding to each frame of the image.

[0100] In this way, a corresponding region of interest is determined for each frame of the target video. During the playback of the target video, the display size of the region of interest is adjusted frame by frame according to the region of interest corresponding to each frame, so as to clearly display the region of interest in each frame and enhance the user's video viewing experience.

[0101] (2) Analyze the target video frame by frame to determine the region of interest in each frame and obtain multiple image regions; determine a region of interest corresponding to the target video based on the multiple image regions.

[0102] Furthermore, steps 420 and 430 are performed on the region of interest corresponding to the target video to determine the target display size of the region of interest in the window to be switched, so that when the target video is played in the window to be switched, each frame of the image is displayed according to the target display size corresponding to the region of interest.

[0103] In other words, in this implementation, the size of the region of interest corresponding to each frame of the target video is the same, so when the target video is played in the window to be switched, the target display size of the region of interest in each frame is also the same.

[0104] In this way, by defining the same region of interest for each frame of the target video, the display size of the region of interest does not need to be adjusted frame by frame during the playback of the target video. This reduces the amount of data processing during the playback of the target video, ensuring smooth playback and reducing the possibility of stuttering.

[0105] Furthermore, frame-by-frame detection can only determine the region of interest (ROI) based on the current frame or several previous frames, and cannot predict the ROI for subsequent frames. However, when the video type can be determined, the ROI in the target video is relatively fixed. Therefore, the overall ROI of the target video can be determined by detecting a preset number of frames. In this way, the number of frames to be recognized in the target video can be greatly reduced when determining the ROI.

[0106] The preset frame rate can be any pre-set value, such as 5 frames, 10 frames, 30 frames, etc., and this application embodiment does not limit it.

[0107] It should be noted that in the two implementation methods described above, frame-by-frame parsing can employ machine algorithms such as human face following algorithms and region of interest (ROI) recognition algorithms to determine the ROI corresponding to each frame. This application does not limit the machine algorithms that can be used to identify ROIs from images.

[0108] In addition, to improve the efficiency of identifying regions of interest (ROIs) in images, machine learning models, such as deep neural network models, can be pre-trained. These trained models can then be used to analyze the target video frame by frame to determine the ROI for each frame or the ROI for the entire target video.

[0109] If the target video is a fitness video, then the region of interest (ROI) of the target video includes the safe zones corresponding to the various fitness movements within the video. For example... Figure 5 As shown, when the display device performs step 410 above, it may include the following steps:

[0110] Step 412: Based on the video tracking log of the fitness video, obtain the set of motion trajectories corresponding to each fitness movement in the fitness video; the video tracking log includes the motion identifiers and start and end times of multiple fitness movements.

[0111] The action identifier can be, but is not limited to, action ID, action symbol, action name, action information code, or other information that can uniquely represent an action. This application does not impose any restrictions on this.

[0112] It should be understood that a fitness video may include one or more fitness movements, and each fitness movement corresponds to a set of movement trajectories.

[0113] In some embodiments, the video logging may also include video information such as video name and frame rate, but this application does not limit this.

[0114] In one possible implementation, step 412 can be implemented as follows: obtain the motion identifier of each fitness movement based on the video dot log of the fitness video; and obtain the set of motion trajectories corresponding to each fitness movement from the preset motion database based on the motion identifier of each fitness movement.

[0115] The motion trajectory set includes a set of key human body points corresponding to multiple sub-movements in a fitness exercise. Key human body points may include, but are not limited to, points such as the brow, shoulder joint, elbow joint, wrist joint, hip joint, knee joint, and ankle joint.

[0116] Furthermore, each fitness movement may correspond to multiple sub-movements, and each sub-movement corresponds to a set of key points on the human body. This application does not limit the number of sub-movements corresponding to a single fitness movement.

[0117] As an example, sub-actions can be determined based on at least one frame of image content corresponding to a fitness movement in the target video, and the number of sub-actions is the same as the number of image frames corresponding to the fitness movement in the fitness video.

[0118] As another example, sub-movements can be determined by analyzing the fitness movement. For instance, in a fitness movement, a sub-movement could be arms open at shoulder level; it could be hands clapping above the head; or it could be a half-squatting posture with hands clenched into fists and tucked to the waist. This application does not limit the number of sub-movements that a fitness movement can be broken down into.

[0119] Specifically, for any fitness movement in a fitness video, the movement information of the fitness movement is obtained from a preset movement database based on the movement identifier of the fitness movement, and then the set of human body key points of multiple sub-movements corresponding to the fitness sub-movement is obtained based on the movement information of the fitness sub-movement.

[0120] In other words, a fitness movement corresponds to a set of movement trajectories, and the set of movement trajectory points includes the sets of human body key points corresponding to multiple sub-movements in the fitness movement; that is, a set of movement trajectory points includes multiple sets of human body key points that form the movement trajectory.

[0121] As an example, see Figure 6 The video log 1 corresponding to fitness video 1 includes three fitness movements: movement 1, movement 3, and movement 7, as well as the start and end times of movement 1, movement 3, and movement 7. The playback duration of this fitness video is the difference between the end time of movement 7 and the start time of movement 1.

[0122] Furthermore, based on the action identifier of action 1, action information 1 corresponding to action 1 is obtained from the preset action database. Action information 1 includes the action name and action ID of action 1, as well as multiple sets of human key points (human key point set 1, human key point set 2, ... human key point set H) corresponding to the completion of action 1.

[0123] Step 414: Determine the safe zone for the target fitness movement based on the set of movement trajectories of the target fitness movement; the target fitness movement is any fitness movement included in the video tracking log.

[0124] In one possible implementation, step 414 can be implemented as follows: based on the set of human body key points corresponding to multiple sub-movements in the target fitness movement, determine the sub-movement region corresponding to each sub-movement in the target fitness movement; based on the sub-movement region corresponding to each sub-movement in the target fitness movement, determine the movement safety region corresponding to the target fitness movement.

[0125] The safe zone for movement includes the sub-movement areas corresponding to all sub-movements in the target fitness movement.

[0126] As an example, suppose the target fitness movement includes three sub-movements: sub-movement A, sub-movement B, and sub-movement C, corresponding to three sets of human body key points: human body key point set a, human body key point set b, and human body key point set c.

[0127] join Figure 7 Based on the set of human body key points 'a' corresponding to sub-action A, determine the sub-action region corresponding to sub-action A, i.e. Figure 7 Region 1 in the text; based on the set of human key points b corresponding to sub-action B, determine the sub-action region corresponding to sub-action B, i.e. Figure 7 Region 2 in the text; based on the set of human key points c corresponding to sub-action B, determine the sub-action region corresponding to sub-action C, i.e. Figure 7 Region 3 in the middle.

[0128] Furthermore, based on regions 1, 2, and 3, the safe movement zones corresponding to the target fitness movements are determined, namely... Figure 7 The area indicated by the dashed box.

[0129] In another possible implementation, step 414 can be implemented as follows: determine the action edge information corresponding to the target fitness movement based on the set of human body key points corresponding to multiple sub-movements in the target fitness movement; determine the action safety area corresponding to the target fitness movement based on the action edge information.

[0130] Among them, the motion edge information includes the maximum and minimum values ​​of the horizontal coordinates of human key points determined based on each set of human key points, as well as the maximum and minimum values ​​of the vertical coordinates of human key points.

[0131] As an example, let's assume that the target fitness movement includes three sub-movements: sub-movement A, sub-movement B, and sub-movement C, corresponding to three sets of human body key points: human body key point set a, human body key point set b, and human body key point set c.

[0132] join Figure 8 According to the set of key points of the human body a( Figure 8 (white dots in the image), set of key human body points b ( Figure 8 The triangle symbol in the image), and the set of key points of the human body c ( Figure 8 The coordinates of key human body points (black dots in the diagram) are used to determine the maximum and minimum values ​​of the horizontal and vertical coordinates.

[0133] It should be noted that the origin of the image coordinate system is located at the upper left corner of the image, the horizontal direction is the x-axis of the image coordinate system, and the vertical direction is the y-axis of the image coordinate system.

[0134] Furthermore, by connecting the human body key point g1 with the smallest x-coordinate, the human body key point g2 with the largest x-coordinate, the human body key point g3 with the smallest y-coordinate, and the human body key point g4 with the largest y-coordinate along the coordinate axes, the safe movement area corresponding to the target fitness movement is obtained, i.e. Figure 8 The area indicated by the dashed box.

[0135] It should be understood that multiple human keypoints may have the same x-coordinate or y-coordinate. Therefore, there can be one or more human keypoints with the largest x-coordinate. Similarly, there can be one or more human keypoints with the smallest x-coordinate, the largest y-coordinate, and the smallest y-coordinate. For example, Figure 8 The human body key point g3 with the smallest vertical coordinate includes two coordinate points.

[0136] Step 420: Adjust the display size of the region of interest to determine the target display size of the region of interest in the window to be switched.

[0137] The aspect ratio of the target display size is the same as that of the window to be switched. It should be understood that the aspect ratio of the target display size is the ratio between the display width and the display height, while the aspect ratio of the window to be switched is the ratio between the window width and the window height.

[0138] As an example, the aspect ratio can be 16:9, 9:16, 4:3, 3:4, 1:1, etc., and this application does not limit this.

[0139] In some of these embodiments, the implementation process of step 420 can be as follows: Obtain the original size information of the region of interest corresponding in the target video; obtain the window size information of the window to be switched; according to the original size information and the window size information, adjust the original size information to determine the target display size of the region of interest in the window to be switched.

[0140] Among them, the original size information includes a region width value and a region height value, and the window size information includes a window width value and a window height value. It should be understood that in an image, the region width value and the window width value are the number of pixels displayed horizontally, and the region height value and the window height value are the number of pixels displayed vertically, and their unit can be px.

[0141] It should be noted that the size information of the region of interest when the display device is in the landscape display state is different from the size information of the region of interest when the display device is in the portrait display state, and the aspect ratios in the landscape and portrait display states are reciprocals of each other. For example, if the aspect ratio of the region of interest in the landscape display state is 16:9, then the aspect ratio of the region of interest in the portrait display state is 9:16.

[0142] In a possible implementation manner, according to the original size information of the region of interest and the window size information of the window to be switched, adjust the original size information. The specific display width value or display height value to be adjusted can be judged according to the original aspect ratio of the region of interest in the target video and the aspect ratio of the window to be switched.

[0143] Suppose the region width value in the original size information is aW, and the region height value is aH, then the original aspect ratio aR of the region of interest in the target video = aW / aH. Suppose the window width value in the window size information is sW, and the window height value is sH, then the aspect ratio sR of the window to be switched = sW / sH.

[0144] If sR > aR, the region of interest may be in the portrait display state, and the window to be switched may be in the landscape display state, resulting in insufficient horizontal display width of the region of interest in the window to be switched, and it is necessary to expand and adjust the display width value of the region of interest; if sR < aR, the region of interest may be in the landscape display state, and the window to be switched may be in the portrait display state, resulting in insufficient vertical display height of the region of interest in the window to be switched, and it is necessary to expand and adjust the display height value of the region of interest.

[0145] It should be noted that if sR = aR, there is no need to expand the region of interest additionally, and the region of interest can be expanded and displayed proportionally according to the aspect ratio of the window to be switched.

[0146] Furthermore, adjusting the original size information based on the original size information and the window size information can include the following two cases:

[0147] (1) If the region of interest is in portrait mode and the window to be switched is in landscape mode, the process of the controller adjusting the original size information can be as follows: determine the horizontal expansion size based on the original size information and the window size information; expand and adjust the display width value of the region of interest based on the horizontal expansion size to determine the target display size of the region of interest in the window to be switched.

[0148] When switching to landscape mode, the display height of the target display size can be the same as the window height of the window to be switched.

[0149] It should be noted that in this case, the display height of the region of interest (ROI) within the corresponding display area of ​​the window to be switched is fixed, and its maximum height can be the same as the window height of the window to be switched. Since the aspect ratio of the window to be switched is preset, when the target video is displayed in the window to be switched, the display area of ​​the ROI within the window to be switched will also be adaptively adjusted proportionally. However, the adaptively adjusted display width will be smaller than the window width of the window to be switched, resulting in a black border around the window.

[0150] Based on this, in order to ensure that the content of the region of interest can be displayed to the maximum extent, this application expands the display width of the region of interest when the display height value is the same as the window height value of the window to be switched.

[0151] In one possible implementation, the display area of ​​the region of interest within the window to be switched is determined based on the window width and region height values. Then, a target width value for the display area is calculated based on the ratio between the display area range and the window height. Finally, the horizontal expansion size is determined based on the difference between the target width value and the region width value.

[0152] As an example, the lateral expansion dimension w′ can be determined by the following formula (1):

[0153]

[0154] In some of these embodiments, see Figure 9 Based on the center point of the region of interest, the expansion dimensions on both sides of the region of interest in the horizontal direction are w′ / 2.

[0155] (2) If the region of interest is in landscape mode and the window to be switched is in portrait mode, the process of the controller adjusting the original size information can be as follows: determine the vertical expansion size based on the original size information and the window size information; expand and adjust the display height value of the region of interest based on the vertical expansion size to determine the target display size of the region of interest in the window to be switched.

[0156] When switching to portrait mode, the display width of the target display size can be the same as the window width of the window to be switched.

[0157] It should be noted that in this case, the display width of the region of interest (ROI) within the corresponding display area of ​​the window to be switched is fixed, and its maximum width can be the same as the window width of the window to be switched. Since the aspect ratio of the window to be switched is preset, when the target video is displayed in the window to be switched, the display area of ​​the ROI within the window to be switched will also be adaptively adjusted proportionally according to the aspect ratio. However, the adaptively adjusted display height will be less than the window height of the window to be switched, resulting in a black border in the window.

[0158] Based on this, in order to ensure that the content of the region of interest can be displayed to the maximum extent, this application expands the display height of the region of interest when the display width value is the same as the window width value of the window to be switched.

[0159] In one possible implementation, the display area of ​​the region of interest within the window to be switched is determined based on the window height and area width values. Then, a target height value for the display area is calculated based on the ratio between the display area range and the window width. Finally, the vertical expansion dimension is determined based on the difference between the target height value and the area height value.

[0160] As an example, the longitudinal extension dimension h′ can be determined by the following formula (2):

[0161]

[0162] In some of these embodiments, see Figure 10 Based on the center point of the region of interest, the expansion dimensions on both sides of the region of interest in the vertical direction are h′ / 2.

[0163] Step 430: Control the monitor to display the target video in the window to be switched according to the target display size.

[0164] In one possible implementation, step 430 can be implemented as follows: based on the center point of the region of interest, the region of interest is expanded uniformly along the horizontal or vertical direction according to the target display size to determine the display area corresponding to the target video in the window to be switched; in response to the window switching command, the display is controlled to display the target video in the display area according to the target display size.

[0165] Among them, window switching commands can be used by users through Figure 1 The control device shown can be triggered by voice, but this application embodiment does not limit this.

[0166] It should be noted that this application can execute the above-described display area adjustment method when the display device detects a window switching operation; alternatively, it can pre-determine the target display size of the area of ​​interest in the window to be switched based on the original size information corresponding to the region of interest in the target video, and then directly control the display to display the target video in the window to be switched according to the target display size after receiving the window switching instruction. This application embodiment does not limit the timing of executing steps 410 and 420.

[0167] In addition, since the center of the region of interest may not coincide with the center of the window to be switched, when displaying the target video according to the target display size, if one side coincides with the window edge when expanding the region based on the horizontal expansion size (or vertical expansion size), the remaining expansion size can be added to the other side to ensure that the sum of the expansion sizes on both sides of the region of interest is the same as the calculated horizontal expansion size (or vertical expansion size).

[0168] As an example, such as Figure 11 As shown, when adjusting the display width of the region of interest based on the horizontal expansion size, if the right side coincides with the edge of the window, the left side is expanded to ensure that the sum of the left expansion width value X1 and the right expansion width value X2 is the same as the calculated horizontal expansion size.

[0169] In this embodiment, the display device parses the target video and determines the region of interest (ROI) within it. Then, it adjusts the display size of the ROI to determine the target display size of the ROI within the window to be switched. Finally, it controls the display to show the target video in the window according to the target display size. This adjustment of the ROI display size within the window maximizes the display of the ROI content while reducing black borders and improving the user's viewing experience.

[0170] In addition, based on the same technical concept, this application also provides a display area adjustment method, which can be applied to the above-mentioned display device or other electronic devices. The method includes: parsing a target video to determine a region of interest in the target video; the region of interest is an image area in the target video where an object of interest exists; adjusting the display size of the region of interest to determine the target display size of the region of interest in the window to be switched; and displaying the target video in the window to be switched according to the target display size.

[0171] The implementation principle and beneficial effects of the display area adjustment method provided in the above embodiments can be found in the specific limitations of the various embodiments regarding the controller configuration content in the display device mentioned above, and will not be repeated here.

[0172] In one exemplary embodiment, this application also provides a computer-readable storage medium. This computer-readable storage medium may store a computer program, which, when invoked and executed by a controller in a display device or a processor in other electronic devices, can implement some or all of the steps in the display area adjustment method provided in this application.

[0173] As an example, the computer-readable storage medium can be a magnetic disk, optical disk, read-only memory, or random access memory, etc.

[0174] It should be understood that the technical solutions in the embodiments of this application can be implemented using software plus necessary general-purpose hardware platforms. Therefore, the technical solutions in the embodiments of this application, in essence or in part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in a computer-readable storage medium.

[0175] In one exemplary embodiment, this application also provides a computer program product. This computer program product includes a computer program that is invoked and executed by a controller in a display device or a processor in other electronic devices, and can implement some or all of the steps in the display area adjustment method provided in this application.

[0176] The above are merely specific implementations of the embodiments of this application and are not intended to limit the protection scope of the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solutions of the embodiments of this application should be included within the protection scope of the embodiments of this application.

Claims

1. A display device, characterized in that, The device includes: The monitor is configured to display the target video; The controller is configured as follows: The target video is parsed. If the target video is a fitness video, the motion trajectory set corresponding to each fitness movement in the fitness video is obtained according to the video point log of the fitness video. The video point log includes the action identifier and start and end time of multiple fitness movements. Based on the set of motion trajectories of the target fitness movement, the safe zone corresponding to the target fitness movement is determined; the target fitness movement is any fitness movement included in the video tracking log, and the region of interest of the fitness video includes the safe zones corresponding to multiple fitness movements in the fitness video. The display size of the region of interest is adjusted to determine the target display size of the region of interest in the window to be switched. Control the display to display the target video in the window to be switched according to the target display size.

2. The display device according to claim 1, characterized in that, The controller is further configured to: Obtain the original size information of the region of interest in the target video; Obtain the window size information of the window to be switched; Based on the original size information and the window size information, the original size information is adjusted to determine the target display size of the region of interest in the window to be switched. The original size information includes the region width value and the region height value, and the window size information includes the window width value and the window height value.

3. The display device according to claim 2, characterized in that, If the region of interest is displayed in portrait mode and the window to be switched is displayed in landscape mode, then the controller is further configured as follows: The horizontal expansion size is determined based on the original size information and the window size information; Based on the horizontal expansion dimension, the display width value of the region of interest is expanded and adjusted to determine the target display size of the region of interest in the window to be switched.

4. The display device according to claim 2, characterized in that, If the region of interest is in landscape mode and the window to be switched is in portrait mode, the controller is further configured to: Based on the original size information and the window size information, determine the vertical expansion size; Based on the vertical expansion dimension, the display height value of the region of interest is expanded and adjusted to determine the target display size of the region of interest in the window to be switched.

5. The display device according to any one of claims 1 to 4, characterized in that, The controller is further configured to: Based on the center point of the region of interest, the region of interest is expanded uniformly along the horizontal or vertical direction according to the target display size to determine the display area of ​​the target video in the window to be switched. In response to a window switching command, the display is controlled to display the target video in the display area according to the target display size.

6. The display device according to claim 1, characterized in that, The motion trajectory set includes a set of human body key points corresponding to multiple sub-movements in the fitness movement; the controller is further configured to: Based on the set of human body key points corresponding to multiple sub-movements in the target fitness movement, determine the sub-movement area corresponding to each sub-movement in the target fitness movement; Based on the sub-action regions corresponding to each of the sub-actions in the target fitness movement, a safe movement region corresponding to the target fitness movement is determined; the safe movement region includes the sub-action regions corresponding to all the sub-actions in the target fitness movement.

7. The display device according to claim 1, characterized in that, The motion trajectory set includes a set of human body key points corresponding to multiple sub-movements in the fitness movement; the controller is further configured to: Based on the set of human body key points corresponding to multiple sub-movements in the target fitness movement, determine the movement edge information corresponding to the target fitness movement; the movement edge information includes the maximum and minimum values ​​of the horizontal coordinates of the human body key points determined based on each set of human body key points, and the maximum and minimum values ​​of the vertical coordinates of the human body key points. Based on the motion edge information, the motion safety area corresponding to the target fitness motion is determined.

8. The display device according to claim 1, characterized in that, The controller is further configured to: Based on the video tracking log of the fitness video, obtain the motion identifiers for each fitness movement; Based on the action identifier of each fitness action, the set of motion trajectories corresponding to each fitness action is obtained from a preset action database.

9. A method for adjusting a display area, characterized in that, The method includes: The target video is parsed. If the target video is a fitness video, the motion trajectory set corresponding to each fitness movement in the fitness video is obtained according to the video point log of the fitness video. The video point log includes the action identifier and start and end time of multiple fitness movements. Based on the set of motion trajectories of the target fitness movement, the safe zone corresponding to the target fitness movement is determined; the target fitness movement is any fitness movement included in the video tracking log, and the region of interest of the fitness video includes the safe zones corresponding to multiple fitness movements in the fitness video. The display size of the region of interest is adjusted to determine the target display size of the region of interest in the window to be switched. The target video is displayed in the window to be switched according to the target display size.

Images