Video picture control method and device, equipment and storage medium

Abstract

The invention relates to the technical field of intelligent equipment, and discloses a video picture control method and device, equipment and a storage medium. The method comprises the following steps: intercepting a current processing canvas comprising all faces from a current input picture of a collected video; determining a current zoom value of the current processing canvas according to the maximum face width in all the faces, and determining a corresponding current offset according to a current canvas zoom center point corresponding to the current processing canvas and a device screen center point; and according to the current scaling value and the current offset, carrying out corresponding scaling and moving on the current processing canvas, and presenting the current processing canvas on the equipment screen. Therefore, the face is displayed at the central position on the equipment screen in the video application, a camera does not need to be adjusted, a complex tracking algorithm is not needed either, the hardware cost is saved, and resources are also saved.

Description

Technical Field

[0001] This application relates to the field of smart device technology, such as methods, apparatus, devices, and storage media for video screen control. Background Technology

[0002] With the rapid development of the national economy and the advancement of smart technology, home appliances such as televisions, tablets, and projectors have become smart devices for video playback. Furthermore, with the development of the Internet era, these smart video playback devices have functions such as video calls, remote conferencing, and distance education.

[0003] Currently, most video playback devices and related apps rely on cameras for functionality. Most camera tracking features utilize deep learning algorithms (such as YOLO and SSD) for object detection, identifying objects (like faces and bodies) in real-time. After detection, tracking algorithms (such as CamShift, KCF, and DeepSORT) continuously track the target. However, these methods have high hardware requirements, and the algorithms themselves are also quite expensive.

[0004] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this application, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] To provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. This summary is not intended as a general commentary, nor is it intended to identify key / important components or describe the scope of protection of these embodiments, but rather as a prelude to the detailed description that follows.

[0006] This disclosure provides a method, apparatus, device, and storage medium for video screen control, to address the technical problem of high cost in displaying the center position of a face in video applications.

[0007] In some embodiments, the method includes:

[0008] From the current input preview screen of the captured video, determine the current processing canvas that includes all faces;

[0009] Based on the maximum face width among all faces, determine the current scaling value of the current processing canvas, and based on the current canvas scaling center point and the device screen center point, determine the corresponding current offset.

[0010] Based on the current zoom value and the current offset, the current processing canvas is scaled and moved accordingly, and then displayed on the device screen.

[0011] In some embodiments, determining the current processing canvas, which includes all faces, includes:

[0012] Perform face detection on the current input preview screen;

[0013] When a face is detected, the image corresponding to the rectangle containing all faces is determined as the current processing canvas, where the aspect ratio of the rectangle is consistent with the aspect ratio of the device screen.

[0014] In some embodiments, determining the current zoom value of the current processing canvas includes:

[0015] Determine the current straight-line distance between the current face center point and the previous face center point for all faces in the current processing canvas;

[0016] If the current straight-line distance is greater than the set distance value, the current magnification value corresponding to the maximum face width is determined based on the correspondence between the saved face width range value and the magnification value, and is determined as the current scaling value.

[0017] In some embodiments, determining the corresponding current offset includes:

[0018] If the current face center point is located within a defined area in the current processing canvas, the current face center point is determined as the center point to be calculated.

[0019] If the center point of the current face is not located in the set area of ​​the current processing canvas, the center point of the current processing canvas will be determined as the center point to be calculated.

[0020] Based on the current scaling value, perform corresponding scaling calculations on the center point to be calculated to obtain the current canvas scaling center point.

[0021] In some embodiments, scaling and moving the current processing canvas accordingly includes:

[0022] Based on the current scaling value, determine the position of the canvas border reference point after the corresponding scaling calculation of the current processing canvas;

[0023] Based on the positions of the canvas border reference points and the device screen border reference points, the corresponding allowable offsets are obtained.

[0024] Based on the current offset and the allowed offset, determine the offset to be made without missing the black border of the screen, and based on the offset to be made and the current scaling value, move and scale the current processing canvas accordingly.

[0025] In some embodiments, the corresponding movement and scaling of the current processing canvas includes:

[0026] If the canvas exceeds the device screen after scaling calculation, based on the reference point positions of the canvas border and the device screen border, the canvas to be processed is moved according to the offset amount, and then scaled according to the current scaling value.

[0027] If the canvas does not exceed the device screen after scaling calculation, during the scaling process of the current processing canvas according to the current scaling value, the scaling canvas is moved according to the offset to be offset.

[0028] In some embodiments, after scaling and moving the current processing canvas accordingly, the method further includes:

[0029] Save the current face center point as the previous face center point.

[0030] In some embodiments, the apparatus for video screen control includes a processor and a memory storing program instructions, the processor being configured to execute the above-described method for video screen control when executing the program instructions.

[0031] In some embodiments, the device includes a device body; the aforementioned means for controlling video images is installed on the device body.

[0032] In some embodiments, the storage medium stores program instructions that, when executed, perform the above-described method for controlling video feeds.

[0033] The method, apparatus, and device for video screen control provided in this disclosure can achieve the following technical effects:

[0034] When previewing the captured video, if it is determined that the current processing canvas, which includes all faces, needs to be scaled, the current scaling value of the current processing canvas can be determined based on the maximum face width among all faces. Then, based on the scaling center point of the current canvas and the center point of the device screen, the corresponding current offset can be determined. Finally, based on the current scaling value and the current offset, the current processing canvas is scaled and moved accordingly and displayed on the device screen. In this way, by scaling and moving the canvas including all faces, faces in the video application can be displayed in the center of the device screen without adjusting the camera or using complex tracking algorithms, saving hardware costs and resources, and improving the intelligence of the video device and the user experience.

[0035] The above general description and the description below are exemplary and illustrative only and are not intended to limit this application. Attached Figure Description

[0036] One or more embodiments are illustrated by way of example with reference to the accompanying drawings. These illustrations and drawings do not constitute a limitation on the embodiments. Elements having the same reference numerals in the drawings are shown as similar elements. The drawings are not to be scaled. And wherein:

[0037] Figure 1 This is a flowchart illustrating a video screen control method provided in an embodiment of this disclosure;

[0038] Figure 2 This is a schematic diagram of a video frame pixel coordinate system provided in an embodiment of this disclosure;

[0039] Figure 3 This is a schematic diagram of a video frame pixel coordinate system provided in an embodiment of this disclosure;

[0040] Figure 4 This is a flowchart illustrating a video screen control method provided in an embodiment of this disclosure;

[0041] Figure 5 This is a schematic diagram of a video screen control device provided in an embodiment of the present disclosure;

[0042] Figure 6 This is a schematic diagram of a video screen control device provided in an embodiment of the present disclosure;

[0043] Figure 7 This is a schematic diagram of a video screen control device provided in an embodiment of the present disclosure;

[0044] Figure 8 This is a schematic diagram of a device provided in an embodiment of this disclosure. Detailed Implementation

[0045] To provide a more detailed understanding of the features and technical content of the embodiments of this disclosure, the implementation of the embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. The accompanying drawings are for illustrative purposes only and are not intended to limit the embodiments of this disclosure. In the following technical description, for ease of explanation, several details are used to provide a full understanding of the disclosed embodiments. However, one or more embodiments may still be implemented without these details. In other cases, well-known structures and devices may be simplified in their depiction to simplify the drawings.

[0046] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0047] Unless otherwise stated, the term "multiple" means two or more.

[0048] In this embodiment of the disclosure, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0049] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0050] Smart devices, such as televisions, tablets, and projectors, all have functions such as video calls, remote conferencing, and remote education. In this embodiment, when a smart device acquires video data for playback, it previews the captured video and determines a current processing canvas including all faces in the current input preview screen. When it is determined that the current processing canvas needs to be scaled, the current scaling value of the current processing canvas can be determined based on the maximum face width among all faces. The corresponding current offset is determined based on the scaling center point of the current canvas and the center point of the device screen. Then, based on the current scaling value and the current offset, the current processing canvas is scaled and moved accordingly and displayed on the device screen. In this way, by scaling and moving the canvas including all faces, the faces in the video application are displayed in the center position on the device screen, achieving face center-position tracking. This eliminates the need to adjust the camera or use complex tracking algorithms, saving hardware costs and resources, and improving the intelligence of the video device and the user experience.

[0051] Figure 1 This is a flowchart illustrating a video screen control method provided in an embodiment of this disclosure. Figure 1 As shown, the video display control process includes:

[0052] Step 101: Determine the current processing canvas, which includes all faces, from the current input preview screen of the captured video.

[0053] Devices capable of video playback, such as televisions, tablets, and projectors, acquire video data via cameras or cloud servers and display it on their screens. The displayed image can then be previewed. The device's input preview screen is the screen that captures the video at the current moment. From this current input preview screen, a processing canvas including all faces can be determined.

[0054] In some embodiments, determining the current processing canvas including all faces includes: performing face detection on the current input preview screen; and, if a face is detected, determining the screen corresponding to the rectangle including all faces as the current processing canvas, wherein the aspect ratio of the rectangle is consistent with the aspect ratio of the device screen.

[0055] The size of the rectangle can be the same as the device screen size, or it can be the smallest rectangle that includes all faces, with the aspect ratio of the rectangle matching that of the device screen. Of course, the current processing canvas size can also be other forms, such as square, circle, etc., or the aspect ratio of the rectangle can be configured based on the device screen, the area of ​​the region, the user's location, etc.

[0056] Step 102: Determine the current scaling value of the current processing canvas based on the maximum face width among all faces, and determine the corresponding current offset based on the current canvas scaling center point and the device screen center point.

[0057] Video applications on the device, such as video calls, remote conferencing, and remote education, require the face to be displayed in the center of the device screen. Therefore, in some implementations, the current input preview screen can be directly scaled and moved before being displayed.

[0058] Alternatively, since the device presents images sequentially in chronological order, some implementations require determining whether the center points of all faces in the processing canvas have changed position. If the change is significant, the processing canvas needs to be adjusted by scaling, moving, or other methods before presentation. Therefore, a significant positional change in the center points of all faces in the processing canvas can be determined if the current straight-line distance between the current center point and the previous center point is greater than a set distance value. Of course, at the time the video application starts, the previous center point can be the device screen center point or other preset positions. Therefore, determining the current scaling value of the current processing canvas includes: determining the current straight-line distance between the current center point and the previous center point; if the current straight-line distance is greater than a set distance value, determining the current magnification value corresponding to the maximum face width based on the saved correspondence between face width range values ​​and magnification values, and setting this as the current scaling value.

[0059] In this embodiment of the disclosure, the coordinate values ​​of the current processing canvas and each pixel on the device screen can be determined through the pixel coordinate system, and then the distance between the points can be obtained through the coordinate values. Figure 2 This is a schematic diagram of a video frame pixel coordinate system provided in an embodiment of this disclosure. Figure 2 The origin (0,0) of the coordinate system can be set to the top left corner of the device screen, with the positive x-axis pointing to the right and the positive y-axis pointing downwards. If the screen contains 1920*1080 pixels, then the coordinates (Sx, Sy) of the center point S of the device screen can be (1920 / 2, 1080 / 2). Figure 2 In the diagram, the dashed box corresponds to the current processing canvas, where point C can be the center point of the current face.

[0060] Because the preview image is mirrored, when performing face detection on the current input preview image, the detected face coordinates are oriented left to right. Therefore, by flipping the x-axis coordinates, the position coordinates of each pixel on each face in the current input preview image can be obtained. Specifically, the position coordinates of features such as the eyebrows, nose, or mouth can be used to determine the position coordinates of that face. Alternatively, the position coordinates of each facial feature can be calculated to obtain the position coordinates of each face. After obtaining the position coordinates of each face, the center point coordinates of all faces can be obtained, thus determining the center point of the current face.

[0061] In this way, the current straight-line distance between the current face center point and the previous face center point can be obtained by running the algorithm based on the coordinates of the current face center point and the saved coordinates of the current face center point.

[0062] Of course, the pixel coordinate system in this embodiment can also be other, such as: the origin is at the lower right boundary of the device screen, the leftward direction is the positive X-axis, and the upward direction is the positive Y-axis. Specific examples will not be listed one by one.

[0063] Once the pixel coordinate system is established, it can be used as a reference to obtain the position coordinates of a set pixel in the current input preview screen. Examples include the position coordinates of the current face center point, the device screen center point, and the center point of the current processing canvas. Furthermore, based on these position coordinates, the distances between set points can be obtained, such as the current straight-line distance between the current face center point and the previous face center point.

[0064] The set distance value can be determined based on the device screen size and performance, such as 80px, 100px, 120px, etc. If the current straight-line distance exceeds the set distance value, it indicates a significant change in the position of the face's center point. In this case, the processing canvas needs to be adjusted by scaling or moving. During the adjustment process, the scaling value of the processing canvas can be determined first. Here, the current scaling value of the processing canvas can be determined based on the maximum face width among all faces.

[0065] Video capture is typically done via a camera. Therefore, the relationship between the face width range and magnification can be determined based on the camera's performance and the size of the area where the device is located. For example, after multiple verifications, a maximum magnification of 1.5x is reasonable for a 2K camera. If it exceeds 1.5x, the image clarity will decrease, and the face will be magnified too much, appearing out of place in the current scene. Furthermore, based on the width of most users' living rooms (4-5 meters) (3.6 meters is suitable), it is determined that when a person is within 1.5 meters of the camera, no scaling is needed; above 3.5 meters, a 1.5x magnification is applied. At a distance of 1.5 meters, the face width is 120px, and at 3.5 meters, the face width is 70px. Therefore, the relationship between the saved face width range and magnification is shown in Table 1.

[0066]

[0067]

[0068] Table 1

[0069] Based on the identified face coordinates, the maximum face width among all faces can be determined, and according to Table 1, the current magnification value corresponding to the maximum face width can be determined. This current magnification value is then set as the current scaling value. For example, if the identified maximum face width is 88px, then according to Table 1, the current scaling value can be determined as a magnification ratio of 1.3. Therefore, based on the saved correspondence between face width range values ​​and magnification values, the current magnification value corresponding to the maximum face width can be determined; this current magnification value is then set as the current scaling value.

[0070] During the adjustment of the current processing canvas, after determining the scaling value of the processing canvas, it is also necessary to determine the current offset corresponding to moving the current processing canvas, that is, moving the current canvas scaling center point corresponding to the current processing canvas to the center point of the device screen. In other words, after determining the current scaling value, the current canvas scaling center point corresponding to the current processing canvas can be determined based on the current scaling center value and the device screen center point, and the corresponding current offset can be determined based on the current canvas scaling center value and the device screen center point.

[0071] In some embodiments, the current canvas scaling center point can be obtained directly based on the center point of the current processing canvas and the current scaling value. Alternatively, the current canvas scaling center point can be obtained directly based on the current face center point and the current scaling value. Alternatively, determining the corresponding current offset includes: determining the current canvas scaling center value corresponding to the current processing canvas, specifically including: if the current face center point is located within a set area in the current processing canvas, determining the current face center point as the center point to be calculated; if the current face center point is not located within a set area in the current processing canvas, determining the center point of the current processing canvas as the center point to be calculated; and performing corresponding scaling calculations on the center point to be calculated based on the current scaling value to obtain the current canvas scaling center point.

[0072] For example, the current canvas can be divided into four equal parts in length and three equal parts in height. If the distance between the center point of the current face and the left edge is less than 1 / 4 of the length, or the distance between the center point of the current face and the right edge is greater than 3 / 4 of the length, or the distance between the center point of the current face and the top edge is less than 1 / 3 of the height, or the distance between the center point of the current face and the bottom edge is greater than 2 / 3 of the height, then it can be determined that the center point of the current face is not located within the defined area of ​​the current processing canvas. Conversely, it can be determined that the center point of the current face is located within the defined area of ​​the current processing canvas. When the center point of the current face is not located within the defined area of ​​the current processing canvas, the center point of the current processing canvas is determined as the center point to be calculated. When the center point of the current face is located within the defined area of ​​the current processing canvas, the center point of the current face is determined as the center point to be calculated. Then, based on the current scaling value, the center point to be calculated is subjected to corresponding scaling calculations to obtain the scaled center point of the current canvas.

[0073] It is possible to do as Figure 2 In the coordinate system shown, the position coordinates (fx, fy) of the center point to be calculated are determined. If the current scaling value is the magnification value c, then the position coordinates (cfx, cfy) of the current canvas scaling center point are obtained. Then, based on the current canvas scaling center value and the device screen center point, the corresponding current offset can be determined. This yields the first offset dx in the x-axis direction and the second offset dy in the y-axis direction. Similarly, based on the current canvas scaling center point position coordinates (cfx, cfy) and the device screen center point position coordinates (Sx, Sy), the first offset dx in the x-axis direction and the second offset dy in the y-axis direction can be obtained.

[0074] Step 103: Based on the current zoom value and the current offset, scale and move the current processing canvas accordingly, and display it on the device screen.

[0075] Determine the current zoom value and the current offset. Based on these values, the current processing canvas can be scaled and moved accordingly. Specifically, the current processing canvas can be scaled first according to the current zoom value, and then moved according to the current offset. Alternatively, the current processing canvas can be moved first according to the current offset, and then scaled according to the current zoom value. Or, the current processing canvas can be scaled simultaneously according to the current zoom value, and the scaled processing canvas can be moved according to the current offset.

[0076] The boundaries of the processed canvas after scaling and moving may be within the device screen, which may result in black borders appearing on the screen. Therefore, in some embodiments, scaling and moving the current processed canvas includes: determining the canvas border reference point position after scaling calculation based on the current scaling value; obtaining the corresponding allowable offset based on the canvas border reference point position and the device screen border reference point position; determining the offset to be determined without missing black borders based on the current offset and the allowable offset; and moving and scaling the current processed canvas accordingly based on the offset to be determined and the current scaling value.

[0077] Figure 3 This is a schematic diagram of a video frame pixel coordinate system provided in an embodiment of this disclosure. Figure 3 As shown, with the top left corner of the device screen as the origin (0, 0), and the rightward direction as the positive x-axis and the downward direction as the positive y-axis, if the screen contains 1920*1080 pixels, then the coordinates of the center point S of the device screen (Sx, Sy) can be (1920 / 2, 1080 / 2). Figure 3 In the diagram, the dashed box corresponds to the current processing canvas, while the bold dashed box corresponds to the schematic box of the canvas after scaling calculation. Here, D is the scaling center point of the current canvas after scaling calculation, and S is the center point of the device screen. This yields the first offset dx and the second offset dy, as shown in the diagram. Since the canvas needs to be moved to the right and upwards respectively, the upper right corner of the canvas border after scaling calculation can be determined as the reference point position of the canvas border. Therefore, the x-axis distance and y-axis distance between the upper right corner of the canvas border and the upper right corner of the device screen border can be obtained, which can be respectively the first allowed offset rx and the second allowed offset ry. The larger value between the first offset dx and the first allowed offset rx is determined as the first offset to be determined without revealing the black border of the screen. The larger value between the second offset dy and the second allowed offset ry is determined as the second offset to be determined without revealing the black border of the screen. Therefore, based on the offsets to be determined and the current scaling value, the current processing canvas can be moved and scaled accordingly.

[0078] Of course, in some implementations, the first offset dx and the second offset dy, as shown in the figure, are obtained. Since the canvas needs to be moved to the right and upward respectively, the upper left corner of the canvas border after scaling calculation can be determined as the reference point position of the canvas border. Thus, the x-axis distance and y-axis distance between the upper left corner of the canvas border and the upper left corner of the device screen border can be obtained, which can be the first allowable offset r'x and the second allowable offset r'y, respectively. Therefore, the smaller value between the first offset dx and the first allowable offset r'x can be determined as the first offset to be determined without missing the black border of the screen, and the smaller value between the second offset dy and the second allowable offset r'y can be determined as the second offset to be determined without missing the black border of the screen. Thus, the canvas being processed can be moved and scaled accordingly based on the offset to be determined and the current scaling value.

[0079] As can be seen, there are multiple ways to determine the offset without missing the black border of the screen, based on the current offset and the allowed offset. These will not be listed one by one.

[0080] In some embodiments, moving and scaling the current processing canvas includes: if it is determined that the canvas exceeds the device screen after scaling calculation based on the canvas border reference point position and the device screen border reference point position, moving the current processing canvas according to the offset amount, and then scaling the current processing canvas according to the current scaling value; if it is determined that the canvas does not exceed the device screen after scaling calculation, moving the scaling canvas according to the offset amount during the scaling process of the current processing canvas according to the current scaling value.

[0081] Of course, assuming the canvas does not exceed the device screen after scaling calculation, movement can be performed during the scaling process. After each scaling step, a new offset can be obtained without missing the black borders on the screen, and then the movement can be performed accordingly. Furthermore, if no face is detected for a duration exceeding a certain threshold in the currently scaled canvas, the scaled canvas can be restored to match the device screen. This threshold can be 10 seconds, 12 seconds, 15 seconds, etc.

[0082] As can be seen, in this embodiment, when the smart device acquires video data for playback, it previews the captured video and determines the current processing canvas, which includes all faces, within the current input preview screen. When it is determined that the current processing canvas needs to be scaled, the current scaling value of the current processing canvas can be determined based on the maximum face width among all faces. The corresponding current offset is determined based on the scaling center point of the current canvas and the center point of the device screen. Then, based on the current scaling value and the current offset, the current processing canvas is scaled and moved accordingly and displayed on the device screen. In this way, by scaling and moving the canvas including all faces, the faces in the video application are displayed in the center position on the device screen, achieving face center-position tracking. This eliminates the need to adjust the camera or use complex tracking algorithms, saving hardware costs and resources, and improving the intelligence and user experience of the video device. Furthermore, it ensures that no black borders are missed during the movement, further enhancing the intelligence and user experience of the video device.

[0083] Of course, in some embodiments, after scaling and moving the current processing canvas, the method further includes saving the current face center point as the previous face center point. This ensures the continuity of the display of the center position of the face in the video frame, further improving the intelligence of the device and the user experience.

[0084] The following describes the operation process in a specific embodiment, illustrating the video screen control process provided by the embodiments of the present invention.

[0085] In one embodiment of this disclosure, the smart device capable of video playback can be a television, and the television is equipped with a camera. The camera can capture video within a functional area, and the video can be previewed and displayed on the television screen. The television maintains the correspondence shown in Table 1, and the pixel coordinate system corresponding to the preview image can be as follows: Figure 2 As shown, the distance can be set to 100px. The TV can control the corresponding position in the preview screen to display the center of the face.

[0086] Figure 4 This is a flowchart illustrating a video screen control method provided in an embodiment of this disclosure. Figure 4 As shown, the video display control process includes:

[0087] Step 401: The TV performs facial recognition on the current input preview screen of the captured video.

[0088] Step 402: Determine if the current input preview screen includes a face? If yes, proceed to step 403; otherwise, the current input preview screen does not need to be controlled and can be directly displayed on the TV screen, thus ending this control process.

[0089] Step 403: The TV determines the image corresponding to the rectangle containing all faces as the current processing canvas, wherein the size of the rectangle is the same as the size of the TV display screen.

[0090] Step 404: The TV determines the current face center point corresponding to all faces in the current processing canvas, and obtains the current straight-line distance between the current face center point and the previous face center point.

[0091] Television can be based on Figure 2 The pixel coordinates shown on the left can be used to obtain the coordinates of each pixel on each face after mirroring, thus obtaining the coordinates of the current face center point for all faces. Then, based on the retained coordinates of the previous face center point, the current straight-line distance between the current face center point and the previous face center point can be obtained.

[0092] Of course, when the TV video application is launched, the center point of the face corresponding to the first time the video screen is controlled can be the center point of the TV screen.

[0093] Step 405: Determine if the current straight-line distance is greater than 100px. If yes, proceed to step 406; otherwise, the current input preview screen does not need to be controlled and can be directly displayed on the TV screen, thus ending the current control process.

[0094] Step 406: The TV determines the maximum face width from all faces, and determines the current magnification value corresponding to the maximum face width according to the correspondence shown in Table 1, and sets it as the current scaling value.

[0095] Similarly, television can be based on Figure 2 The left side of the pixel system shown can be used to obtain the coordinate values ​​of each pixel on each face after mirroring, thereby obtaining the width of each face, and then obtaining the maximum face width. According to Table 1, the corresponding current magnification value can be determined and set as the current scaling value.

[0096] Step 407: Determine whether the center point of the current face is located within the set area of ​​the current processing canvas. If yes, proceed to step 408; otherwise, proceed to step 409.

[0097] Step 408: The TV determines the current center point of the face as the center point to be calculated, and performs corresponding scaling calculations on the center point to be calculated based on the current scaling value to obtain the current canvas scaling center point, and then proceeds to step 410.

[0098] Step 409: The TV determines the center point of the current processing canvas as the center point to be calculated, and performs corresponding scaling calculations on the center point to be calculated based on the current scaling value to obtain the current canvas scaling center point, and then proceeds to step 410.

[0099] Step 410: The TV determines the corresponding current offset based on the current canvas scaling center value and the device screen center point.

[0100] Step 411: The TV determines the position of the canvas border reference point after the corresponding scaling calculation of the current processing canvas based on the current scaling value, and obtains the corresponding allowable offset based on the position of the canvas border reference point and the position of the device screen border reference point.

[0101] Step 412: The TV determines the offset to be made without missing the black border of the screen based on the current offset and the allowed offset.

[0102] Step 413: Determine whether the canvas exceeds the TV screen after scaling calculation. If yes, proceed to step 414; otherwise, proceed to step 415.

[0103] Step 414: The TV moves the current processing canvas according to the offset amount, scales the current processing canvas according to the current scaling value, and saves the current face center point as the previous face center point.

[0104] Step 415: During the scaling process of the current processing canvas based on the current scaling value, the TV moves the scaling canvas according to the offset to be offset, and saves the current face center point as the previous face center point.

[0105] During the simultaneous scaling and movement, it's crucial to ensure no black borders are visible on the screen. Therefore, after each scaling step, a new offset value is obtained to ensure no black borders are missed. Then, the movement is performed based on this new offset value. Furthermore, if no face is detected for more than 15 seconds in the currently scaled canvas, the scaled canvas can be restored to match the device screen.

[0106] As can be seen, in this embodiment, when the TV acquires video data for playback, it previews the captured video and determines the current processing canvas, which includes all faces, within the current input preview screen. When it is determined that the current processing canvas needs to be scaled, the current scaling value of the current processing canvas can be determined based on the maximum face width among all faces. The corresponding current offset is determined based on the scaling center point of the current canvas and the center point of the TV screen. Then, based on the current scaling value and the current offset, the current processing canvas is scaled and moved accordingly and displayed on the TV screen. In this way, by scaling and moving the canvas including all faces, the faces in the video application are displayed in the center position on the TV screen, achieving face center-position tracking. This eliminates the need to adjust the camera or use complex tracking algorithms, saving hardware costs and resources, and improving the TV's intelligence and user experience. Furthermore, it ensures that no black borders are missed during the movement, further enhancing the TV's intelligence and user experience.

[0107] Based on the above process for controlling video images, a device for controlling video images can be constructed and applied to video playback devices.

[0108] Figure 5 This is a schematic diagram of a video screen control device provided in an embodiment of this disclosure. Figure 5 As shown, the video screen control device 500 includes: a first determining module 510, a second determining module 520, and a zoom and move module 530.

[0109] The first determining module 510 is configured to determine the current processing canvas, which includes all faces, from the current input preview screen of the captured video.

[0110] The second determining module 520 is configured to determine the current scaling value of the current processing canvas based on the maximum face width among all faces, and to determine the corresponding current offset based on the current canvas scaling center point and the device screen center point.

[0111] The scaling and moving module 530 is configured to scale and move the current processing canvas according to the current scaling value and the current offset, and display it on the device screen.

[0112] In some embodiments, the first determining module 510 is specifically configured to perform face detection on the current input preview screen; if a face is detected, the screen corresponding to the rectangle containing all faces is determined as the current processing canvas, wherein the aspect ratio of the rectangle is consistent with the aspect ratio of the device screen.

[0113] In some embodiments, the second determining module 520 includes:

[0114] The first determining unit is configured to determine the current straight-line distance between the current face center point and the previous face center point for all faces in the current processing canvas; if the current straight-line distance is greater than a set distance value, the current magnification value corresponding to the maximum face width is determined according to the correspondence between the saved face width range value and the magnification value, and is determined as the current scaling value.

[0115] In some embodiments, the second determining module 530 includes:

[0116] The second determining unit is configured to determine the current face center point as the center point to be calculated when the current face center point is located in a set area in the current processing canvas; and to determine the center point of the current processing canvas as the center point to be calculated when the current face center point is not located in a set area in the current processing canvas, and to perform corresponding scaling calculations on the center point to be calculated to obtain the current canvas scaling center point.

[0117] In some embodiments, the scaling and moving module 530 includes:

[0118] The offset determination unit is configured to determine the canvas border reference point position after the current scaling calculation is performed on the current processing canvas based on the current scaling value; obtain the corresponding allowable offset based on the canvas border reference point position and the device screen border reference point position; and determine the offset to be performed without missing the black border of the screen based on the current offset and the allowable offset.

[0119] The scaling and moving unit is configured to move and scale the current processing canvas according to the offset to be offset and the current scaling value.

[0120] In some embodiments, the scaling and moving unit is specifically configured to, when determining that the canvas exceeds the device screen after scaling calculation based on the canvas border reference point position and the device screen border reference point position, move the currently processed canvas according to the offset amount, and then scale the currently processed canvas according to the current scaling value; when determining that the canvas does not exceed the device screen after scaling calculation, during the scaling process of the currently processed canvas according to the current scaling value, move the canvas to be scaled according to the offset amount.

[0121] In some embodiments, it also includes:

[0122] The save module is configured to save the current face center point as the previous face center point.

[0123] The video screen control process for the video screen control device is further described below with reference to embodiments.

[0124] In this embodiment, the smart device can be a television, i.e., the television launches a video app, such as a video call, remote conferencing, or distance education. The television stores the correspondence shown in Table 1, and the distance can be set to 80px.

[0125] Figure 6 This is a schematic diagram of a video screen control device provided in an embodiment of this disclosure. Figure 6 As shown, the video screen control device 500 includes: a first determining module 510, a second determining module 520, a scaling and moving module 530, and a saving module 540. The second determining module 520 includes: a first determining unit 521 and a second determining unit 522, while the scaling and moving module 530 includes: an offset determining unit 531 and a scaling and moving unit 532.

[0126] In this embodiment, after the TV starts the corresponding video application, it can capture video within the effective area through the camera, or acquire video through communication. After previewing, the first determining module 510 can perform face recognition on the current input preview screen of the captured video, and if the current input preview screen includes faces, determine the screen corresponding to the smallest rectangle containing all faces as the current processing canvas, wherein the aspect ratio of the rectangle is consistent with the aspect ratio of the TV display screen.

[0127] The first determining unit 521 of the second determining module 520 can determine the current face center point corresponding to all faces in the current processing canvas, obtain the current straight-line distance between the current face center point and the previous face center point, and when the current straight-line distance is greater than 80px, determine the maximum face width from all faces, and determine the current magnification value corresponding to the maximum face width according to the correspondence shown in Table 1, and determine it as the current scaling value.

[0128] Thus, when the current face center point is located within a set area in the current processing canvas, the second determining unit 522 in the second determining module 520 can determine the current face center point as the center point to be calculated; conversely, if the current face center point is not located within the set area in the current processing canvas, the second determining unit 520 can determine the center point of the current processing canvas as the center point to be calculated. Furthermore, the second determining unit 522 can perform corresponding scaling calculations on the center point to be calculated based on the current scaling value to obtain the current canvas scaling center point, and then determine the corresponding current offset based on the current canvas scaling center value and the device screen center point.

[0129] The offset determination unit 531 in the scaling and moving module 530 can determine the position of the canvas border reference point after the current scaling calculation is performed on the current processing canvas according to the current scaling value, and obtain the corresponding allowable offset according to the position of the canvas border reference point and the position of the device screen border reference point, and determine the offset to be determined without missing the black border of the screen according to the current offset and the allowable offset.

[0130] If the canvas exceeds the TV screen after scaling calculation, the scaling and moving unit 532 in the scaling and moving module 530 can move the currently processed canvas according to the offset amount, and then scale the currently processed canvas according to the current scaling value. If the canvas does not exceed the TV screen after scaling calculation, the scaling and moving unit 532 can move the scaled canvas according to the offset amount while scaling the currently processed canvas according to the current scaling value. Furthermore, when scaling and moving are performed simultaneously, the scaling and moving unit 532 can obtain a new offset amount without missing the black border of the screen after each scaling, and then perform the corresponding movement according to the new offset amount. Moreover, if no face is detected for more than 15 seconds in the currently scaled canvas, the scaled canvas can be restored to be consistent with the device screen.

[0131] Of course, the saving module 540 also needs to save the current face center point as the previous face center point.

[0132] As can be seen, in this embodiment, when the television acquires video data for playback, after previewing the captured video, the device for video screen control can determine the current processing canvas, including all faces, in the current input preview screen. When it is determined that the current processing canvas needs to be scaled, the current scaling value of the current processing canvas can be determined based on the maximum face width among all faces. The corresponding current offset can be determined based on the scaling center point of the current canvas and the center point of the television screen. Then, based on the current scaling value and the current offset, the current processing canvas is scaled and moved accordingly and displayed on the television screen. In this way, by scaling and moving the canvas including all faces, the faces in the video application are displayed in the center position on the television screen, achieving face center-position tracking. This eliminates the need to adjust the camera or use complex tracking algorithms, saving hardware costs and resources, and improving the intelligence and user experience of the television. Furthermore, it ensures that no black borders are missed during the movement, further enhancing the intelligence and user experience of the television.

[0133] Combination Figure 7 This disclosure provides an apparatus 700 for video screen control, comprising:

[0134] The processor 1000 and memory 1001 may further include a communication interface 1002 and a bus 1003. The processor 1000, communication interface 1002, and memory 1001 can communicate with each other via the bus 1003. The communication interface 1002 can be used for information transmission. The processor 1000 can call logical instructions stored in the memory 1001 to execute the video screen control method described in the above embodiment.

[0135] Furthermore, the logic instructions in the aforementioned memory 1001 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium.

[0136] The memory 1001, as a computer-readable storage medium, can be used to store software programs and computer-executable programs, such as program instructions / modules corresponding to the methods in the embodiments of this disclosure. The processor 1000 executes functional applications and data processing by running the program instructions / modules stored in the memory 1001, that is, it implements the method for video screen control in the above method embodiments.

[0137] The memory 1001 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the terminal device. Furthermore, the memory 1001 may include high-speed random access memory and may also include non-volatile memory.

[0138] This disclosure provides a video screen control device, including: a processor and a memory storing program instructions, wherein the processor is configured to execute a video screen control method when executing the program instructions.

[0139] Combination Figure 8 This disclosure provides a device 800, which can be a video playback device, such as a television, tablet computer, or projector, etc., including: a device body, and the aforementioned video screen control device 500 (700). The video screen control device 500 (700) is installed on the device body. The installation relationship described herein is not limited to placement inside the product, but also includes installation connections with other components of the product, including but not limited to physical connections, electrical connections, or signal transmission connections. Those skilled in the art will understand that the video screen control device 500 (700) can be adapted to feasible device bodies to achieve other feasible embodiments.

[0140] This disclosure provides a storage medium storing program instructions that, when executed, perform the method for video screen control as described above.

[0141] This disclosure provides a computer program product, which includes a computer program stored on a storage medium. The computer program includes program instructions, which, when executed by a computer, cause the computer to perform the above-described video screen control method.

[0142] The aforementioned storage medium can be a transient computer-readable storage medium or a non-transitory computer-readable storage medium.

[0143] The technical solutions of this disclosure can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes one or more instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in this disclosure. The aforementioned storage medium can be a non-transitory storage medium, including: a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and other media capable of storing program code; it can also be a transient storage medium.

[0144] The foregoing description and accompanying drawings fully illustrate embodiments of the present disclosure to enable those skilled in the art to practice them. Other embodiments may include structural, logical, electrical, procedural, and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included or replace parts and features of other embodiments. The scope of the embodiments of this disclosure includes the entire scope of the claims and all available equivalents of the claims. While the terms “first,” “second,” etc., may be used in this application to describe elements, these elements should not be limited by these terms. These terms are used only to distinguish one element from another. For example, a first element may be called a second element without changing the meaning of the description, and similarly, a second element may be called a first element, provided that all occurrences of “first element” are consistently renamed and all occurrences of “second element” are consistently renamed. First and second elements are both elements, but may not be the same element. Moreover, the terminology used in this application is only for describing embodiments and is not intended to limit the claims. As used in the description of the embodiments and claims, unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “the” are intended to also include the plural forms. Similarly, the term “and / or” as used herein means including one or more of the associated listed elements and all possible combinations thereof. Additionally, when used herein, the terms “comprise” and its variations “comprises” and / or “comprising” refer to the presence of stated features, integrals, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof. Without further limitations, an element defined by the phrase “comprising an…” does not exclude the presence of additional identical elements in the process, method, or apparatus that includes said element. In this document, each embodiment may focus on the differences from other embodiments, and similar or identical parts between embodiments can be referred to mutually. For methods, products, etc., disclosed in the embodiments, if they correspond to the method section disclosed in the embodiments, the relevant parts can be referred to the description of the method section.

[0145] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this disclosure. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0146] The methods and products (including but not limited to devices and equipment) disclosed in the embodiments herein can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of units may be merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed units may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to implement this embodiment according to actual needs. Furthermore, the functional units in the embodiments of this disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0147] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than that shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. In the descriptions corresponding to the flowcharts and block diagrams in the accompanying drawings, the operations or steps corresponding to different blocks may also occur in a different order than disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. Each block in a block diagram and / or flowchart, and combinations of blocks in a block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

Claims

1. A method for controlling video images, characterized in that, include: From the current input preview screen of the captured video, determine the current processing canvas that includes all faces; Based on the maximum face width among all faces, determine the current scaling value of the current processing canvas, and based on the current canvas scaling center point and the device screen center point, determine the corresponding current offset. Based on the current zoom value and the current offset, the current processing canvas is scaled and moved accordingly, and then displayed on the device screen.

2. The method according to claim 1, characterized in that, The determination of the current processing canvas, which includes all faces, includes: Perform face detection on the current input preview screen; When a face is detected, the image corresponding to the rectangle containing all faces is determined as the current processing canvas, where the aspect ratio of the rectangle is consistent with the aspect ratio of the device screen.

3. The method according to claim 1, characterized in that, Determining the current zoom value of the current processing canvas includes: Determine the current straight-line distance between the current face center point and the previous face center point for all faces in the current processing canvas; If the current straight-line distance is greater than the set distance value, the current magnification value corresponding to the maximum face width is determined based on the correspondence between the saved face width range value and the magnification value, and is determined as the current scaling value.

4. The method according to claim 1, characterized in that, Determining the corresponding current offset includes: If the current face center point is located within a defined area in the current processing canvas, the current face center point is determined as the center point to be calculated. If the center point of the current face is not located in the set area of ​​the current processing canvas, the center point of the current processing canvas will be determined as the center point to be calculated. Based on the current scaling value, perform corresponding scaling calculations on the center point to be calculated to obtain the current canvas scaling center point.

5. The method according to claim 1, characterized in that, The scaling and moving of the current processing canvas includes: Based on the current scaling value, determine the position of the canvas border reference point after the corresponding scaling calculation of the current processing canvas; Based on the positions of the canvas border reference points and the device screen border reference points, the corresponding allowable offsets are obtained. Based on the current offset and the allowed offset, determine the offset to be made without missing the black border of the screen, and based on the offset to be made and the current scaling value, move and scale the current processing canvas accordingly.

6. The method according to claim 5, characterized in that, The corresponding movement and scaling of the current processing canvas includes: If the canvas exceeds the device screen after scaling calculation, based on the reference point positions of the canvas border and the device screen border, the canvas to be processed is moved according to the offset amount, and then scaled according to the current scaling value. If the canvas does not exceed the device screen after scaling calculation, during the scaling process of the current processing canvas according to the current scaling value, the scaling canvas is moved according to the offset to be offset.

7. The method according to any one of claims 1-6, characterized in that, After scaling and moving the current processing canvas accordingly, the process also includes: Save the current face center point as the previous face center point.

8. A device for controlling video images, the device comprising a processor and a memory storing program instructions, characterized in that, The processor is configured to perform the method for video screen control as described in any one of claims 1 to 7 when executing the program instructions.

9. A device, characterized in that, include: Equipment body; The device for video screen control as described in claim 8 is installed on the device body.

10. A storage medium storing program instructions, characterized in that, When the program instructions are executed, they perform the method for video screen control as described in any one of claims 1 to 7.

Images