Character close-up method, device and equipment based on conference tablet and storage medium

Abstract

The embodiment of the invention relates to the technical field of display equipment, and discloses a character close-up method and device based on a conference tablet, equipment and a storage medium. The method comprises the steps of determining a target person area in a display picture, determining an interception frame corresponding to the target person area and movement information of the interception frame relative to the target person area, controlling the interception frame to move according to the movement information to obtain a moved interception frame, intercepting a close-up picture containing the target person area in the display picture according to the moved interception frame, and displaying the close-up picture in the display picture. And controlling the conference panel to display the close-up picture. According to the embodiment, it can be guaranteed that the target figure can be completely displayed in the interception frame when moving so as to present a complete close-up picture, and it is guaranteed that the interception frame cannot be blocked when moving along with the target figure.

Description

Technical Field

[0001] The present invention relates to the technical field of display devices, and in particular to a person close-up method, device, equipment and storage medium based on a conference tablet. Background Art

[0002] In a video conference, the conference tablet can take a close-up of the speaker and display the close-up image of the speaker separately. The related technology uses a fixed interval detection method to perform close-up processing on the speaker. At every time interval, the related technology re-detects the position of the speaker's face and recomposes the image based on the detected current position. Since the time interval is too long and the image update is too long, when the speaker moves too fast, the speaker is likely to walk out of the close-up image, resulting in the close-up image being unable to track the speaker. Summary of the invention

[0003] An object of an embodiment of the present invention is to provide a person close-up method, device, equipment and storage medium based on a conference tablet, which is used to solve the technical problem that related technologies cannot reliably track a target person and take a close-up of the target person.

[0004] In a first aspect, an embodiment of the present invention provides a method for taking a close-up of a person based on a conference tablet, comprising:

[0005] Get the display screen captured by the conference tablet;

[0006] Determine a target person area according to the display image;

[0007] Determine a capture frame corresponding to the target person area and movement information of the capture frame relative to the target person area;

[0008] Control the movement of the capture frame according to the movement information to obtain the moved capture frame;

[0009] intercepting a close-up picture including the target person area in the display picture according to the moved interception frame;

[0010] Control the conference tablet to display the close-up image.

[0011] In combination with the first aspect, in a possible implementation method, the movement information includes a target movement distance and a target movement direction, and determining the movement information of the capture frame relative to the target person area includes: respectively determining a reference point of the target person area and a focusing point of the capture frame; calculating the distance from the reference point to the focusing point to obtain the target movement distance; and determining the target movement direction based on the relative position of the reference point and the focusing point.

[0012] In combination with the first aspect, in a possible implementation method, determining the reference point of the target person area includes: generating a face recognition frame of the target person area; obtaining the midpoint of the first horizontal line of the face recognition frame and a first three-point line perpendicular to the first horizontal line; and determining the intersection of the midpoint of the first horizontal point and the first three-point point as the reference point.

[0013] In combination with the first aspect, in a possible implementation method, determining the focus point of the cropped frame includes: obtaining the midpoint of a second horizontal line of the cropped frame and a second third line perpendicular to the second horizontal line; and determining the intersection of the midpoint of the second horizontal point and the second third point as the focus point.

[0014] In combination with the first aspect, in a possible implementation manner, determining the target movement direction according to the relative position of the reference point and the focus point includes: determining the target movement direction by the straight line direction from the focus point to the reference point.

[0015] In combination with the first aspect, in a possible implementation method, the movement information includes a target moving distance and a target moving direction, and the movement of the capture frame is controlled according to the movement information to obtain the moved capture frame, including: calculating the moving speed according to the target moving distance and a preset speed coefficient; and controlling the capture frame to move in the target moving direction according to the moving speed to obtain the moved capture frame.

[0016] In combination with the first aspect, in a possible implementation manner, before controlling the capture frame to move, the method further includes: when a focus point of the capture frame is aligned with a reference point of the target person area, controlling the capture frame to stop moving.

[0017] In combination with the first aspect, in a possible implementation manner, when capturing a close-up image for the first time, a focus point of the capture frame is aligned with a reference point of the target person area.

[0018] In combination with the first aspect, in a possible implementation method, controlling the conference tablet to display the close-up picture includes: when it is detected that the close-up picture contains an irrelevant character area, eliminating the irrelevant character area to obtain a processed close-up picture; controlling the conference tablet to display the processed close-up picture.

[0019] In combination with the first aspect, in a possible implementation manner, determining the target person area according to the display screen includes: acquiring voice information collected by the conference tablet; and determining the target person in the display screen according to the voice information.

[0020] In a second aspect, an embodiment of the present invention provides a person close-up device based on a conference tablet, comprising:

[0021] An acquisition unit, used to acquire the display image captured by the conference tablet;

[0022] A determination unit, used to determine a target person area according to the display picture;

[0023] The determination unit is further used to determine a clipping frame corresponding to the target person area and movement information of the clipping frame relative to the target person area;

[0024] A control unit, used for controlling the movement of the capture frame according to the movement information to obtain the moved capture frame;

[0025] A capture unit, configured to capture a close-up image including the target person area in the display image according to the moved capture frame;

[0026] The control unit is also used to control the conference tablet to display the close-up picture.

[0027] In a third aspect, an embodiment of the present invention provides a conference tablet, including:

[0028] A camera, used to collect and display images;

[0029] Microphone array, used to collect voice information;

[0030] Memory;

[0031] A processor is respectively communicatively connected to the camera, the microphone array and the memory, wherein the processor is used to execute one or more computer programs stored in the memory, and when the processor executes the one or more computer programs, the conference tablet implements the method described in the first aspect.

[0032] In a fourth aspect, a computer device is provided, comprising a memory and one or more processors, wherein the memory is connected to the one or more processors, and the one or more processors are used to execute one or more computer programs stored in the memory, and when the one or more processors execute the one or more computer programs, the computer device implements the method described in the first aspect.

[0033] In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the method as described in the first aspect.

[0034] In the scheme implemented by the above-mentioned method, device, equipment and storage medium for taking close-up photos of people based on conference tablet, the display screen collected by the conference tablet is first obtained, and the target person area is determined according to the display screen. Secondly, the interception frame corresponding to the target person area and the movement information of the interception frame relative to the target person area are determined, and the interception frame is controlled to move according to the movement information to obtain the moved interception frame, and then, the close-up picture containing the target person area is intercepted in the display screen according to the moved interception frame, and finally, the conference tablet is controlled to display the close-up picture. In this embodiment, by moving the interception frame, the moved interception frame can track the target person area in time, and intercept the target person area from the display screen to obtain a close-up picture, so that the user corresponding to the target person area can be prevented from easily walking out of the close-up picture, so that the embodiment can reliably display the close-up picture containing the target person area. In addition, in this embodiment, the close-up picture is intercepted in a smooth manner by moving the interception frame. This method simulates the follow-up shooting method of the camera, so that the close-up picture output by this embodiment is smoother and more natural. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0036] Figure 1A This is a hardware architecture diagram of a conference tablet in one embodiment of the present invention;

[0037] Figure 1B This is a module diagram of a conference tablet in one embodiment of the present invention;

[0038] Figure 1C It is a scene diagram of a conference tablet in one embodiment of the present invention;

[0039] Figure 2 It is a flowchart of a method for taking a close-up of a person based on a conference tablet in one embodiment of the present invention;

[0040] Figure 3A This is a schematic diagram of a display screen of a conference tablet in one embodiment of the present invention;

[0041] Figure 3B is a schematic diagram of a cutout frame containing a target person area in one embodiment of the present invention;

[0042] Figure 3C is a schematic diagram of reference points of a target person area in one embodiment of the present invention;

[0043] Figure 3D is a schematic diagram of a focus point of a capture frame in one embodiment of the present invention;

[0044] Figure 4 It is a structural schematic diagram of a person close-up device based on a conference tablet in one embodiment of the present invention;

[0045] Figure 5 It is a schematic diagram of the structure of an electronic device in one embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical scheme and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0047] It should be noted that, if there is no conflict, the various features in the embodiments of the present invention can be combined with each other, all within the scope of protection of the present invention. In addition, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a different order from the module division in the device or the flow chart. Furthermore, the words "first", "second", "third", etc. used in the present invention do not limit the data and execution order, but only distinguish the same items or similar items with basically the same functions and effects.

[0048] The technical solution of this application can be applied to various conference scenarios, such as remote conferences, etc.

[0049] The technical solution of the present application can be specifically applied to conference tablets, wherein the conference tablet can be a tablet device with multifunctional imaging such as display, broadcasting, and recording, and can specifically be a TV screen, a display, and an electronic whiteboard, etc. For ease of understanding, the conference tablet of the present application is first introduced.

[0050] See also Figure 1A , Figure 1A A schematic diagram of the hardware structure of a conference tablet provided in an embodiment of the present application is shown in FIG. Figure 1AAs shown, the conference tablet 10 includes a display panel 101, a microphone array 102, a camera module 103, a processor 104 and a memory 105. The display panel 101 is used to display the video stream image captured by the conference tablet and the close-up image captured by the capture frame; the microphone array 102 is used to locate the sound source information in the scene and calculate the sound source orientation angle. The layout of the microphone array 102 can be a linear array, a planar array, or a stereo array; the camera module 103 includes at least one camera, which is mainly used to obtain scene images or video streams. It can be a fixed-focus camera, a zoom camera, or a combination of the two; the processor 104 and the memory 105 are devices for storing and processing various sensor signals, such as processing audio information collected by the microphone array 102, or processing video information collected by the camera.

[0051] Among them, Figure 1B As shown, the conference tablet 10 includes four major functional modules: an audio and video acquisition input module, which mainly acquires original audio and video information through a microphone array and a camera; a person position detection module: calculates the current position and rough distance of the person to be tracked based on the original audio and video information; a person display area calculation module: calculates the position and size of the current display area based on the calculated person position and distance; an image cropping module: crops and outputs the image based on the position and size of the display area.

[0052] like Figure 1C As shown, Figure 1C The screen displayed on the display panel of the conference tablet is shown in the figure. For example, in a remote meeting, the screen displayed on the conference tablet is as follows: Figure 1C As shown in A in A, the content in the dotted box in A can be the PPT content being shown or other meeting materials, etc. In A, area a is an independent display area for participants A, B, and C at the meeting, area B shows an enlarged view of area a, area C is a specific picture of person C obtained by cutting the person in area a when person C is selected as the target person to speak, and area D is the final display effect of presenting the specific picture of area C on the display screen of the conference tablet.

[0053] In view of this, the present application proposes a close-up method of a person based on a conference tablet to solve the above problems, which is described in detail below.

[0054] See also Figure 2 , Figure 2 A flowchart of a method for close-up of a person provided by an embodiment of the present invention is provided, and the method comprises the following steps:

[0055] S10: Obtain the display screen captured by the conference tablet.

[0056] Among them, the display picture is the picture collected by the camera module. The camera module can shoot the physical space where the conference tablet is located to obtain the display picture.

[0057] In some embodiments, the camera module is disposed in a conference tablet, and the conference tablet controls the camera module to capture and display images.

[0058] In some embodiments, the conference tablet can be connected to an external camera module, and the camera module receives control instructions from the conference tablet to collect information about the current space. The camera module can communicate with the conference tablet to transmit audio and video. The camera module equipment must cover the area where the conference tablet is located. The camera module can be placed above the space area where the conference tablet is located or on both sides parallel to the conference tablet, etc. No sole limitation is made here for the camera module.

[0059] Among them, in addition to displaying the screen captured by the conference tablet, the above-mentioned display screen can also display other content. The display screen can be divided into a first display area, a second display area, etc. For example, the first display area can display content such as meeting minutes, the second display area can display the screen captured by the conference tablet of the meeting participants, and the third display area can be the screen of connected non-meeting participants. There is no unique limitation on the display screen here, and the specific display can be achieved according to the user's operating instructions.

[0060] S20: Determine a target person area according to the display image.

[0061] In this step, the target person area is the image area corresponding to the target person in the display screen, wherein the target person can be customized by the user or automatically selected by the conference tablet according to preset rules.

[0062] In some embodiments, the target person region includes not only the target person region corresponding to the target person, but also regions of irrelevant persons other than the target person. The regions corresponding to such irrelevant persons may be processed by Gaussian blurring or virtualization.

[0063] For example, if Figure 1C In the example, person C is identified as speaking in area a, person C is selected as the target person, and the area corresponding to C is framed.

[0064] In one embodiment, determining the target person area according to the display screen includes: acquiring voice information collected by the conference tablet; and determining the target person area in the display screen according to the voice information.

[0065] The above-mentioned voice information is audio information collected by the microphone array in the conference tablet.

[0066] Optionally, determining the target person area in the display screen according to the voice information includes: locating the sound source of the voice information to obtain the target sound source distance and sound source direction in the voice information; and determining the target person area according to the sound source distance and the sound source direction.

[0067] The above-mentioned voice information is collected by a microphone array in the conference flat panel, and the microphone array may be a linear array, a planar array, or a stereo array. The above-mentioned microphone array can perform sound source localization processing on the voice information.

[0068] Among them, the above-mentioned sound source localization refers to the use of multiple microphones to measure sound signals at different locations in the environment. Since the time for the sound signal to reach each microphone has different degrees of delay, the measured sound signal is processed by an algorithm to obtain the arrival direction (including azimuth, pitch angle) and distance of the sound source point relative to the microphone.

[0069] It can be seen that the voice information collected by the conference tablet can quickly and accurately obtain the speaker in the current meeting, and then determine that the speaker is the target person.

[0070] Optionally, determining the target person area according to the display screen includes: receiving a user instruction, and determining the target person area according to the user instruction, wherein the user instruction is obtained by a user operating in the display screen of the conference tablet.

[0071] Among them, the above-mentioned user instructions can be issued by the user operating the control end of the conference tablet, and the control end can be the display panel of the conference tablet. According to the display screen on the operation display panel, the target person is selected and the user instructions are issued to the selected target person.

[0072] Optionally, determining the target person region according to the user instruction includes: selecting the target person according to the user instruction, extracting the region where the target person is located, and obtaining the target person region.

[0073] It can be seen that by directly confirming the target person by the user and extracting the area of ​​the target person, the confirmation of the target person can be faster and more accurate, without starting other devices for collection and analysis, which can save power consumption to a certain extent.

[0074] S30: Determine a capture frame corresponding to the target person region and movement information of the capture frame relative to the target person region.

[0075] Among them, the above-mentioned capture box is a selection box set in the conference tablet for selecting the target person area.

[0076] The above movement information is the movement information of the capture frame relative to the target person. The capture frame is moved according to the movement information to achieve the purpose of tracking the target person.

[0077] The above movement information may include a target movement distance and a target movement direction. The above target movement distance refers to the distance between the focus of the target person and the focus point of the current capture frame; the above target movement direction refers to the movement direction of the target person relative to the capture frame.

[0078] The area size of the above-mentioned interception frame can be obtained according to the physical distance between the conference tablet and the target person, and the area size of the interception frame can be obtained in a variety of ways.

[0079] Optionally, the capture ratio is determined based on the proportion of the target person in the displayed screen, and the area size of the capture frame is set according to the capture ratio; or the actual distance from the target person to the conference tablet is detected based on a ranging module, and the width and height of the capture frame are determined based on the actual distance. The above-mentioned ranging module is a module inside the conference tablet.

[0080] In a specific example, according to the proportion of the target person in the display screen, the interception ratio is determined according to the screen proportion, and the area size of the interception frame is set according to the interception ratio. It is assumed that in the display screen of S10, the target person area occupies 40% of the entire screen height. Assume: the height of the display screen in the conference tablet is 1080 pixels, and the width of the screen is 1920 pixels. Calculate the height of the target person in the screen (H_target): Htarget==screen height×target person proportion=1080 pixels×0.4=432 pixelsHtarget=screen height×target person proportion=1080 pixels×0.4=432 pixels; then calculate the height (H_crop) and width (W_crop) of the interception frame to ensure that the target person is displayed in the correct proportion in the interception frame. Suppose we want the interception frame to leave some margins around the target person, such as leaving 10% blank margins above and below. This means that the height of the interception frame will be 120% of the target person's height (100% + 10% + 10%), and the width will be determined according to the aspect ratio of the picture. Hcrop = 1.2 × Ht arget = 1.2 × 432 pixels = 518.4 pixels; Calculate the width of the interception frame (W_crop): Wcrop = picture aspect ratio Hcrop = 1920 / 1080518.4 pixels ≈ 1395.75 pixels. The final height of the interception frame is about 518.4 pixels and the width is about 1395.75 pixels to ensure that the target person is displayed in the correct proportion in the interception frame, and leave 10% blank margins at the top and bottom.

[0081] In a specific example, according to the distance measurement module, the actual distance from the target person to the conference flat panel is detected, and the width and height of the interception frame are determined according to the actual distance. Assuming that there is a participant on the conference flat panel and the participant stands in front of the conference flat panel, the distance is measured by the distance measurement module as 2 meters through laser or ultrasonic technology. Assuming that the display panel size of the conference flat panel is 10 inches (about 25.4 cm), and assuming that the resolution of the conference flat panel is 1920x1080 pixels. Based on the above data, we can perform the following calculations: Since the participant is 2 meters away from the flat panel, we can use triangle similarity to determine the height of the participant on the screen. The physical height of the screen (H_screen) is 25.4 cm. The high resolution of the screen (R_screen_height) is 1080 pixels. It can be calculated by Hdisplay = (Dreal / actual distance from the screen to the target) × Hscreen that the height of the participant on the screen will be 25.4 cm. Next, we can calculate the height of the interception frame (H_crop) according to the high resolution of the screen. Assuming that we want to display the participants in full height, the height of the cropping frame will be equal to the high resolution of the screen: Hcrop = Rscreenh eight = 1080 pixels; Now, we can calculate the width of the cropping frame (W_crop) according to the aspect ratio of the screen to ensure that the image of the participant is not stretched: Wcrop = screen aspect ratio Hcrop, assuming that the width of the screen is 19.2 cm, so the aspect ratio of the screen is 1920 / 1080 = 1.77778. Wcrop = 1.777781080 pixels ≈ 608.4 pixels, so in this specific numerical case, the conference tablet determines the width and height of the cropping frame based on the 2-meter distance measured by the ranging module. The height of the cropping frame is 1080 pixels and the width is about 608.4 pixels to adapt to the aspect ratio of the screen, so as to correctly display the image of the participant.

[0082] It can be seen that by determining the area size of the cropping frame by the target person, the set cropping frame can be made to fit the size of the target person area, there can be enough space to display the target person area, and the situation where too much blank space may occur when directly generating the cropping frame is avoided to a great extent.

[0083] Optionally, the target moving direction can also be determined by: capturing the first frame image and the second frame image of the target portrait in the video stream in the conference tablet, the first frame image being the previous frame image of the second frame image; obtaining the first position of the target portrait in the first frame image; obtaining the second position of the target portrait in the second frame image; and determining the direction in which the second position is located at the first position, the direction being the moving direction of the target portrait.

[0084] It can be seen that according to the movement information of the capture frame and the target person, it can be ensured that when the capture frame is subsequently controlled to move, there will be no lag or the target person will move out of the screen, and the capture frame will track the target person in real time according to the movement information.

[0085] S40: Control the movement of the capture frame according to the movement information to obtain a moved capture frame.

[0086] Among them, the above-mentioned moved screenshot frame is a relative concept. For the screenshot frame presented on the conference tablet, it will not move in position, nor will it jump from a fixed display area to another area; but for the target person that the screenshot frame needs to capture, it is not a fixed position. The screenshot frame will track the target person according to the movement information, that is, the area size of the screenshot frame remains unchanged, but the content of the screenshot frame will chase the target person and move dynamically to ensure that the screenshot frame can completely capture the target person area.

[0087] S50: capturing a close-up image including the target person area in the display image according to the moved capture frame.

[0088] Among them, the above-mentioned close-up picture is a picture containing the target person area displayed on the conference tablet, and can be displayed through the display panel.

[0089] For example, in a remote meeting, the conference tablet only displays Figure 3A The picture on the left, as shown in the left picture of 3A, shows three characters, namely character 31, character 32 and character 33. The cutout frame is the border of the picture on the right, and the close-up picture is Figure 3A In the right picture, only a close-up of person 32 is shown.

[0090] S60: Control the conference tablet to display the close-up image.

[0091] The close-up picture may be a picture including the target person area, or a picture only including the target person area.

[0092] Among them, the above-mentioned conference tablet can partially display the close-up picture in a specific area or display the close-up picture in the entire display panel of the conference tablet, which is not limited here.

[0093] For example, if the conference tablet displays a close-up image in a specific area, you can Figure 1C The D area in the figure shows a specific area.

[0094] Optionally, if the close-up image is fully displayed on the display panel of the conference tablet, the display image in step S10 is placed in the background, or the close-up image is placed on the top of the display panel. It can be seen that the close-up image captured by the capture frame and movement information can ensure the smoothness and integrity of the target person when displayed.

[0095] In general, this embodiment moves the capture frame, and the moved capture frame can track the target person area in time, and capture the target person area from the display screen to obtain a close-up picture. This can prevent the user corresponding to the target person area from easily walking out of the close-up picture, so that this embodiment can reliably display the close-up picture including the target person area.

[0096] In addition, the related technology can also track the target person area and lock the target person area in the middle of each frame of the display screen each time. This method will make the target person area too stiff and unnatural. This embodiment captures the close-up picture in a smooth manner by moving the capture frame. This method simulates the camera's follow-up shooting method, so that the close-up picture output by this embodiment is smoother and more natural.

[0097] In one embodiment, the movement information includes a target movement distance and a target movement direction, and determining the movement information of the capture frame relative to the target person area includes: respectively determining a reference point of the target person area and a focus point of the capture frame; calculating the distance from the reference point to the focus point to obtain the target movement distance; and determining the target movement direction based on the relative position of the reference point and the focus point.

[0098] For example, if Figure 3B As shown, Figure 3B The captured frame corresponding to the target person area is shown. In this figure, the gray circle is the focus point of the captured frame, and the black circle is the reference point of the target person area. The d shown in the figure is the target moving distance. Since the captured frame moves with the target person, the direction of the focus point toward the reference point is the target moving direction.

[0099] Among them, in determining the reference point of the target person area, the reference point is determined by the face area of ​​the target person in the target person area, and a face recognition algorithm is required to recognize the regional image of the face in the target person area.

[0100] Optionally, before determining the reference point of the target person area, the picture clarity of the target person area is detected. If the clarity is less than a preset clarity, the target person area is restored to obtain a picture corresponding to the target person area that meets the preset clarity.

[0101] The preset clarity may be a preset value or may be obtained according to the factory setting of the display panel of the conference tablet, and is not limited here.

[0102] Among them, the above-mentioned image quality restoration may include but is not limited to adjusting the clarity and details of the image by increasing contrast, fine-tuning color balance and curves, etc.; or using interpolation technology, which can increase the number of video frames, thereby improving the smoothness and clarity of the image in the video stream; or adjusting the resolution of the display panel to a higher resolution to make the image clearer, which is not the only limitation here.

[0103] It can be seen that by repairing the image quality of the target person area, a clearer target person area image can be obtained, and then in the subsequent steps, reference point processing can be performed based on the clear target person area image, thereby improving the accuracy of this solution.

[0104] In one embodiment, determining the reference point of the target person area includes: generating a face recognition frame of the target person area; obtaining the midpoint of the first horizontal line of the face recognition frame and a first three-point line perpendicular to the first horizontal line; and determining the intersection of the midpoint of the first horizontal point and the first three-point point as the reference point.

[0105] For example, if Figure 3C As shown, Figure 3C This is a schematic diagram of reference points in the target person area. As shown in the figure, the intersection of the midpoint of the first horizontal point and the first three-point point is determined as the reference point.

[0106] Optionally, when generating the face recognition frame of the target person area, it also includes: if the face area cannot be completely recognized when identifying the target person area, then identifying the area of ​​the person's head in the target person area, and placing the face recognition frame outside the area of ​​the person's head.

[0107] The inability to fully recognize the facial area may be due to occlusion by the person's bangs or the person's posture.

[0108] The first third line refers to evenly dividing the vertical line segment of the face recognition frame into three segments, drawing a horizontal line segment at the segment position close to the forehead of the target person, and setting the horizontal line segment as the first third line.

[0109] Among them, the first horizontal center line refers to the horizontal line segment of the face recognition frame being evenly divided into two segments, and the midpoint is taken as the vertical line segment as the first horizontal center line.

[0110] It can be seen that by setting the intersection of the horizontal midpoint and the third point of the face recognition frame as the reference point, the target point that the capture frame follows is set.

[0111] In one embodiment, determining the focus point of the clipping frame includes: obtaining the midpoint of a second horizontal line of the clipping frame and a second third line perpendicular to the second horizontal line; and determining the intersection of the midpoint of the second horizontal point and the second third point as the focus point.

[0112] The second third line is to divide the vertical line segment of the interception frame into three segments evenly, draw a horizontal line segment near the upper segment position, and set the horizontal line segment as the second third line.

[0113] The second horizontal center line refers to the horizontal line segment of the interception frame being evenly divided into two segments, and the midpoint is taken as a vertical line segment as the second horizontal center line.

[0114] For example, if Figure 3D As shown, Figure 3D This is a schematic diagram of the focus point of the interception frame. As shown in the figure, the intersection of the midpoint of the second horizontal point and the second third point is determined as the reference point.

[0115] It can be seen that by setting the intersection of the horizontal midpoint of the interception frame and the third point as the reference point, the focus point of the interception frame is set, and the movement information required by this solution is obtained more accurately and conveniently through the positions of the focus point and the reference point.

[0116] In one embodiment, determining the target moving direction according to the relative position of the reference point and the focus point comprises: determining the target moving direction by the direction of a straight line from the focus point to the reference point.

[0117] Optionally, in the horizontal direction, the target moving direction can be a left moving direction and a right moving direction. If the reference point is to the right of the focusing point, the left moving direction is determined as the target moving direction; if the reference point is to the left of the focusing point, the right moving direction is determined as the target moving direction.

[0118] Optionally, in the vertical direction, the target moving direction can be an upward moving direction or a downward moving direction. If the reference point is above the focusing point, the upward moving direction is determined as the target moving direction; if the reference point is below the focusing point, the downward moving direction is determined as the target moving direction.

[0119] Since the target moving direction is determined by the straight line direction from the focus point to the reference point in this solution, no diagonal movement will occur in the upward or downward moving direction, and the movement is only in relative position.

[0120] In one embodiment, the movement information includes a target moving distance and a target moving direction, and the movement of the capture frame is controlled according to the movement information to obtain the moved capture frame, including: calculating a moving speed according to the target moving distance and a preset speed coefficient; and controlling the capture frame to move in the target moving direction according to the moving speed to obtain the moved capture frame.

[0121] Specifically, in calculating the moving speed according to the target moving distance and the preset speed coefficient, it can be calculated by the following formula: s=(d / w1)*W, where W is the speed coefficient, d is the target moving distance, w1 is the radius of the long frame of the interception frame, and s represents the moving speed.

[0122] The speed coefficient can be obtained according to the display resolution of each conference tablet, and is not limited here.

[0123] It can be seen that by moving the capture frame according to the target moving direction and target speed, real-time tracking of the target person can be achieved, and the target person can be brought back into focus of the picture, which can not only respond to the movement of the person in a timely manner, but also well simulate the following effect of a real camera.

[0124] In one embodiment, before controlling the capture frame to move, the method further includes: when the focus point of the capture frame is aligned with the reference point of the target person area, controlling the capture frame to stop moving.

[0125] When the focus point of the capture frame is aligned with the reference point of the target person area, that is, the current moving speed is 0, it is determined that the capture frame stops moving, and it can be further determined that the target person does not move significantly.

[0126] In one embodiment, when capturing a close-up image for the first time, the focus point of the capture frame is aligned with the reference point of the target person area.

[0127] Among them, when the above-mentioned capture frame is set for the first time, it is aligned with the reference point of the target person area, that is, the capture frame is set with the target person area as the focus of the picture. If the reference point of the target person area is no longer in the aligned position, it is judged that the current target person has moved, and the capture frame needs to be controlled to move.

[0128] In one embodiment, controlling the conference tablet to display the close-up picture includes: when it is detected that the close-up picture contains an irrelevant person area, eliminating the irrelevant person area to obtain a processed close-up picture; controlling the conference tablet to display the processed close-up picture.

[0129] The elimination process may be Gaussian blur or virtualization.

[0130] Gaussian blurring means taking the center pixel as the origin and assigning different weights to all pixels within a specified radius (N*N rectangle, N is an odd number) according to the distance from the center pixel. Finally, the pixel value is multiplied and translated (convolved) by the weight to calculate the pixel value of the center point. The same calculation is performed for each pixel, and finally a Gaussian blurred image is obtained.

[0131] It can be seen that by eliminating the irrelevant person area of ​​the close-up picture, a neater and clearer close-up picture can be obtained, and the close-up picture that only retains the target person can be tracked more smoothly.

[0132] For example, the camera in the conference tablet captures the first display screen, and the target portrait area of ​​the speaker in the first display screen is determined based on the voice information collected by the microphone in the conference tablet. Initially, a capture frame is set, the focus point of the capture frame is aligned with the reference point of the target portrait area, and the first display screen is captured according to the capture frame to obtain the second display screen, and the conference tablet is controlled to display the second display screen, wherein, in the second display screen at this time, the reference point of the target portrait area is on the focus point. After the target person moves, the moving speed is calculated according to the distance between the previous capture frame and the current target person, and the capture frame is controlled to move to obtain the moved capture frame, and the screen is captured according to the moved capture frame, and the conference tablet is controlled to display the captured screen, which is a close-up screen, so that the target person can be tracked, and the person's movement can be followed in time to prevent the person from walking out of the display screen.

[0133] It should be noted that, in each of the above-mentioned embodiments, there is not necessarily a certain order between the above-mentioned steps. A person skilled in the art can understand, based on the description of the embodiments of the present invention, that in different embodiments, the above-mentioned steps may have different execution orders, that is, they may be executed in parallel, may be executed interchangeably, and so on.

[0134] As another aspect of the embodiments of the present invention, the embodiments of the present invention provide a person close-up device based on a conference tablet. The person close-up device based on a conference tablet can be a software module, which includes several instructions stored in a memory, and the processor can access the memory and call the instructions for execution to complete the person close-up method based on the conference tablet described in the above embodiments.

[0135] In some embodiments, the character close-up device based on the conference tablet can also be built by hardware devices. For example, the character close-up device based on the conference tablet can be built by one or more chips, and the chips can work in coordination with each other to complete the character close-up method based on the conference tablet described in the above embodiments.

[0136] See also Figure 4 , Figure 4 Schematic diagram of a device for close-up of a person based on a conference tablet provided in an embodiment of the present application. Figure 4 As shown, the person close-up device based on the conference tablet includes an acquisition unit 401, a determination unit 402, a control unit 403, and a capture unit 404:

[0137] The acquisition unit 401 is used to acquire the display screen collected by the conference tablet;

[0138] The determining unit 402 is used to determine a target person area according to the display picture;

[0139] The determining unit 402 is further configured to determine a clipping frame corresponding to the target person region and movement information of the clipping frame relative to the target person region;

[0140] The control unit 403 is used to control the movement of the capture frame according to the movement information to obtain the moved capture frame;

[0141] The capture unit 404 is configured to capture a close-up image containing the target person area in the display image according to the moved capture frame;

[0142] The control unit 403 is further configured to control the conference tablet to display the close-up image.

[0143] This embodiment moves the capture frame, and the moved capture frame can track the target person area in time, and capture the target person area from the display screen to obtain a close-up picture, so that the user corresponding to the target person area can be prevented from easily walking out of the close-up picture, so that this embodiment can reliably display the close-up picture containing the target person area. In addition, this embodiment captures the close-up picture in a smooth manner by moving the capture frame, which simulates the camera's follow-up shooting method, so that the close-up picture output by this embodiment has a smoother and more natural effect.

[0144] In one embodiment, the movement information includes a target movement distance and a target movement direction, and in the movement information of the capture frame relative to the target person area, the determination unit 402 is further used to: respectively determine a reference point of the target person area and a focus point of the capture frame; calculate the distance from the reference point to the focus point to obtain the target movement distance; and determine the target movement direction according to the relative position of the reference point and the focus point.

[0145] In one embodiment, in determining the reference point of the target person area, the determination unit 402 is also used to: generate a face recognition frame of the target person area; obtain the midpoint of the first horizontal line of the face recognition frame and the first three-point line perpendicular to the first horizontal line; and determine the intersection of the midpoint of the first horizontal point and the first three-point point as the reference point.

[0146] In one embodiment, in determining the focus point of the captured frame, the determination unit 402 is further used to: obtain the midpoint of a second horizontal line of the captured frame and a second third line perpendicular to the second horizontal line; and determine the intersection of the midpoint of the second horizontal point and the second third point as the focus point.

[0147] In one embodiment, in determining the target moving direction according to the relative positions of the reference point and the focus point, the determining unit 402 is further configured to: determine the target moving direction using a straight line direction from the focus point to the reference point.

[0148] In one embodiment, the movement information includes a target movement distance and a target movement direction. When controlling the movement of the capture frame according to the movement information to obtain the captured frame after movement, the control unit 403 is further used to: calculate the movement speed according to the target movement distance and a preset speed coefficient; and control the capture frame to move in the target movement direction according to the movement speed to obtain the captured frame after movement.

[0149] In one embodiment, before controlling the capture frame to move, the control unit 403 is further configured to: when the focus point of the capture frame is aligned with the reference point of the target person area, control the capture frame to stop moving.

[0150] In one embodiment, when capturing a close-up image for the first time, the focus point of the capture frame is aligned with the reference point of the target person area.

[0151] In one embodiment, in controlling the conference tablet to display the close-up picture, the control unit 403 is also used to: when it is detected that the close-up picture contains an irrelevant character area, eliminate the irrelevant character area to obtain a processed close-up picture; and control the conference tablet to display the processed close-up picture.

[0152] In one embodiment, in the area of ​​determining the target person in the display screen, the determining unit 402 is further used to: obtain voice information collected by the conference tablet; and determine the target person in the display screen according to the voice information.

[0153] It should be noted that the above-mentioned person close-up device based on a conference tablet can execute the person close-up method based on a conference tablet provided in the embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not fully described in the embodiment of the person close-up device based on a conference tablet, please refer to the person close-up method based on a conference tablet provided in the embodiment of the present invention.

[0154] See also Figure 5 , Figure 5 Schematic diagram of the structure of a computer device provided in an embodiment of the present application. Figure 5 As shown, the computer device 50 includes a processor 501 and a memory 502. The memory 502 is connected to the processor 501, for example, via a bus.

[0155] The processor 501 is configured to support the computer device 50 to perform the corresponding functions in the method in the above method embodiment. The processor 501 can be a central processing unit (CPU), a network processor (NP), a hardware chip or any combination thereof. The above hardware chip can be an application specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof. The above PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0156] Specifically, the processor 501 may include a sending card, a receiving card and a driving chip.

[0157] The memory 502 is used to store program codes, etc. The memory 502 may include a volatile memory (VM), such as a random access memory (RAM); the memory 502 may also include a non-volatile memory (NVM), such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); the memory 502 may also include a combination of the above-mentioned types of memories.

[0158] The processor 501 may call the program code to perform the following operations:

[0159] Get the display screen captured by the conference tablet;

[0160] Determine a target person area according to the display image;

[0161] Determine a capture frame corresponding to the target person area and movement information of the capture frame relative to the target person area;

[0162] Control the movement of the capture frame according to the movement information to obtain the moved capture frame;

[0163] intercepting a close-up picture including the target person area in the display picture according to the moved interception frame;

[0164] Control the conference tablet to display the close-up image.

[0165] An embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a computer, the computer executes the method described in the above embodiment.

[0166] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0167] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A person close-up method based on a conference tablet, characterized in that: include: Get the display screen captured by the conference tablet; Determine a target person area according to the display image; Determine a capture frame corresponding to the target person area and movement information of the capture frame relative to the target person area; Control the movement of the capture frame according to the movement information to obtain the moved capture frame; intercepting a close-up picture including the target person area in the display picture according to the moved interception frame; Control the conference tablet to display the close-up image.

2. The method according to claim 1, characterized in that The movement information includes the target movement distance and the target movement direction, and the determination of the movement information of the capture frame relative to the target person area includes: Determine the reference point of the target person area and the focus point of the capture frame respectively; Calculate the distance from the reference point to the focus point to obtain the target moving distance; The target moving direction is determined according to the relative position of the reference point and the focus point.

3. The method according to claim 2, characterized in that The step of determining the reference point of the target person area includes: Generate a face recognition frame of the target person area; Obtaining the midpoint of a first horizontal line of the face recognition frame and a first third line perpendicular to the first horizontal line; An intersection point between the midpoint of the first horizontal point and the first three-point point is determined as a reference point.

4. The method according to claim 2, characterized in that: The determining the focus point of the capture frame includes: Obtaining a midpoint of a second horizontal line of the interception frame and a second third line perpendicular to the second horizontal line; An intersection point of the second horizontal point midpoint and the second third point is determined as a focus point.

5. The method according to claim 2, characterized in that: Determining the target moving direction according to the relative position of the reference point and the focus point comprises: The target moving direction is determined by the straight line direction from the focus point to the reference point.

6. The method according to claim 1, characterized in that The movement information includes the target movement distance and the target movement direction. The capture frame is controlled to move according to the movement information, and the captured frame after the movement includes: Calculate the moving speed according to the target moving distance and a preset speed coefficient; The capture frame is controlled to move according to the moving speed in the target moving direction to obtain a moved capture frame.

7. The method according to claim 1, characterized in that Before controlling the capture frame to move, the method further includes: When the focus point of the capture frame is aligned with the reference point of the target person area, the capture frame is controlled to stop moving.

8. The method according to claim 1, characterized in that When capturing a close-up image for the first time, the focus point of the capture frame is aligned with the reference point of the target person area.

9. The method according to any one of claims 1 to 8, characterized in that: The controlling the conference tablet to display the close-up picture includes: When it is detected that the close-up picture contains an irrelevant person area, the irrelevant person area is eliminated to obtain a processed close-up picture; The conference tablet is controlled to display the processed close-up image.

10. The method according to claim 1, characterized in that The determining the target person area according to the display picture includes: Acquire the voice information collected by the conference tablet; A target person area is determined in the display image according to the voice information.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the character image display method based on a conference tablet as described in any one of claims 1 to 10.

12. A conference tablet, characterized in that: include: A camera, used to collect and display images; Microphone array, used to collect voice information; Memory; A processor is communicatively connected to the camera, the microphone array and the memory respectively, wherein the processor is used to execute one or more computer programs stored in the memory, and when the processor executes the one or more computer programs, the conference tablet implements the character image display method based on the conference tablet as described in any one of claims 1-10.

Images