Remote Video Picture Interaction Method, Device and Computer Readable Storage Medium

Abstract

The present invention discloses a remote video screen interaction method, device and computer-readable storage medium, belonging to the technical field of remote video screen interaction. The present invention obtains original image data of the scene area in real time, provides an image basis for remote image display, and determines the character frame in the original video image, and then monitors in real time whether the position of the character frame is updated. If the position of the character frame is updated, the target display frame is determined based on the updated character frame, effectively boxing the participants, and determining the target display frame after the position change to achieve human figure tracking of the video image, avoiding secondary processing of the video image at the remote end, thereby converting the target display frame into a target display image according to a preset resolution, and pushing the target display image to the remote end, realizing the function of remote video interaction human figure following, effectively improving the use flexibility of remote video screen interaction and improving the user interaction experience.

Description

Technical Field

[0001] The present invention relates to the technical field of remote video screen interaction, and particularly to a remote video screen interaction method, device and computer-readable storage medium. Background Art

[0002] With the rapid development of social economy and science and technology, coupled with the wide spread of high-speed networks, remote video interaction devices are being increasingly widely used in scenarios such as work and office, and home communication. In recent years, due to the complex changes in the social environment, working from home has become an alternative way of working in the unit, which makes it a mainstream trend for many workers to have a remote video interaction device. Currently, remote video interaction devices generally obtain the original video image data through a shooting sensor, and then transmit the original image data to a computer interaction interface for display, or transmit it to a remote end for display through the network, so as to realize the function of video screen interaction with remote users.

[0003] However, traditional remote video interaction devices can only fixedly display a specific position in the environment. In actual video conferences, it is more desired to display a video screen centered on the participants to highlight the main object and improve the meeting communication experience. However, with the movement of people, the video screens displayed by traditional remote video interaction devices cannot meet this aspect of requirements. For this problem, general users can only manually adjust the shooting angle and distance of traditional remote video interaction devices in order to display a video screen centered on the participants for remote users in real time. And this method is also very lacking in flexibility and will affect the meeting efficiency, thus not being able to well meet the needs of users. Summary of the Invention

[0004] The main purpose of the present invention is to provide a remote video screen interaction method, device and computer-readable storage medium, aiming to solve the technical problem of how to improve the flexibility of remote video screen interaction.

[0005] To achieve the above object, the present invention provides a remote video screen interaction method, and the remote video screen interaction method includes the following steps:

[0006] Obtain the original video image in real time, and determine the human body frame diagram in the original video image;

[0007] Monitor whether the position of the human body frame diagram is updated. If it is monitored that the position of the human body frame diagram is updated, determine the target display frame based on the updated human body frame diagram;

[0008] Convert the target display frame into a target display image according to a preset resolution, and push the target display image to the remote end.

[0009] Optionally, the step of determining the target display box based on the updated character block diagram includes:

[0010] Determine the starting coordinates of the character in the original video image for the updated character block diagram, and detect the size information of the character block diagram according to the starting coordinates of the character;

[0011] If there is a character block diagram, determine the edge positions in each direction in the original video image, and use the character block diagram with the smallest distance to the edge positions as the target character block diagram;

[0012] Construct a target display box according to the size information of at least one target character block diagram corresponding to the edge positions in each direction of the original video image.

[0013] Optionally, after the step of monitoring whether there is an update in the position of the character block diagram, it further includes:

[0014] Determine the current target display image of the remote end. If it is monitored that the position of the character block diagram does not change within the first preset time range, use the current target display image as the target display image;

[0015] Wherein, obtain the size information of the current target display image, determine the current target display box based on the position of the character block diagram corresponding to the end moment of the first preset time range, and determine whether the horizontal viewing angle of the current target display box is greater than or equal to a preset angle;

[0016] If the horizontal viewing angle is greater than or equal to the preset angle, determine a first preset condition according to the size information of the current target display image and the center point coordinate information of the character block diagram. If within the first preset time range, the character block diagram corresponding to the character block diagram of the historical display box meets the first preset condition, it is determined that the position of the character block diagram does not change within the first preset time range; or,

[0017] If the horizontal viewing angle is less than the preset angle, determine a second preset condition according to the center point coordinate information of the character block diagram. If within the first preset time range, the character block diagram corresponding to the character block diagram of the historical display box meets the second preset condition, it is determined that the position of the character block diagram does not change within the first preset time range.

[0018] Optionally, the character block diagrams corresponding to the end moment of the first preset time range include the leftmost character block diagram, the rightmost character block diagram, the uppermost character block diagram, and the lowermost character block diagram. The step of determining the first preset condition according to the size information of the current target display image box and the center point coordinate information of the character block diagram includes:

[0019] Determine the height of the current target display image, the width of the current target display image, and the image starting coordinates of the current target display image in the size information of the current target display image, and determine the leftmost historical figure block diagram, the rightmost historical figure block diagram, the topmost historical figure block diagram, and the bottommost historical figure block diagram among all the figure block diagrams corresponding to the historical display box in the current target display image;

[0020] Take that the ordinate of the center point of the topmost figure block diagram is greater than the ordinate of the image starting coordinates and less than the sum of the ordinate of the center point of the historical topmost figure block diagram and the height of the historical topmost figure block diagram; the ordinate of the center point of the bottommost figure block diagram is greater than the difference between the ordinate of the center point of the historical bottommost figure block diagram and the height of the historical bottommost figure block diagram and less than the sum of the ordinate of the image starting coordinates and the height of the current target display image; the abscissa of the center point of the leftmost figure block diagram is less than the sum of the abscissa of the center point of the historical leftmost figure block diagram and the width of the historical leftmost figure block diagram and greater than the abscissa of the image starting coordinates; the ordinate of the center point of the rightmost figure block diagram is greater than the difference between the abscissa of the center point of the historical rightmost figure block diagram and the width of the historical rightmost figure block diagram and less than the sum of the abscissa of the image starting coordinates and the width of the current target display image as the first preset condition;

[0021] The step of determining the second preset condition according to the center point coordinate information of the figure block diagram includes:

[0022] Take that the absolute value of the difference between the ordinate of the center point of the topmost figure block diagram and the ordinate of the center point of the historical topmost figure block diagram is less than the height of the historical topmost figure block diagram; the absolute value of the difference between the ordinate of the center point of the bottommost figure block diagram and the ordinate of the center point of the historical bottommost figure block diagram is less than the height of the historical bottommost figure block diagram; the absolute value of the difference between the abscissa of the center point of the leftmost figure block diagram and the abscissa of the center point of the historical leftmost figure block diagram is less than the width of the historical leftmost figure block diagram; the absolute value of the difference between the abscissa of the center point of the rightmost figure block diagram and the abscissa of the center point of the historical rightmost figure block diagram is less than the width of the historical rightmost figure block diagram as the second preset condition.

[0023] Optionally, the step of determining the target display box based on the updated figure block diagram when it is detected that the position of the figure block diagram is updated includes:

[0024] If the position of the figure block diagram is updated within the first preset time range and the target display box is updated, detect whether the updated target display box continues to be updated within the second preset time range;

[0025] If there is a continued update, determine the current target display image of the remote end, and use the historical display box in the current target display image as the target display box;

[0026] If there is no continued update, perform the steps of determining the starting coordinates of the person in the video original image according to the person block diagram after the position change, and detecting the size information of the person block diagram according to the starting coordinates of the person.

[0027] Optionally, the person block diagrams corresponding to the end time of the first preset time range include the leftmost person block diagram, the rightmost person block diagram, the uppermost person block diagram, and the lowermost person block diagram. The step of detecting whether the updated target display box continues to be updated within the second preset time range includes:

[0028] Determine the positions of all the person block diagrams corresponding to the end time of the second preset time range, and based on the positions of each person block diagram corresponding to the end time of the second preset time range, determine the updated leftmost person block diagram, the updated rightmost person block diagram, the updated uppermost person block diagram, and the updated lowermost person block diagram among all the person block diagrams corresponding to the end time of the second preset time range;

[0029] Based on whether the person block diagrams corresponding to the end time of the second preset time range and the person block diagrams corresponding to the end time of the first preset time range satisfy the third preset condition, determine whether the updated target display box continues to be updated within the second preset time range;

[0030] The third preset condition includes:

[0031] The offset value between the ordinate of the center point of the updated uppermost person block diagram and the ordinate of the center point of the uppermost person block diagram is greater than or equal to the uppermost height equal division ratio;

[0032] And / or, the offset value between the ordinate of the center point of the updated lowermost person block diagram and the ordinate of the center point of the lowermost person block diagram is greater than or equal to the lowermost height equal division ratio;

[0033] And / or, the offset value between the abscissa of the center point of the updated leftmost person block diagram and the abscissa of the center point of the leftmost person block diagram is greater than or equal to the leftmost width equal division ratio;

[0034] And / or, the offset value between the abscissa of the center point of the updated rightmost person block diagram and the abscissa of the center point of the rightmost person block diagram is greater than or equal to the rightmost width equal division ratio.

[0035] Optionally, the person block diagram includes the leftmost person block diagram, the rightmost person block diagram, the uppermost person block diagram, and the lowermost person block diagram. The step of converting the target display box into a target display image according to the preset resolution includes:

[0036] Obtain the current target display image of the remote end, and determine the height of the current target display image, the width of the current target display image, and the image starting coordinates of the current target display image;

[0037] Determine the historical display box in the current target display image, and determine the historical leftmost person block diagram, the historical rightmost person block diagram, the historical uppermost person block diagram, and the historical lowermost person block diagram among all the person block diagrams corresponding to the historical display box;

[0038] If the person block diagram and the person block diagram corresponding to the historical display box meet the preset magnification condition, magnify the target display box according to the preset resolution, and generate a target display image based on the magnified target display box;

[0039] The preset magnification condition includes:

[0040] Determine the current target display box based on the position of the person block diagram, and determine whether the horizontal viewing angle of the current target display box is greater than or equal to the preset angle;

[0041] When the horizontal viewing angle is greater than or equal to the preset angle, the ordinate of the center point of the uppermost person block diagram is less than or equal to the ordinate of the current target display image, and / or the ordinate of the center point of the lowermost person block diagram is greater than or equal to the sum of the ordinate of the image starting coordinates and the height of the current target display image, and / or the abscissa of the center point of the leftmost person block diagram is less than or equal to the abscissa of the image starting coordinates, and / or the ordinate of the center point of the rightmost person block diagram is greater than or equal to the sum of the abscissa of the image starting coordinates and the width of the current target display image;

[0042] When the horizontal viewing angle is less than or equal to the preset angle, the ordinate of the center point of the uppermost person block diagram is less than or equal to the difference between the ordinate of the center point of the historical uppermost person block diagram and the height of the historical uppermost person block diagram, and / or the ordinate of the center point of the lowermost person block diagram is greater than or equal to the sum of the ordinate of the center point of the historical lowermost person block diagram and the height of the historical lowermost person block diagram, and / or the abscissa of the center point of the leftmost person block diagram is less than or equal to the difference between the abscissa of the center point of the historical leftmost person block diagram and the width of the historical leftmost person block diagram, and / or the ordinate of the center point of the rightmost person block diagram is greater than or equal to the sum of the abscissa of the center point of the historical rightmost person block diagram and the width of the historical rightmost person block diagram.

[0043] Optionally, the step of converting the target display box into a target display image according to the preset resolution includes:

[0044] When the person block diagram corresponding to the person block diagram and the historical display box meets the preset reduction condition, reduce the target display box according to the preset resolution, and generate a target display image based on the reduced target display box;

[0045] The preset reduction conditions include:

[0046] The ordinate of the center point of the uppermost person block diagram is greater than or equal to the sum of the ordinate of the center point of the historical uppermost person block diagram and the height of the historical uppermost person block diagram, and / or the ordinate of the center point of the lowermost person block diagram is less than or equal to the difference between the ordinate of the center point of the historical lowermost person block diagram and the height of the historical uppermost person block diagram, and / or the abscissa of the center point of the leftmost person block diagram is greater than or equal to the sum of the abscissa of the center point of the historical leftmost person block diagram and the width of the historical leftmost person block diagram, and / or the ordinate of the center point of the rightmost person block diagram is less than or equal to the difference between the abscissa of the center point of the historical rightmost person block diagram and the width of the historical rightmost person block diagram.

[0047] In addition, to achieve the above object, the present invention also provides a remote video picture interaction device, the remote video picture interaction device includes a memory, a processor, and a remote video picture interaction program stored on the memory and executable on the processor, wherein: when the remote video picture interaction program is executed by the processor, the steps of the remote video picture interaction method described above are implemented.

[0048] In addition, to achieve the above object, the present invention also provides a computer-readable storage medium, on which a remote video picture interaction program is stored, and when the remote video picture interaction program is executed by a processor, the steps of the remote video picture interaction method described above are implemented.

[0049] A remote video picture interaction method, device and computer-readable storage medium proposed by the present invention, which can obtain the original video image in real time, obtain the original image data of the on-site area in real time, provide an image basis for remote image display, and determine the person block diagram in the original video image, so as to determine the possible participants in the original video image, and then monitor in real time whether the position of the person block diagram is updated. If it is detected that the position of the person block diagram is updated, determine the target display box based on the updated person block diagram, effectively block the participants into diagrams, and determine the target display box through the changed position, so as to realize the human form following of the video image, avoid the secondary processing of the video image at the remote end, and then convert the target display box into a target display image according to the preset resolution, and push the target display image to the remote end, realizing the function of remote video interaction human form following, effectively improving the flexibility of using remote video pictures and enhancing the user interaction experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 It is a schematic structural diagram of a remote video screen interaction device for the hardware operating environment involved in the embodiment solution of the present invention;

[0051] Figure 2 It is a schematic flowchart of the first embodiment of a remote video screen interaction method of the present invention;

[0052] Figure 3 It is a schematic diagram of an application scenario of an embodiment of the present invention;

[0053] Figure 4 It is a schematic diagram of the application interface display of an embodiment of the present invention;

[0054] Figure 5 It is a schematic diagram of a block diagram of a person of an embodiment of the present invention;

[0055] Figure 6 It is a schematic diagram of a horizontal angle scene of an embodiment of the present invention.

[0056] The realization, functional features and advantages of the object of the present invention will be further described with reference to the embodiments and the accompanying drawings. Detailed implementation manners

[0057] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0058] Refer to Figure 1 , Figure 1 It is a schematic structural diagram of a remote video screen interaction device for the hardware operating environment involved in the embodiment solution of the present invention.

[0059] Such as Figure 1As shown in the figure, the remote video picture interaction device may include: a shooting device 1001, a communication module 1002, a power module 1003, and a control device 1004. The control module 1004 includes: a processor, such as a Central Processing Unit (CPU), a timer, and a memory. The memory may be a high-speed Random Access Memory (RAM) or a stable Non-Volatile Memory (NVM), such as a disk memory. The memory 1005 may optionally be a storage device independent of the aforementioned processor 1001. The remote video picture interaction device may also include a communication bus, a user interface, and a network interface. Among them, the communication bus is used to realize the connection and communication between these components. The user interface may include a display screen (Display) and an input unit such as a keyboard (Keyboard). Optionally, the user interface may also include a standard wired interface and a wireless interface. The network interface may optionally include a standard wired interface and a wireless interface (such as a Wireless-Fidelity (WI-FI) interface). Those skilled in the art can understand that Figure 1 the structure shown in the figure does not constitute a limitation on the remote video picture interaction device, and it may include more or fewer components than shown in the figure, or combine some components, or have different component arrangements.

[0060] As Figure 1 shown, the memory, as a computer-readable storage medium, may include a remote video picture interaction program. In addition, in addition to the illustrated part, it may also include an operating system, a data storage module, a network communication module, and a user interface module.

[0061] In Figure 1 the remote video picture interaction device shown, the communication module 1002 is mainly used to perform data communication with other devices and data interaction with users. For example, the remote video picture interaction device transmits video to a PC through the communication interface of the communication module 1002, where the communication interface is a USB interface or a network interface. The remote video picture interaction device calls the remote video picture interaction program stored in the memory through the central processor and executes the remote video picture interaction method provided by the embodiments of the present invention.

[0062] The embodiments of the present invention provide a remote video picture interaction method. Referring to Figure 2 , Figure 2 which is a schematic flowchart of the first embodiment of a remote video picture interaction method of the present invention.

[0063] In this embodiment, the remote video picture interaction method includes:

[0064] Step S100: Obtain the original video image in real time and determine the human frame diagram in the original video image.

[0065] In this embodiment, it should be noted that the remote video screen interaction is the interaction between the on-site area and the remote area. The remote user can connect the video signal with the on-site meeting through a computer and the Internet. The on-site remote video screen interaction device transmits the participation information in the on-site area to the remote computer through the output of the on-site meeting video signal and the Internet. Finally, the remote user can feel the face-to-face remote video interaction experience with the on-site meeting by watching the screen. And the signal output by the video is a video screen with the main body centered, which consists of all the participants within the visible range of the on-site remote video screen interaction device.

[0066] In the on-site area, there are no strict restrictions on the meeting information such as participants, electronic devices, and meeting table types. Refer to Figure 3 , in an application scenario, it includes an on-site area and a remote video interface of the on-site area seen by the relevant cloud area. In the on-site area, the remote video interaction humanoid following device (i.e., the remote video screen interaction device) is placed on the meeting table, and the position can be arbitrary, and the lens is oriented towards the target scene; Participants A - G are seated around the meeting table to participate in the meeting; The meeting table can be a long table or any type of table; A computer for connecting with the remote end for video interaction and used for displaying the remote scene video signal; A monitor for displaying the remote scene video signal. In the remote area, it can include a computer for receiving the video signal sent by the remote video interaction humanoid following device in the on-site meeting and used for displaying the on-site meeting scene video signal; A monitor for displaying the on-site scene video signal; Person P is a remote participant.

[0067] The remote video interface of the on-site area is the video signal output by the remote video interaction humanoid following device in the on-site meeting. Specifically, refer to Figure 4, including a scene simulation diagram in an application scenario and the video picture output by the corresponding remote video interaction humanoid following device. Among them, the R area is the real area, or the actual scene area, which is used to represent the scene within the visible range of the camera; the V area is the video area, or the video display area, which is used to represent the video imaging scene output by the device to the computer terminal; the bottom of the V area is provided with a "scale for the range of video horizontal viewing angle", which is only used to describe the range of the video horizontal viewing angle of the image in the V area and to guide developers in designing the UI (User Interface) ratio, and is not a part of the video content; the OS area is the original scale area, or the original scale area. This area uses scales to describe the "scale for the range of camera horizontal viewing angle". The scale above it describes the "range of the camera horizontal viewing angle", and the bold black font in the scale represents the "range of the video horizontal viewing angle in the V area (i.e., the same as the scale range of V)", and also describes the proportion of the "current video horizontal viewing angle in the V area" to the "camera horizontal viewing angle". Among them, the camera horizontal viewing angle refers to the horizontal angle range visible by the camera, which is the angle range from the left boundary of the camera viewing angle (with the front-facing direction of the camera as the reference, the boundary area of the leftmost visible range of the camera) to the right boundary of the camera viewing angle (with the front-facing direction of the camera as the reference, the boundary area of the rightmost visible range of the camera); the video horizontal viewing angle is the horizontal viewing angle of the actual scene displayed in the video, which is less than or equal to the size of the "camera horizontal viewing angle", and the "starting angle" and "ending angle" are within the range of the camera horizontal viewing angle. In this embodiment, the camera horizontal viewing angle is set to 90°, and this camera horizontal viewing angle can be a camera with any angle, without strict logical restrictions. In other embodiments, it can also be a camera with other horizontal angles.

[0068] The video original image refers to the original image determined based on the range of the camera horizontal viewing angle. At this time, the "video horizontal viewing angle" is equal to the "camera horizontal viewing angle". The person block diagram refers to the image information of the participants in the on-site area. The person block diagram can be in the shape of a square, rectangle, circle, polygon, etc. In this embodiment, it is displayed through a rectangular block diagram. The face detection algorithm, upper body detection algorithm of a person, or humanoid detection algorithm is used to detect the personnel information of the participants, and the starting coordinates of each person block diagram are determined (with the upper left corner of the person block diagram as the starting point, and the starting coordinates of the person block diagram are determined with reference to the starting coordinates of the video original image, and the starting coordinates of the video original image are the upper left corner of the video original image), as well as the width and height information of each person block diagram. Then, based on the starting coordinates and the width and height information of the person block diagram, the intersection position of the diagonal of the person block diagram is determined as the center point coordinates of the person block diagram.

[0069] Specifically, it is determined whether there is a person block diagram in the original video image. When there is such a person block diagram, relevant information of the person block diagram is obtained. If no person block diagram is detected in the original video image, it can be considered that there are no participants at the scene of the current meeting, and the original video image is directly sent to the remote end.

[0070] Step S200: Monitor whether there is an update in the position of the person block diagram. When it is detected that there is an update in the position of the person block diagram, determine a target display frame based on the updated person block diagram.

[0071] In this embodiment, it should be noted that the target display frame is an overall human frame representation including all person block diagrams. The target display frame is not directly equal to the overall range displayed in the video display area. For a better visual experience, the target display frame is default smaller than the overall range displayed in the video display area. Only in extreme cases will there be a situation where the target display frame is default equal to the overall range displayed in the video display area. Determine whether there is an update in the position of the person block diagram to determine whether there is movement of participants in the on-site area. When there is movement of participants in the on-site area, there will also be an update in the person block diagram. It can be understood that in some scenarios, subtle movements of participants, such as slight movement, slight head tilt, shrugging, rubbing hands, etc., which cause movement, do not affect the overall state of the participants, that is, the above-mentioned subtle movements do not affect the target display image. That is, the subtle movements of participants do not necessarily correspond to an update in the position of the person block diagram, but through certain conditions, the determination of whether there is an update in the position of the person block diagram is restricted to reduce the frequent change of the target display image caused by the subtle movements of participants. Refer to Figure 5 , in an application scenario, the target display frame takes the overall human frame (ROI) area as an example. Among them, the V frame is the frame corresponding to the V area (video display area). It contains two pieces of information: "starting point coordinates V(X V , Y V )" and "the size of the width and height of the V frame W V , H V "; the P frame is the frame corresponding to the "overall human frame", containing two pieces of information: "starting point coordinates P(X P , Y P )" and "the size of the width and height of the overall human frame W P , H P "; the U frame is the border corresponding to the person with the uppermost position among all person block diagrams in the overall human frame, that is, the uppermost person frame, containing two pieces of information: "center point coordinates U(X U , Y U )" and "the size of the width and height of the uppermost person frame W U , H U "; the D frame is the border corresponding to the person with the lowermost position among all person block diagrams in the overall human frame, that is, the lowermost person frame, containing "center point coordinates D(XD , Y D )” and the width and height of the bottommost frame, W D , H D ” Two pieces of information; the L frame is the border corresponding to the leftmost person in all the person diagrams within the overall person frame, that is, the leftmost person frame, which contains the “center point coordinates L(X L , Y L )” and the width and height of the leftmost person frame, W L , H L ” Two pieces of information; the R frame is the border corresponding to the rightmost person in all the person diagrams within the overall person frame, that is, the rightmost person frame, which contains the “center point coordinates R(X R , Y R )” and the width and height of the rightmost person frame, W R , H R ” Two pieces of information. Among them, all starting coordinates are determined based on the starting coordinates of the original video image as the reference coordinates, and the starting coordinates of the original video image are the upper left corner of the original video image of this video. It is not based on the upper left corner coordinates of the V area, that is, if a certain coordinate is (0, 0) in the original video image, its coordinate may not be the zero point coordinate in the scaled video mode.

[0072] Step S300, convert the target display frame into a target display image according to the preset resolution, and push the target display image to the remote end.

[0073] In this embodiment, it should be noted that the preset resolution is the resolution requested by the computer side. Image conversion is performed according to the preset resolution to ensure the effective transmission of the image. The target display image is the image finally displayed on the remote end interface. To ensure the image viewing experience, the target display frame is set to be centered and fully displayed as much as possible.

[0074] Specifically, the steps to achieve the centered display of the target display frame include:

[0075] If the horizontal viewing angle corresponding to the target display frame is less than or equal to the preset angle, fix the horizontal viewing angle of the target display image to the preset expansion angle, where the preset expansion angle is greater than the preset angle;

[0076] Based on the preset angle and the preset expansion angle, determine the position of the target display frame in the target display image, where the left and right boundaries of the target display frame are respectively equidistant from the left and right boundaries of the target display image; the upper and lower boundaries of the target display frame are respectively equidistant from the upper and lower boundaries of the target display image;

[0077] If the horizontal viewing angle corresponding to the target display frame is greater than a preset angle, set the horizontal viewing angle corresponding to the target display image to be greater than a preset expansion angle, where the preset expansion angle is greater than the preset angle;

[0078] Determine the position of the target display frame in the target display image based on the preset angle and the preset expansion angle, where the upper and lower boundaries of the target display frame are respectively equidistant from the upper and lower boundaries corresponding to the target display image; the distances between the left and right boundaries of the target display frame and the left and right boundaries corresponding to the target display image each have at least a preset reserved angle.

[0079] The distances between the upper and lower boundaries of the above-mentioned target display frame and the upper and lower boundaries corresponding to the target display image are calculated through the resolution ratio requested by the computer after determining the distances between the left and right boundaries of the target display frame and the left and right boundaries corresponding to the target display image. The preset angle, the preset expansion angle, and the preset reserved angle are all angles that can be changed according to the actual situation. Among them, the preset angle is the angle range in which the target display frame is displayed in the target display image; the preset expansion angle is the preset angle plus the angle range between the target display frame and the left and right boundaries of the target display image; the preset reserved angle is the angle range between the target display frame and the left and right boundaries of the target display image respectively. For example, the preset angle is N°, the preset expansion angle is M°, and the preset reserved angle is B°. If the horizontal viewing angle corresponding to the target display image ≤ N°, the target display frame needs to be fixed at M°, and ensure that the target display frame is centered and completely displayed in the target display image. If the horizontal viewing angle corresponding to the target display image > N°, the target display frame needs to be > M°, and determine that the distances between the left and right boundaries of the target display frame and the left and right boundaries corresponding to the target display image each have at least B°, and at this time ensure that the target display frame is horizontally centered in the target display image. In an embodiment, if the distance between one of the left and right boundaries of the target display frame and the left and right boundaries corresponding to the target display image is less than B°, at this time, the target display frame may not be horizontally centered in the target display image. In the extreme case, the distances between the left and right boundaries of the target display frame and the left and right boundaries corresponding to the target display image can infinitely tend to 0. In this embodiment, it is preferably restricted that M = N + 10. For example, N takes the value of 50, M takes the value of 60, and B takes the value of 5. Refer to Figure 6, in the application scenario of an embodiment, the R area is the scenario of the actual environment, where the horizontal angle of the "overall human frame (ROI) area" is 60°; the V area is a video display simulation diagram, and the horizontal viewing angle of the "overall human frame (ROI) area" in this video display simulation diagram is the same as that of the R area, both being 60°, but the "video horizontal viewing angle" of the V area remains 70°; the "overall human frame (ROI) area" is centered and is equidistant from the left and right boundaries of the video display simulation diagram and is also equidistant from the upper and lower boundaries of the video display simulation diagram.

[0080] In an embodiment of the present invention, the original video image is obtained in real time, and the original image data of the on-site area is obtained in real time, providing an image basis for remote image display, and determining the human figure frame in the original video image, so as to determine all possible participant states in the original video image. Furthermore, it is monitored in real time whether the position of the human figure frame is updated. If it is monitored that the position of the human figure frame is updated, the target display frame is determined based on the updated human figure frame. The target display frame is determined based on the human figure frame, effectively diagramming the participants, and determining the target display frame through the changed position, realizing the human figure following of the video image, avoiding secondary processing of the video image at the remote end, and then converting the target display frame into a target display image according to the preset resolution and pushing the target display image to the remote end, realizing the function of remote video interaction human figure following, effectively improving the flexibility of use of the remote video screen interaction and enhancing the user interaction experience.

[0081] Further, based on the first embodiment of the remote video screen interaction method of the present invention, a second embodiment of the remote video screen interaction method of the present invention is proposed. The step S200, the step of determining the target display frame based on the human figure frame, includes:

[0082] Step a, determine the starting coordinates of the person in the updated human figure frame in the original video image, and detect the size information of the human figure frame according to the starting coordinates of the person.

[0083] Step b, if there is a human figure frame, determine the edge positions in each direction of the original video image, and use the human figure frame with the smallest distance from the edge positions as the target human figure frame.

[0084] Step c, construct the target display frame according to the size information of at least one target human figure frame corresponding to the edge positions in each direction of the original video image.

[0085] In this embodiment, it should be noted that the preset original starting point refers to the starting point in the original video image, and by default, it is the upper left corner coordinate of the preset original image. The starting point coordinate of the person block diagram corresponding to a person is the upper left corner coordinate of the person block diagram with the preset original starting point as the reference coordinate. The size information of the person block diagram includes the width, height, and the center point coordinate of the person block diagram. The target display frame is the largest block diagram range formed by the person block diagrams. If there is a single person block diagram, it is directly used as the target person block diagram; when there are multiple person block diagrams, the edge positions in each direction of the original video image are determined, and the person block diagram with the smallest distance from each of these edge positions is used as the target person block diagram. For example, if the target display frame is a rectangular frame, the target display frame will include at least four target person block diagrams, namely the leftmost person block diagram, the rightmost person block diagram, the uppermost person block diagram, and the lowermost person block diagram. The leftmost person block diagram is the person block diagram with the leftmost position among all the person block diagrams; the rightmost person block diagram is the person block diagram with the rightmost position among all the person block diagrams; the uppermost person block diagram is the person block diagram with the uppermost position among all the person block diagrams; the lowermost person block diagram is the person block diagram with the lowermost position among all the person block diagrams. The target display frame is constructed based on the size information of at least one target person block diagram corresponding to the edge positions in each direction of the original video image. For example, if the target display frame is a rectangular frame, the ordinate of the starting point coordinate of the target display frame is the ordinate of the starting point coordinate of the uppermost person block diagram; the width of the target display frame is the sum of the distance between the abscissa of the starting point coordinate of the leftmost person block diagram and the abscissa of the starting point coordinate of the rightmost person block diagram plus the width of the rightmost person block diagram; the height of the target display frame is the sum of the distance between the ordinate of the starting point coordinate of the lowermost person block diagram and the ordinate of the starting point coordinate of the uppermost person block diagram plus the height of the lowermost person block diagram.

[0086] In this embodiment, the position of all person block diagrams is used to determine the target display frame, ensuring that all person block diagrams are included and excluding other non-person factors to the greatest extent, effectively ensuring the accuracy of the target display frame.

[0087] Further, after the step of monitoring whether the position of the person block diagram is updated, it further includes:

[0088] Step d, determining the current target display image of the remote end. If it is monitored that the position of the person block diagram does not change within the first preset time range, the current target display image is used as the target display image;

[0089] Step e, wherein, obtaining the size information of the current target display image, determining the current target display frame based on the position of the person block diagram corresponding to the end moment of the first preset time range, and determining whether the horizontal viewing angle of the current target display frame is greater than or equal to the preset angle;

[0090] Step f: If the horizontal view angle is greater than or equal to the preset angle, determine a first preset condition based on the size information of the current target display image and the center point coordinate information of the human figure frame. If, within the first preset time range, the human figure frame corresponding to the historical display frame satisfies the first preset condition, it is determined that the position of the human figure frame has not been updated within the first preset time range.

[0091] After step e, or, execute step g. If the horizontal view angle is less than the preset angle, determine a second preset condition based on the center point coordinate information of the human figure frame. If, within the first preset time range, the human figure frame corresponding to the historical display frame satisfies the second preset condition, it is determined that the position of the human figure frame has not been updated within the first preset time range.

[0092] In this embodiment, it should be noted that the current target display image refers to the image currently displayed on the remote end, that is, the image displayed in the video display area. In the remote video screen interaction, the image that can be displayed remotely can change according to the changes of the participants in the on-site area, and the basis for the change is the image currently displayed on the remote end. The current target display image is also the image displayed on the remote end when the human figure frame has not been updated. Among them, when there are no participants in the on-site area, the remote end directly displays the original image collected by the remote video screen interaction device in the on-site area. The historical display frame refers to the overall human frame including all human figure frames in the current target display image, and is the outermost boundary frame of all these human figure frames. The human figure frame is obtained in real time, and the position of the obtained human figure frame will be determined after the human figure frame is obtained.

[0093] In this embodiment, it should be noted that an update means that when there is a movement of the participants in the on-site area after the image is displayed on the remote end, the position of the corresponding human figure frame changes. To avoid excessive consumption of system resources caused by the remote display image following the real-time changes of the participants due to the too frequent slight amplitude changes of the participants, and since the slight movement of the participants has a relatively small impact on the overall meeting, such as slight actions like rubbing hands and shrugging shoulders, the update in this embodiment refers to the change of the human figure frame with a relatively large movement amplitude. Whether there is an update is determined through a certain position limit range. If the movement amplitude of the participant is within this certain limit range, it can be considered that the human figure frame corresponding to the participant has not been updated.

[0094] If the horizontal viewing angle is greater than or equal to a preset angle, and within a first preset time range, the person block diagram does not meet the first preset condition corresponding to the historical display box, it is determined that the position of the person block diagram has been updated within the first preset time range; if the horizontal viewing angle is less than the preset angle, and within the first preset time range, the person block diagram meets the second preset condition corresponding to the historical display box, it is determined that the position of the person block diagram has been updated within the first preset time range.

[0095] The person block diagrams corresponding to the end moment of the first preset time range include the leftmost person block diagram, the rightmost person block diagram, the uppermost person block diagram, and the lowermost person block diagram.

[0096] Specifically, the steps of determining the first preset condition according to the size information of the current target display image and the center point coordinate information of the person block diagram include:

[0097] Step f1, determine the height, width, and image starting point coordinates of the current target display image in the size information of the current target display image, and determine the historical leftmost person block diagram, historical rightmost person block diagram, historical uppermost person block diagram, and historical lowermost person block diagram among all the person block diagrams corresponding to the historical display box in the current target display image;

[0098] Step f2, use the fact that the ordinate of the center point of the uppermost person block diagram is greater than the ordinate of the image starting point coordinate and less than the sum of the ordinate of the center point of the historical uppermost person block diagram and the height of the historical uppermost person block diagram; the ordinate of the center point of the lowermost person block diagram is greater than the difference between the ordinate of the center point of the historical lowermost person block diagram and the height of the historical lowermost person block diagram and less than the sum of the ordinate of the image starting point coordinate and the height of the current target display image; the abscissa of the center point of the leftmost person block diagram is less than the sum of the abscissa of the center point of the historical leftmost person block diagram and the width of the historical leftmost person block diagram and greater than the abscissa of the image starting point coordinate; the ordinate of the center point of the rightmost person block diagram is greater than the difference between the abscissa of the center point of the historical rightmost person block diagram and the width of the historical rightmost person block diagram and less than the sum of the abscissa of the image starting point coordinate and the width of the current target display image as the first preset condition;

[0099] Specifically, the steps of determining the second preset condition according to the center point coordinate information of the person block diagram include:

[0100] Step g1, taking the absolute value of the difference between the ordinate of the center point of the uppermost person block diagram and the ordinate of the center point of the historical uppermost person block diagram to be less than the height of the historical uppermost person block diagram; taking the absolute value of the difference between the ordinate of the center point of the lowermost person block diagram and the ordinate of the center point of the historical lowermost person block diagram to be less than the height of the historical lowermost person block diagram; taking the absolute value of the difference between the abscissa of the center point of the leftmost person block diagram and the abscissa of the center point of the historical leftmost person block diagram to be less than the width of the historical leftmost person block diagram; and taking the absolute value of the difference between the abscissa of the center point of the rightmost person block diagram and the abscissa of the center point of the historical rightmost person block diagram to be less than the width of the historical rightmost person block diagram as the second preset condition.

[0101] This embodiment is described with a specific application scenario. In an application scenario, the current target display image at the remote end is denoted as V S , then in this current target display image, the corresponding relevant person block diagram information is: current target display image: image starting point coordinates V S (X Vs , Y Vs ), width is W Vs , height is H Vs ; the historical display frame in this current target display image: starting point coordinates P S (X Ps , Y Ps ), width is W Ps , height is H Ps ; historical uppermost person block diagram U S : starting point coordinates U S (X Us , Y Us ), width is W Us , height is H Us ; historical lowermost person block diagram: starting point coordinates D S (X Ds , Y Ds ), width is W Ds , height is H Ds ; historical leftmost person block diagram: starting point coordinates L S (X Ls , Y Ls ), width is W Ls , height is H Ls ; historical rightmost person block diagram: starting point coordinates R S (X Rs , Y Rs ), width is W Rs , height is H Rs .

[0102] The character block diagrams corresponding to the end moment of the first preset time range include the leftmost character block diagram, the rightmost character block diagram, the uppermost character block diagram, and the lowermost character block diagram. Based on the position information corresponding to the leftmost character block diagram, the rightmost character block diagram, the uppermost character block diagram, and the lowermost character block diagram, the current target display box P is determined. d , and the relevant character block diagram information is: the current target display box P d : starting coordinate P d (X Pd , Y Pd ), with a width of W Pd , and a height of H Pd ; the uppermost character block diagram U d : starting coordinate U d (X Ud , Y Ud ), with a width of W Ud , and a height of H Ud ; the lowermost character block diagram: starting coordinate D d (X Dd , Y Dd ), with a width of W Dd , and a height of H Dd ; the leftmost character block diagram: starting coordinate L d (X Ld , Y Ld ), with a width of W Ld , and a height of H Ld ; the rightmost character block diagram: starting coordinate R d (X Rd , Y Rd ), with a width of W Rd , and a height of H Rd .

[0103] n is a preset angle. In this embodiment, it is taken as 50, and the specific degree can be selected according to the actual situation without strict data size limitation.

[0104] When the horizontal viewing angle corresponding to the current target display box > n°, the first preset condition is satisfied when all of the following 4 conditions are met simultaneously:

[0105] 1. Y Vs < Y Ud < (Y Us + H Us );

[0106] 2. (Y Ds - H Ds ) < Y Dd < (Y Vs + H Vs );

[0107] 3. X Vs < XLd <(X Ls +W Ls );

[0108] 4、(X Rs -W Rs )<X Rd <(X Vs +W Vs )。

[0109] When the horizontal viewing angle corresponding to the current target display box ≤ n°, the second preset condition is satisfied when the following 4 conditions are simultaneously satisfied:

[0110] 1. ∣Y Ud -Y Us ∣<H Us ;

[0111] 2. ∣Y Dd -Y Ds ∣<H Ds ;

[0112] 3. ∣X Ld -X Ls ∣<W Ls ;

[0113] 4. ∣X Rd -X Rs ∣<W Rs 。

[0114] Based on the difference in the horizontal viewing angle of the current target display box, after satisfying the first preset condition or the second preset condition, it is considered that there is no update. At this time, no scaling action is required, and the detection logic is re-executed.

[0115] In this embodiment, it should be noted that if the position of the character block diagram does not update within the first preset time range, the current target display image is used as the target display image, which is applicable to two scenarios. One is the scenario where it is considered that no participants in the on-site area move, and the other is that in the on-site area, although some participants move, but within the first preset time range, the moving participants return to their positions before moving, which is also considered as no update, that is, within the first preset time range, the starting point displacement of the participant is within a certain range that satisfies the first preset condition or the second preset condition.

[0116] Furthermore, when it is detected that the position of the character block diagram is updated, the steps for determining the target display box based on the updated character block diagram include:

[0117] Step h, if the position of the character block diagram is updated within the first preset time range and the target display box is updated, then detect whether the updated target display box continues to be updated within the second preset time range;

[0118] Step i, if further update is needed, determine the current target display image of the remote end, and use the historical display frame in the current target display image as the target display frame;

[0119] Step j, if no further update is needed, execute the step of determining the starting coordinates of the person in the video original image according to the person block diagram after the position change, and detecting the size information of the person block diagram according to the starting coordinates of the person.

[0120] The person block diagrams corresponding to the end moment of the first preset time range include the leftmost person block diagram, the rightmost person block diagram, the uppermost person block diagram, and the lowermost person block diagram. Specifically, the step of detecting whether the updated target display frame continues to be updated within the second preset time range includes:

[0121] Step k, determine the positions of all person block diagrams corresponding to the end moment of the second preset time range, and determine the updated leftmost person block diagram, updated rightmost person block diagram, updated uppermost person block diagram, and updated lowermost person block diagram among all person block diagrams corresponding to the end moment of the second preset time range based on the positions of the person block diagrams corresponding to the end moment of the second preset time range;

[0122] Step l, determine whether the updated target display frame continues to be updated within the second preset time range based on whether the person block diagrams corresponding to the end moment of the second preset time range and the person block diagrams corresponding to the end moment of the first preset time range meet the third preset condition;

[0123] The third preset condition includes:

[0124] The offset value between the ordinate of the center point of the updated uppermost person block diagram and the ordinate of the center point of the uppermost person block diagram is greater than or equal to the uppermost height equal division ratio;

[0125] And / or, the offset value between the ordinate of the center point of the updated lowermost person block diagram and the ordinate of the center point of the lowermost person block diagram is greater than or equal to the lowermost height equal division ratio;

[0126] And / or, the offset value between the abscissa of the center point of the updated leftmost person block diagram and the abscissa of the center point of the leftmost person block diagram is greater than or equal to the leftmost width equal division ratio;

[0127] And / or, the offset value between the abscissa of the center point of the updated rightmost person block diagram and the abscissa of the center point of the rightmost person block diagram is greater than or equal to the rightmost width equal division ratio.

[0128] In this embodiment, it should be noted that the offset value refers to the absolute value of the difference. For example, the offset value between the vertical coordinate of the center point of the updated topmost person block diagram and the vertical coordinate of the center point of the topmost person block diagram is the absolute value of the difference between the vertical coordinate of the center point of the updated topmost person block diagram and the vertical coordinate of the center point of the topmost person block diagram. The equal division ratio refers to the amplitude reference value of the position movement of the person block diagram. For example, obtain the current target display image displayed on the current remote end, determine all the person block diagrams included in the current target display image, and determine the height of the topmost person block diagram among all the person block diagrams. The top height equal division ratio refers to the ratio of the height of the topmost person block diagram to a preset equal division ratio, indicating the amplitude of the position change of the topmost person block diagram; determine the height of the bottommost person block diagram among all the person block diagrams. The bottom height equal division ratio refers to the ratio of the height of the bottommost person block diagram to a preset equal division ratio, indicating the amplitude of the position change of the bottommost person block diagram; determine the width of the leftmost person block diagram among all the person block diagrams. The left width equal division ratio refers to the ratio of the width of the leftmost person block diagram to a preset equal division ratio, indicating the amplitude of the position change of the leftmost person block diagram; determine the width of the rightmost person block diagram among all the person block diagrams. The right height equal division ratio refers to the ratio of the width of the rightmost person block diagram to a preset equal division ratio, indicating the amplitude of the position change of the rightmost person block diagram.

[0129] In an application scenario, obtain the current target display image displayed on the current remote end, and the current target display image is denoted as V S Then, in the current target display image, the corresponding relevant person block diagram information is: current target display image: image starting coordinate V S (X Vs , Y Vs ), with a width of W Vs , and a height of H Vs ; the historical display box in the current target display image: starting coordinate PS(X Ps , Y Ps ), with a width of W Ps , and a height of H Ps ; historical topmost person block diagram U S : starting coordinate U S (X Us , Y Us ), with a width of W Us , and a height of H Us ; historical bottommost person block diagram: starting coordinate D S (X Ds , Y Ds ), with a width of W Ds , and a height of H Ds ; historical leftmost person block diagram: starting coordinate L S (X Ls , Y Ls), with a width of W Ls , with a height of H Ls ; Historical rightmost person block diagram: starting coordinates R S (X Rs , Y Rs ), with a width of W Rs , with a height of H Rs .

[0130] The person block diagrams corresponding to the end time of the first preset time range include the leftmost person block diagram, the rightmost person block diagram, the uppermost person block diagram, and the lowermost person block diagram. Based on the position information corresponding to the leftmost person block diagram, the rightmost person block diagram, the uppermost person block diagram, and the lowermost person block diagram, the current target display frame P is determined d , and the relevant person block diagram information is: current target display frame P d : starting coordinates P d (X Pd , Y Pd ), with a width of W Pd , with a height of H Pd ; Uppermost person block diagram U d1 : starting coordinates U d1 (X Ud1 , Y Ud1 ), with a width of W Ud1 , with a height of H Ud1 ; Lowermost person block diagram: starting coordinates D d1 (X Dd1 , Y Dd1 ), with a width of W Dd1 , with a height of H Dd1 ; Leftmost person block diagram: starting coordinates L d1 (X Ld1 , Y Ld1 ), with a width of W Ld1 , with a height of H Ld1 ; Rightmost person block diagram: starting coordinates R d1 (X Rd1 , Y Rd1 ), with a width of W Rd1 , with a height of H Rd1 .

[0131] n is a preset angle. In this embodiment, it is taken as 50, and the specific degree can be selected according to the actual situation without strict data size limitation

[0132] During the first preset time range, the position of the person block diagram is updated, and the judgment basis for the update of the target display frame is:

[0133] When the horizontal viewing angle corresponding to the current target display frame > n°, any one or more of the following 4 conditions are satisfied:

[0134] 1. (Y Ud1 ≤ Y Vs ) || (Y Ud1 ≥ (Y Us + H Us ));

[0135] 2. (Y Dd1 ≤ (Y Ds - H Ds )) || (Y Dd1 ≥ (Y Vs + H Vs ));

[0136] 3. (X Ld1 ≤ X Vs ) || (X Ld1 ≥ (X Ls + W Ls ));

[0137] 4. (X Rd1 ≤ (X Rs - W Rs )) || (X Rd1 ≥ (X Vs + W Vs ).

[0138] When the horizontal viewing angle corresponding to the current target display box ≤ n°, any one or more of the following 4 conditions are satisfied:

[0139] 1. |Y Ud - Y Us | ≥ H Us ;

[0140] 2. |Y Dd - Y Ds | ≥ H Ds ;

[0141] 3. |X Ld - X Ls | ≥ W Ls ;

[0142] 4. |X Rd - X Rs | ≥ W Rs .

[0143] From T 0sFrom a certain moment, after determining that there is an update in the position of the character block diagram within the first preset time range and the target display box is updated, it is detected whether the updated target display box continues to be updated within the second preset time range. The positions of all character block diagrams corresponding to the end moment of the second preset time range are determined. Based on the positions of each character block diagram corresponding to the end moment of the second preset time range, the leftmost updated character block diagram, the rightmost updated character block diagram, the uppermost updated character block diagram, and the lowermost updated character block diagram among all character block diagrams corresponding to the end moment of the second preset time range are determined, and the updated display box P is determined based on the leftmost updated character block diagram, the rightmost updated character block diagram, the uppermost updated character block diagram, and the lowermost updated character block diagram d2 , P d2 's starting coordinate P d2 (X Pd2 , Y Pd2 ), with a width of W Pd2 , and a height of H Pd2 ; the uppermost updated character block diagram U d2 : starting coordinate U d2 (X Ud2 , Y Ud2 ), with a width of W Ud2 , and a height of H Ud2 ; the lowermost updated character block diagram: starting coordinate D d2 (X Dd2 , Y Dd2 ), with a width of W Dd2 , and a height of H Dd2 ; the leftmost updated character block diagram: starting coordinate L d2 (X Ld2 , Y Ld2 ), with a width of W Ld2 , and a height of H Ld2 ; the rightmost updated character block diagram: starting coordinate R d2 (X Rd2 , Y Rd2 ), with a width of W Rd2 , and a height of H Rd2 .

[0144] If the changed character block diagram meets the third preset condition, it is determined that the updated target display box continues to be updated within the second preset time range, and the historical display box is used as the target display box;

[0145] N is an equal division ratio, which is a preset constant. In this embodiment, 8 is taken, and the specific degree can be selected according to the actual situation without strict data size limitation.

[0146] The third preset condition is that within the preset time range, at least one of the following 4 conditions is met:

[0147] 1.

[0148] 2、

[0149] 3、

[0150] 4、

[0151] Further, the step of converting the target display frame into a target display image according to a preset resolution includes:

[0152] Step m, obtaining the current target display image of the remote end, and determining the height of the current target display image, the width of the current target display image, and the image starting point coordinates of the current target display image;

[0153] Step n, determining the historical display frame in the current target display image, and determining the historical leftmost character block diagram, the historical rightmost character block diagram, the historical uppermost character block diagram, and the historical lowermost character block diagram among all the character block diagrams corresponding to the historical display frame;

[0154] Step o, when the character block diagram satisfies the preset magnification condition corresponding to the historical display frame, magnifying the target display frame according to the preset resolution, and generating a target display image based on the magnified target display frame;

[0155] The preset magnification condition includes:

[0156] Determining the current target display frame based on the position of the character block diagram, and determining whether the horizontal viewing angle of the current target display frame is greater than or equal to a preset angle;

[0157] When the horizontal viewing angle is greater than or equal to the preset angle, the ordinate of the center point of the uppermost character block diagram is less than or equal to the ordinate of the current target display image, and / or the ordinate of the center point of the lowermost character block diagram is greater than or equal to the sum of the ordinate of the image starting point coordinate and the height of the current target display image, and / or the abscissa of the center point of the leftmost character block diagram is less than or equal to the abscissa of the image starting point coordinate, and / or the ordinate of the center point of the rightmost character block diagram is greater than or equal to the sum of the abscissa of the image starting point coordinate and the width of the current target display image;

[0158] When the horizontal view angle is less than or equal to a preset angle, the ordinate of the center point of the uppermost person block diagram is less than or equal to the difference between the ordinate of the center point of the historical uppermost person block diagram and the height of the historical uppermost person block diagram, and / or the ordinate of the center point of the lowermost person block diagram is greater than or equal to the sum of the ordinate of the center point of the historical lowermost person block diagram and the height of the historical lowermost person block diagram, and / or the abscissa of the center point of the leftmost person block diagram is less than or equal to the difference between the abscissa of the center point of the historical leftmost person block diagram and the width of the historical leftmost person block diagram, and / or the ordinate of the center point of the rightmost person block diagram is greater than or equal to the sum of the abscissa of the center point of the historical rightmost person block diagram and the width of the historical rightmost person block diagram.

[0159] Further, the step of converting the target display frame into a target display image according to a preset resolution includes:

[0160] The step of converting the target display frame into a target display image according to a preset resolution includes:

[0161] Step p, if the person block diagram corresponding to the historical display frame satisfies the preset reduction condition, reduce the target display frame according to the preset resolution, and generate a target display image based on the reduced target display frame;

[0162] The preset reduction condition includes:

[0163] The ordinate of the center point of the uppermost person block diagram is greater than or equal to the sum of the ordinate of the center point of the historical uppermost person block diagram and the height of the historical uppermost person block diagram, and / or the ordinate of the center point of the lowermost person block diagram is less than or equal to the difference between the ordinate of the center point of the historical lowermost person block diagram and the height of the historical uppermost person block diagram, and / or the abscissa of the center point of the leftmost person block diagram is greater than or equal to the sum of the abscissa of the center point of the historical leftmost person block diagram and the width of the historical leftmost person block diagram, and / or the ordinate of the center point of the rightmost person block diagram is less than or equal to the sum of the abscissa of the center point of the historical rightmost person block diagram and the width of the historical rightmost person block diagram.

[0164] In this embodiment, it should be noted that the updated target display frame is not updated continuously within the second preset time range. That is, after the participants in the on-site area move, they do not move again immediately, but maintain the moved position within a certain time range. At this time, since the movement amplitude is large and the maintenance time is long, the picture displayed at the remote end needs to be updated. The target display frame determined to be converted into the target display image includes the updated position information of the relevant positions after the participants move. The target display frame at this time is not the original video image, but the updated display frame after including the position movement related information. The target display frame at this time has two states compared with the historical display frame in the current target display image at the remote end, one is the enlarged state and the other is the reduced state. These two states are the relative states of the display frame after resolution adjustment to adapt to the resolution requested by the computer side compared with the historical display frame before the current moment due to the change of the outermost boundary bounding box corresponding in the display frame after the participants move. There is no continuous update, that is, there is only a change in the target display frame compared with the original video image. The relative state of the target display frame compared with the historical display frame in the current target display image can be determined by using the change of each person block diagram in the target display frame compared with each person block diagram in the historical display frame before the current moment. The state of the target display frame is determined based on the judgment process of whether the updated target display frame is updated continuously within the second preset time range.

[0165] When determining whether the updated target display box continues to be updated within the second preset time range, the relative state of different target display boxes compared to the historical display box in the current target display image at the remote end defines different preset time ranges. When determining the relative state of the target display box compared to the historical display box by using the changes of the person block diagrams at the current moment compared to the person block diagrams of the historical display box, when determining that the relative state of the target display box compared to the historical display box is an enlarged state, a third preset time range is defined. When the updated target display box does not continue to be updated within the second preset time range, it is further determined whether to continue to be updated within this third preset time range; when determining that the relative state of the target display box compared to the historical display box is a reduced state, a fourth preset time range is defined. When the updated target display box does not continue to be updated within the second preset time range, it is further determined whether to continue to be updated within this fourth preset time range. Among them, the third preset time range is less than the fourth preset time range. Determining different preset time range limits according to different target display box states reduces error generation and improves the accuracy of the target display image. For example, there are 3 people participating in the meeting in the on-site area. If 1 person comes in, then there are 4 people participating in the meeting at this time, and the corresponding target display box state is an enlarged state. In the enlarged state, the original 3 people always exist and will not change. That is, even if the 4th person who comes in is mis-identified, it will not cause a large interference to the display image; however, if there are 3 people participating in the meeting in the on-site area and 1 person leaves, then there are 2 people participating in the meeting at this time, and the target display box state is a reduced state. One person needs to be deleted from the original target display box. If directly deleted, there may be a mis-deletion of the person, and the mis-deletion of the person will have a greater impact. Therefore, the fourth preset time range defined for the reduced state should be greater than the third preset time range defined for the enlarged state to reduce the possibility of error generation. It can be understood that in another embodiment, when determining whether the updated target display box continues to be updated within the second preset time range, the relative state of different target display boxes compared to the historical display box defines the same time range.

[0166] In the application scenario of an embodiment, obtain the current target display image displayed on the current remote end, and the current target display image is denoted as V S , then in this current target display image, the corresponding relevant person block diagram information is: current target display image: image starting coordinate V S (X Vs , Y Vs ), with a width of W Vs , and a height of H Vs ; the historical display box in this current target display image: starting coordinate P S (X Ps , Y Ps ), with a width of W Ps , and a height of HPs ; The topmost historical character block diagram U S : Starting coordinate U S (X Us , Y Us ), with width W Us , and height H Us ; The bottommost historical character block diagram: Starting coordinate D S (X Ds , Y Ds ),with width W Ds , and height H Ds ; The leftmost historical character block diagram: Starting coordinate L S (X Ls , Y Ls ),with width W Ls , and height H Ls ; The rightmost historical character block diagram: Starting coordinate R S (X Rs , Y Rs ),with width W Rs , and height H Rs .

[0167] The character block diagram includes the leftmost, rightmost, topmost, and bottommost character block diagrams. Based on the position information corresponding to the leftmost, rightmost, topmost, and bottommost character block diagrams, the current target display box P is determined d , and the relevant character block diagram information is: The current target display box P d : Starting coordinate P d (X Pd , Y Pd ),with width W Pd , and height H Pd ; The topmost character block diagram U d1 : Starting coordinate U d1 (X Ud1 , Y Ud1 ),with width W Ud1 , and height H Ud1 ; The bottommost character block diagram: Starting coordinate D d1 (X Dd1 , Y Dd1 ),with width W Dd1 , and height H Dd1 ; The leftmost character block diagram: Starting coordinate L d1 (X Ld1 , Y Ld1 ),with width W Ld1 , and height H Ld1 ; The rightmost character block diagram: Starting coordinate R d1 (X Rd1 , Y Rd1 ),with width WRd1 , with a height of H Rd1 .

[0168] n is a preset angle, which is taken as 50 in this embodiment. The specific degree can be selected according to the actual situation and there is no strict limit on the data size.

[0169] The preset magnification conditions include:

[0170] When the horizontal viewing angle corresponding to the current target display box > n°, any one or more of the following 4 conditions are satisfied:

[0171] 1. Y Ud1 ≤ Y Vs ;

[0172] 2. Y Dd1 ≥ (Y Vs + H Vs );

[0173] 3. X Ld1 ≤ X Vs ;

[0174] 4. X Rd1 ≥ (X Vs + W Vs ).

[0175] When the horizontal viewing angle corresponding to the current target display box ≤ n°, any one or more of the following 4 conditions are satisfied:

[0176] 1. Y Ud1 ≤ (Y Us - H Us );

[0177] 2. Y Dd1 ≥ (Y Ds + H Ds );

[0178] 3. X Ld1 ≤ (X Ls - W Ls );

[0179] 4. X Rd1 ≥ (X Rs + W Rs ).

[0180] n is a preset angle, which is taken as 50 in this embodiment. The specific degree can be selected according to the actual situation and there is no strict limit on the data size.

[0181] The preset reduction conditions include:

[0182] The first case:

[0183] All of the following 4 conditions are satisfied simultaneously:

[0184] 1. Y Ud1 ≥(Y Us +H Us );

[0185] 2. Y Dd1 ≤(Y Ds -H Ds );

[0186] 3. X Ld1 ≥(X Ls +W Ls );

[0187] 4. X Rd1 ≤(X Rs -W Rs )。

[0188] The second case:

[0189] First, satisfy k of the following 4 conditions, at least one (corresponding to the subscript, that is, the human figure diagram on the outermost bounding box: k of the topmost human figure diagram, the bottommost human figure diagram, the leftmost human figure diagram, and the rightmost human figure diagram):

[0190] 1. Y Ud1 ≥(Y Us +H Us );

[0191] 2. Y Dd1 ≤(Y Ds -H Ds );

[0192] 3. X Ld1 ≥(X Ls +W Ls );

[0193] 4. X Rd1 ≤(X Rs -W Rs )。

[0194] Second, the remaining (4 - k) conditions (corresponding to the subscript, that is, the human figure diagram on the outermost bounding box: (4 - k) of the topmost human figure diagram, the bottommost human figure diagram, the leftmost human figure diagram, and the rightmost human figure diagram) satisfy the following conditions:

[0195] When the horizontal viewing angle corresponding to the historical display box > n°, satisfy the following (4 - k) conditions simultaneously:

[0196] 1. Y Vs <Y Ud1 <(Y Us +H Us );

[0197] 2. (Y Ds -H Ds )(Y Dd1 ((Y Vs +H Vs );

[0198] 3. X Vs (X Ld1 ((X Ls +W Ls );

[0199] 4. (X Rs -W Rs )(X Rd1 ((X Vs +W Vs )。

[0200] When the horizontal viewing angle corresponding to the historical display box ≤ n°, the following (4 - k) conditions are simultaneously satisfied:

[0201] 1. ∣Y Ud1 -Y Us ∣ < H Us ;

[0202] 2. ∣Y Dd1 -Y Ds ∣ < H Ds ;

[0203] 3. ∣X Ld1 -X Ls ∣ < W Ls ;

[0204] 4. ∣X Rd1 -X Rs ∣ < W Rs 。

[0205] From time T 0s on, when it is determined that the character block diagram corresponding to the character block diagram in the historical display box satisfies the preset magnification condition, determine all the character block diagrams at the real - time moment, and determine the real - time left - most character block diagram, real - time right - most character block diagram, real - time top - most character block diagram, and real - time bottom - most character block diagram among all the character block diagrams, and determine the real - time display box P dn , P dn 's starting coordinate P dn (X Pdn , Y Pdn ), with a width of W Pdn , and a height of H Pdn ; The real - time top - most character block diagram U dn : Starting coordinate U dn (XUdn , Y Udn ), with a width of W Udn , with a height of H Udn ; Real-time bottommost person block diagram: starting coordinates D dn (X Ddn , Y Ddn ), with a width of W Ddn , with a height of H Ddn ; Real-time leftmost person block diagram: starting coordinates L dn (X Ldn , Y Ldn ), with a width of W Ldn , with a height of H Ldn ; Real-time rightmost person block diagram: starting coordinates R dn (X Rdn , Y Rdn ), with a width of W Rdn , with a height of H Rdn .

[0206] N is an equal division ratio, which is a preset constant. In this embodiment, it is taken as 8. The specific degree can be selected according to the actual situation, and there is no strict data size limit.

[0207] In different target display box conversion methods, when it is determined that the updated target display box has not been updated continuously within the second preset time range, for the enlarged state, it will continue to judge whether there is an update within the third preset time range. If it is determined that there is no update within the third preset time range, at this time, the target display box will be enlarged according to the preset resolution, and a target display image will be generated based on the enlarged target display box. The judgment condition for whether there is an update within the third preset time range is as follows:

[0208] In the enlarged state, if the following 4 conditions are simultaneously met within the third preset time range, it is determined that there is no continuous update within the third preset time range:

[0209] 1.

[0210] 2.

[0211] 3.

[0212] 4.

[0213] For the reduced state, it will continue to judge whether there is an update within the fourth preset time range. If it is determined that there is no update within the fourth preset time range, at this time, the target display box will be reduced according to the preset resolution, and a target display image will be generated based on the reduced target display box. The judgment condition for whether there is an update within the fourth preset time range is as follows:

[0214] In the minimized state, if the following four conditions are simultaneously satisfied within the fourth preset time range, it is determined that there is no further update within the fourth preset time range:

[0215] 1.

[0216] 2.

[0217] 3.

[0218] 4.

[0219] Among them, the third preset time range is less than the fourth preset time range. For example, the third preset time range is 3s and the fourth preset time range is 10s. The third preset time range and the fourth preset time range are calculated starting from the stop moment of the second preset time range. After it is determined that there is no further update within the second preset time range, continue to monitor the third preset time range or the fourth preset time range based on different target display frame conversion methods to reduce image errors and improve the stability of image display.

[0220] In this embodiment, it should be noted that when the position of the character block diagram is updated within the first preset time range and the updated target display frame continues to be updated within the second preset time range, the target display image on the computer side will transition from the display image corresponding to the historical display frame to the display image corresponding to the target display frame. All transition actions have a uniformly scaled transition animation, and the transition actions are completed within 0.5 - 2s. The completion time of the transition actions can also be changed according to actual requirements.

[0221] In addition, the present invention also provides a remote video screen interaction device, characterized in that the remote video screen interaction device includes a memory, a processor, and a remote video screen interaction program stored on the memory and executable on the processor, wherein: when the remote video screen interaction program is executed by the processor, it implements the remote video screen interaction method described in various embodiments of the present invention.

[0222] In addition, the present invention also provides a computer-readable storage medium, on which a remote video screen interaction program is stored. The computer-readable storage medium can be Figure 1 the memory 20 in the terminal, or at least one of ROM (Read-Only Memory), RAM (Random Access Memory), magnetic disk, and optical disc. The computer-readable storage medium includes several instructions for causing a remote video screen interaction device with a processor to execute the remote video screen interaction method described in various embodiments of the present invention.

[0223] It is understood that in the description of this specification, the descriptions referring to terms such as "one embodiment", "another embodiment", "other embodiments", or "the first embodiment to the Nth embodiment", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples.

[0224] It should be noted that in this text, the term "comprising", "including", or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such process, method, article, or system. Without further limitation, an element defined by the statement "comprising one..." does not exclude the existence of additional identical elements in the process, method, article, or system comprising such element.

[0225] The serial numbers of the above embodiments of the present invention are only for description and do not represent the superiority or inferiority of the embodiments.

[0226] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium as described above (such as ROM / RAM, magnetic disk, optical disc), and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention.

[0227] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structural or equivalent process transformation made by using the content of the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. A remote video screen interaction method, characterized in that, The remote video screen interaction method includes the following steps: Obtain the original video image in real time, and determine the person block diagram in the original video image; Monitor whether the position of the person block diagram is updated. If it is detected that the position of the person block diagram is updated, determine the starting coordinates of the person in the original video image of the updated person block diagram, and detect the size information of the person block diagram according to the starting coordinates of the person. If there is a person block diagram, determine the edge positions in each direction of the original video image, and use the person block diagram with the smallest distance from the edge positions as the target person block diagram. Construct a target display frame according to the size information of at least one target person block diagram corresponding to the edge positions in each direction of the original video image. Wherein, the starting coordinates are determined based on the starting coordinates of the original video image as the reference coordinates, the target display frame is determined using the size information of the target person block diagram, and the target display frame is not equal to the overall range displayed in the video display area. The position of the person block diagram being updated means that when the participants in the on-site area move after their images are displayed on the remote end, the position of the person block diagram corresponding to the participants moves significantly; After the step of constructing the target display frame according to the size information of at least one target person block diagram corresponding to the edge positions in each direction of the original video image, it includes: If the position of the person block diagram is updated within the first preset time range and the target display frame is updated, detect whether the updated target display frame continues to be updated within the second preset time range. If it continues to be updated, determine the current target display image at the remote end, and use the historical display frame in the current target display image as the target display frame. If it does not continue to be updated, execute the step of determining the starting coordinates of the person in the original video image of the person block diagram according to the changed position of the person block diagram, and detect the size information of the person block diagram according to the starting coordinates of the person; Convert the target display frame into a target display image according to the preset resolution, and push the target display image to the remote end.

2. The remote video picture interaction method according to claim 1, characterized in that, After the step of monitoring whether the position of the person block diagram is updated, it further includes: Determine the current target display image at the remote end. If it is detected that the position of the person block diagram does not change within the first preset time range, use the current target display image as the target display image; Wherein, obtain the size information of the current target display image, determine the current target display frame based on the position of the person block diagram corresponding to the end moment of the first preset time range, and determine whether the horizontal viewing angle of the current target display frame is greater than or equal to the preset angle; If the horizontal viewing angle is greater than or equal to the preset angle, determine the first preset condition according to the size information of the current target display image and the central point coordinate information of the person block diagram. If within the first preset time range, the person block diagram and the person block diagram corresponding to the historical display frame meet the first preset condition, determine that the position of the person block diagram does not change within the first preset time range; If the horizontal viewing angle is less than a preset angle, a second preset condition is determined according to the center point coordinate information of the human figure block diagram. If, within a first preset time range, the human figure block diagram corresponding to the historical display box satisfies the second preset condition, it is determined that the position of the human figure block diagram has not been updated within the first preset time range.

3. The remote video screen interaction method according to claim 2, characterized in that The human figure block diagrams corresponding to the end time of the first preset time range include the leftmost human figure block diagram, the rightmost human figure block diagram, the uppermost human figure block diagram, and the lowermost human figure block diagram. The step of determining the first preset condition according to the size information of the current target display image and the center point coordinate information of the human figure block diagram includes: Determining the height of the current target display image, the width of the current target display image, and the image starting point coordinates in the size information of the current target display image, and determining the historical leftmost human figure block diagram, the historical rightmost human figure block diagram, the historical uppermost human figure block diagram, and the historical lowermost human figure block diagram among all the human figure block diagrams corresponding to the historical display box in the current target display image; Taking that the vertical coordinate of the center point of the uppermost human figure block diagram is greater than the vertical coordinate of the image starting point and less than the sum of the vertical coordinate of the center point of the historical uppermost human figure block diagram and the height of the historical uppermost human figure block diagram; the vertical coordinate of the center point of the lowermost human figure block diagram is greater than the difference between the vertical coordinate of the center point of the historical lowermost human figure block diagram and the height of the historical lowermost human figure block diagram and less than the sum of the vertical coordinate of the image starting point and the height of the current target display image; the horizontal coordinate of the center point of the leftmost human figure block diagram is less than the sum of the horizontal coordinate of the center point of the historical leftmost human figure block diagram and the width of the historical leftmost human figure block diagram and greater than the horizontal coordinate of the image starting point; the horizontal coordinate of the center point of the rightmost human figure block diagram is greater than the difference between the horizontal coordinate of the center point of the historical rightmost human figure block diagram and the width of the historical rightmost human figure block diagram and less than the sum of the horizontal coordinate of the image starting point and the width of the current target display image as the first preset condition; The step of determining the second preset condition according to the center point coordinate information of the human figure block diagram includes: Taking that the absolute value of the difference between the vertical coordinate of the center point of the uppermost human figure block diagram and the vertical coordinate of the center point of the historical uppermost human figure block diagram is less than the height of the historical uppermost human figure block diagram; the absolute value of the difference between the vertical coordinate of the center point of the lowermost human figure block diagram and the vertical coordinate of the center point of the historical lowermost human figure block diagram is less than the height of the historical lowermost human figure block diagram; the absolute value of the difference between the horizontal coordinate of the center point of the leftmost human figure block diagram and the horizontal coordinate of the center point of the historical leftmost human figure block diagram is less than the width of the historical leftmost human figure block diagram; the absolute value of the difference between the horizontal coordinate of the center point of the rightmost human figure block diagram and the horizontal coordinate of the center point of the historical rightmost human figure block diagram is less than the width of the historical rightmost human figure block diagram as the second preset condition.

4. The remote video picture interaction method according to claim 1, characterized in that The human figure block diagrams corresponding to the end time of the first preset time range include the leftmost human figure block diagram, the rightmost human figure block diagram, the uppermost human figure block diagram, and the lowermost human figure block diagram. The step of detecting whether the updated target display box continues to be updated within a second preset time range includes: Determine the positions of all character frames corresponding to the end moment of the second preset time range, and determine the updated leftmost character frame, updated rightmost character frame, updated topmost character frame, and updated bottommost character frame among all character frames corresponding to the end moment of the second preset time range based on the positions of each character frame corresponding to the end moment of the second preset time range; Determine whether the updated target display frame continues to be updated within the second preset time range based on whether the character frames corresponding to the end moment of the second preset time range satisfy the third preset condition compared with the character frames corresponding to the end moment of the first preset time range; The third preset condition includes: The offset value between the ordinate of the center point of the updated topmost character frame and the ordinate of the center point of the topmost character frame is greater than or equal to the top height equal division ratio; And / or, the offset value between the ordinate of the center point of the updated bottommost character frame and the ordinate of the center point of the bottommost character frame is greater than or equal to the bottom height equal division ratio; And / or, the offset value between the abscissa of the center point of the updated leftmost character frame and the abscissa of the center point of the leftmost character frame is greater than or equal to the left width equal division ratio; And / or, the offset value between the abscissa of the center point of the updated rightmost character frame and the abscissa of the center point of the rightmost character frame is greater than or equal to the right width equal division ratio.

5. The remote video screen interaction method according to claim 1, wherein The character frame includes a leftmost character frame, a rightmost character frame, a topmost character frame, and a bottommost character frame. The step of converting the target display frame into a target display image according to the preset resolution includes: Obtain the current target display image of the remote end, and determine the height of the current target display image, the width of the current target display image, and the image starting coordinate of the current target display image; Determine the historical display frame in the current target display image, and determine the historical leftmost character frame, historical rightmost character frame, historical topmost character frame, and historical bottommost character frame among all character frames corresponding to the historical display frame; If the character frame satisfies the preset magnification condition compared with the character frame corresponding to the historical display frame, magnify the target display frame according to the preset resolution, and generate a target display image based on the magnified target display frame; The preset magnification condition includes: Determine the current target display frame based on the position of the character frame, and determine whether the horizontal viewing angle of the current target display frame is greater than or equal to a preset angle; When the horizontal viewing angle is greater than or equal to the preset angle, the ordinate of the center point of the topmost character frame is less than or equal to the ordinate of the current target display image, and / or the ordinate of the center point of the bottommost character frame is greater than or equal to the sum of the ordinate of the image starting coordinate and the height of the current target display image, and / or the abscissa of the center point of the leftmost character frame is less than or equal to the abscissa of the image starting coordinate, and / or the abscissa of the center point of the rightmost character frame is greater than or equal to the sum of the abscissa of the image starting coordinate and the width of the current target display image; When the horizontal viewing angle is less than or equal to a preset angle, the ordinate of the center point of the uppermost character block diagram is less than or equal to the difference between the ordinate of the center point of the historical uppermost character block diagram and the height of the historical uppermost character block diagram, and / or the ordinate of the center point of the lowermost character block diagram is greater than or equal to the sum of the ordinate of the center point of the historical lowermost character block diagram and the height of the historical lowermost character block diagram, and / or the abscissa of the center point of the leftmost character block diagram is less than or equal to the difference between the abscissa of the center point of the historical leftmost character block diagram and the width of the historical leftmost character block diagram, and / or the ordinate of the center point of the rightmost character block diagram is greater than or equal to the sum of the abscissa of the center point of the historical rightmost character block diagram and the width of the historical rightmost character block diagram.

6. The remote video screen interaction method according to claim 5, wherein The step of converting the target display frame into a target display image according to a preset resolution includes: When the character block diagram and the character block diagram corresponding to the historical display frame meet the preset reduction condition, reducing the target display frame according to the preset resolution, and generating a target display image based on the reduced target display frame; The preset reduction condition includes: The ordinate of the center point of the uppermost character block diagram is greater than or equal to the sum of the ordinate of the center point of the historical uppermost character block diagram and the height of the historical uppermost character block diagram, and / or the ordinate of the center point of the lowermost character block diagram is less than or equal to the difference between the ordinate of the center point of the historical lowermost character block diagram and the height of the historical uppermost character block diagram, and / or the abscissa of the center point of the leftmost character block diagram is greater than or equal to the sum of the abscissa of the center point of the historical leftmost character block diagram and the width of the historical leftmost character block diagram, and / or the ordinate of the center point of the rightmost character block diagram is less than or equal to the sum of the abscissa of the center point of the historical rightmost character block diagram and the width of the historical rightmost character block diagram.

7. A remote video screen interaction device, characterized in that, The remote video picture interaction device includes: a memory, a processor, and a remote video picture interaction program stored on the memory and executable on the processor, and the remote video picture interaction program is configured to implement the steps of the remote video picture interaction method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, A remote video picture interaction program is stored on the computer-readable storage medium, and when the remote video picture interaction program is executed by a processor, it implements the steps of the remote video picture interaction method according to any one of claims 1 to 6.

Images