Intelligent following method, device, terminal device and readable storage medium

Abstract

The embodiment of the present invention discloses an intelligent following method, device, terminal device and readable storage medium, the method comprising obtaining the first center coordinate of a target image in a current frame; calculating the distance between the center of the target image and the center of the display screen according to the first center coordinate and the second center coordinate of the display screen; when the distance is greater than a preset distance threshold, moving the target image so that the center of the target image moves to the center of the display screen. The technical solution moves the first center coordinate of the acquired target image to the second center coordinate of the display screen, so as to display the image acquired by the camera device in the central area of ​​the display screen, without using a hardware pan-tilt that requires a mechanical rotating device, thereby avoiding the vibration, noise and other effects caused by the rotation of the hardware pan-tilt.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an intelligent following method, device, terminal equipment and readable storage medium. Background Art

[0002] Under the control voltage (voltage output by the pan / tilt controller), the traditional intelligent pan / tilt controls the steering device with steering function to rotate horizontally and vertically, so that the camera can aim at and capture the required observation target in a large range, and use mechanical means to realize the camera to rotate according to the moving direction of the observation target to expand the camera's observation angle. Using mechanical means to control the steering device to rotate horizontally and vertically is not only prone to image jitter problems, resulting in blurred images, but also the steering device has a limited service life, resulting in a waste of resources. Summary of the invention

[0003] In view of the above problems, the present invention proposes a smart following method, apparatus, terminal device and readable storage medium.

[0004] An embodiment of the present invention provides an intelligent following method, the method comprising:

[0005] Get the first center coordinates of the target image in the current frame;

[0006] Calculate the distance between the center of the target image and the center of the display screen according to the first center coordinates and the second center coordinates of the display screen;

[0007] When the distance is greater than a preset distance threshold, the target image is moved so that the center of the target image moves to the center of the display screen.

[0008] Further, obtaining the first center coordinates of the target image in the current frame includes:

[0009] Performing face recognition on the target image in the current frame;

[0010] If the target image includes a face image, the face image is annotated using a face frame, and the center position of the face frame is used as the first center coordinate.

[0011] Further, the moving the target image so that the center of the target image moves to the center of the display screen includes:

[0012] During the movement, real-time monitoring of the dynamic distance between the center of the target image and the center of the display screen in a preset moving direction;

[0013] determining a moving speed according to the dynamic distance;

[0014] The target image is moved according to the preset moving direction and the moving speed so that the center of the target image moves to the center of the display screen.

[0015] Furthermore, the preset movement direction includes lateral movement and / or vertical movement;

[0016] When the preset moving direction is lateral movement:

[0017] During the movement, real-time monitoring of the horizontal dynamic distance between the central horizontal coordinate of the target image and the central horizontal coordinate of the display screen;

[0018] determining a lateral movement speed according to the lateral dynamic distance;

[0019] Moving the target image according to the lateral moving speed so that the center of the target image moves laterally to a position corresponding to the central horizontal coordinate of the display screen;

[0020] When the preset moving direction is vertical movement:

[0021] During the movement, real-time monitoring of the vertical dynamic distance between the central vertical coordinate of the target image and the central vertical coordinate of the display screen;

[0022] determining a vertical moving speed according to the vertical dynamic distance;

[0023] The target image is moved according to the vertical moving speed so that the center of the target image moves vertically to a position corresponding to the central longitudinal coordinate of the display screen.

[0024] Further, determining the moving speed according to the dynamic distance includes:

[0025] Obtaining the resolution of the camera device;

[0026] The moving speed is determined according to the resolution and the dynamic distance.

[0027] The above-mentioned determining the moving speed according to the dynamic distance includes:

[0028] Obtaining the field of view angle of the camera device;

[0029] The moving speed is determined according to the field of view angle and the dynamic distance.

[0030] Another embodiment of the present invention provides an intelligent following device, the device comprising:

[0031] An acquisition module, used for acquiring the first center coordinates of the target image in the current frame;

[0032] a calculation module, configured to calculate the distance between the center of the target image and the center of the display screen according to the first center coordinates and the second center coordinates of the display screen;

[0033] The moving module is used to move the target image so that the center of the target image moves to the center of the display screen when the distance is greater than a preset distance threshold.

[0034] Furthermore, the acquisition module includes:

[0035] A recognition unit, configured to perform face recognition on the target image in the current frame;

[0036] The determination unit is configured to use, if the target image includes a face image, the position of the nose tip in the face image as the first center coordinate of the target image.

[0037] An embodiment of the present invention relates to a terminal device, including a memory, a processor and a camera device, wherein the memory is used to store a computer program, the processor runs the computer program so that the terminal device can execute the intelligent following method involved in the embodiment of the present invention, and the camera device is used to obtain a target image.

[0038] An embodiment of the present invention relates to a readable storage medium storing a computer program. When the computer program is run on a processor, the intelligent following method according to the embodiment of the present invention is executed.

[0039] The present invention obtains the first center coordinate of the target image in the current frame; calculates the distance between the center of the target image and the center of the display screen according to the first center coordinate and the second center coordinate of the display screen; when the distance is greater than a preset distance threshold, moves the target image so that the center of the target image moves to the center of the display screen. The technical solution of the present invention moves the first center coordinate of the acquired target image to the second center coordinate of the display screen, so that the image acquired by the camera device is displayed in the central area of ​​the display screen, without using a hardware pan-tilt that requires a mechanical rotating device, thereby avoiding the vibration, noise and other effects caused by the rotation of the hardware pan-tilt. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solution of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope of protection of the present invention. In each of the drawings, similar components are numbered similarly.

[0041] Figure 1 A schematic diagram of a flow chart of an intelligent following method provided by an embodiment of the present invention is shown;

[0042] Figure 2 A schematic diagram of a following process of an intelligent following method provided by an embodiment of the present invention is shown;

[0043] Figure 3 A schematic diagram of a process flow of another intelligent following method provided by an embodiment of the present invention is shown;

[0044] Figure 4 A schematic diagram of a process flow of an intelligent following device provided by an embodiment of the present invention is shown.

[0045] Description of main component symbols:

[0046] 1-intelligent following device; 100-acquisition module; 200-calculation module; 300-movement module. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments.

[0048] The components of the embodiments of the present application generally described and shown in the drawings herein may be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0049] Hereinafter, the terms "including", "having" and their cognates, which may be used in various embodiments of the present application, are intended only to indicate specific features, numbers, steps, operations, elements, components, or combinations of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing items or adding the possibility of one or more features, numbers, steps, operations, elements, components, or combinations of the foregoing items.

[0050] Furthermore, the terms “first”, “second”, “third”, etc. are merely used for distinguishing descriptions and are not to be understood as indicating or implying relative importance.

[0051] Unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meanings as those generally understood by those skilled in the art to which the various embodiments of the present application belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meanings as the contextual meanings in the relevant technical field and will not be interpreted as having idealized meanings or overly formal meanings unless clearly defined in the various embodiments of the present application.

[0052] The applicable scenarios of the technical solution of the present application include scenarios where users record videos through terminal devices with cameras and / or scenarios where small animals record videos through terminal devices with cameras, helping users and / or small animals to achieve the image of users and / or small animals in the middle of the display screen when recording videos by themselves without any auxiliary mechanical equipment. Different from using a mechanical hardware gimbal to enable the terminal device to move with the movement of users and / or small animals, the technical solution of the present application mainly utilizes software capabilities and AI recognition technology to break through the wide-angle limitations of the camera, so as to achieve the maximum acquisition capability of the viewing angle when users and / or small animals move vertically or left and right in front of the screen, produce a smooth following effect, and keep users and / or small animals in the center of the screen, avoiding problems such as vibration, noise, and service life caused by the rotation of the hardware gimbal.

[0053] Example 1

[0054] In this embodiment, see Figure 1 , showing an intelligent following method comprising the following steps:

[0055] S100: Obtain the first center coordinates of the target image in the current frame.

[0056] The acquired image information can be displayed on the display screen through a camera device with a display screen. The acquired image information can be AI recognized to determine the target image in the image information. The target image can be an image of a face, a small animal, or other main objects. Further, the first center coordinate of the target image in the current frame is determined. The first center coordinate of the target image can be expressed in the form of pixels. For example, for an 8-megapixel camera device, there may be 3264 horizontal pixels and 2448 vertical pixels, that is, the total number of pixels is 3264*2448. The first center coordinate of the target image may be (1000, 1500).

[0057] S200: Calculating the distance between the center of the target image and the center of the display screen according to the first center coordinates and the second center coordinates of the display screen.

[0058] It should be understood that for a camera device with a display screen, the position of the camera device relative to the display screen is unchanged, and the second center coordinate of the display screen should coincide with the center coordinate of the field of view formed by the horizontal field of view angle and the vertical field of view angle of the camera device, such as Figure 2 As shown, the display screen and the field of view of the camera device are placed at the same coordinates. If the pixel of the camera device is 8 million pixels and the field of view of the camera device is 3264*2448 pixels, the second center coordinate of the display screen is (1632, 1224).

[0059] Furthermore, if the first center coordinate of the target image is (1000, 1500) and the second center coordinate of the display screen is (1632, 1224), the distance between them can be calculated based on the first center coordinate of the target image being (1000, 1500) and the second center coordinate of the display screen being (1632, 1224).

[0060] S300: When the distance is greater than a preset distance threshold, move the target image so that the center of the target image moves to the center of the display screen.

[0061] When the distance between the first center coordinate of the target image and the second center coordinate of the display screen is greater than a preset distance threshold, indicating that the target image deviates from the center of the display screen, the first center coordinate of the target image can be moved to the position of the second center coordinate of the display screen so that the target image is always located near the center of the display screen, thereby improving the user experience.

[0062] The setting range of the distance threshold can be 50 to 120 pixels.

[0063] This embodiment obtains the first center coordinate of the target image in the current frame; calculates the distance between the first center coordinate of the target image and the second center coordinate of the display screen; when the distance is greater than a preset distance threshold, moves the first center coordinate of the target image to the position where the second center coordinate of the display screen is located. The technical solution of this embodiment moves the first center coordinate of the acquired target image to the second center coordinate of the display screen, so that the image acquired by the camera device is displayed in the central area of ​​the display screen, without using a hardware pan-tilt that requires a mechanical rotating device, and avoids the effects of vibration, noise, etc. caused by the rotation of the hardware pan-tilt.

[0064] Example 2

[0065] Furthermore, when the target image is a face image, step S100: obtaining the first center coordinates of the target image in the current frame, includes: performing face recognition on the target image in the current frame; if the target image includes a face image, annotating the face image with a face frame, and taking the center position of the face frame as the first center coordinates.

[0066] It should be understood that through AI recognition, a face image can be recognized, and a face frame can be used to annotate the face image, and the center position of the face frame can be used as the first center coordinate. The center position coordinate of the face frame is determined, and the distance between the coordinate of the center position of the face frame and the second center coordinate of the display screen is calculated. When the distance between the coordinate of the center position of the face frame and the second center coordinate of the display screen is greater than a preset distance threshold, it indicates that the face image deviates from the center of the display screen, and the center coordinate of the face image can be moved to the position where the second center coordinate of the display screen is located. Optionally, the color of the face frame can be preset as a visible color, for example, red, yellow, blue, green and other visible colors are displayed on the display screen, and the color of the face frame can also be preset as an invisible color so that the face frame is not displayed on the display screen.

[0067] Example 3

[0068] In this embodiment, see Figure 3 , showing that step S300 in the smart following method includes the following steps:

[0069] S310: During the movement, real-time monitoring of the dynamic distance between the center of the target image and the center of the display screen in a preset moving direction.

[0070] The preset moving directions include: a first moving direction from the first center coordinates of the target image to the second center coordinates of the display screen, and / or a second moving direction from the first center coordinates of the target image to the horizontal coordinate of the second center coordinates of the display screen, and / or a third moving direction from the first center coordinates of the target image to the vertical coordinate of the second center coordinates of the display screen.

[0071] Exemplarily, the first moving direction is as follows Figure 2 The first moving direction is the direction indicated by the middle arrow; the second moving direction is the direction of horizontal movement from the coordinate (1000, 1500) to the coordinate (1632, 1500), and the second moving direction is the direction of vertical movement from the coordinate (1000, 1500) to the coordinate (1000, 1224).

[0072] It should be understood that when moving the first center coordinates of the target image to the second center coordinates of the display screen, it can move directly along the first moving direction; it can also first move horizontally along the second moving direction and then move vertically along the third moving direction; it can also first move vertically along the third moving direction and then move horizontally along the second moving direction.

[0073] Exemplarily, if the first center coordinate of the target image is moved to the second center coordinate of the display screen along the first moving direction, the target coordinate is always the second center coordinate of the display screen, and the dynamic distance is the distance between the changing first center coordinate of the target image and the second center coordinate of the display screen.

[0074] Exemplarily, if the target coordinates first move horizontally according to the second moving direction and then move vertically according to the third moving direction, the horizontal coordinate of the target coordinates of the horizontal movement is the horizontal coordinate of the center coordinates of the display screen, and the dynamic distance of the horizontal movement is the distance between the horizontal coordinates of the first center coordinates of the changing target image and the horizontal coordinates of the second center coordinates of the display screen; the vertical coordinate of the target coordinates of the vertical movement is the vertical coordinate of the center coordinates of the display screen, and the dynamic distance of the vertical movement is the distance between the vertical coordinates of the first center coordinates of the changing target image and the vertical coordinates of the second center coordinates of the display screen.

[0075] S320: Determine a moving speed according to the dynamic distance.

[0076] The moving speed of the target image is related to the dynamic distance. The longer the dynamic distance, the faster the moving speed.

[0077] Furthermore, the resolution of the camera device may be acquired; and the moving speed may be determined according to the resolution and the dynamic distance.

[0078] The moving speed of the target image is related to the resolution of the camera device, and the resolution of the camera device is related to the pixels. The larger the resolution, the larger the pixels. The larger the resolution of the camera device, the faster the moving speed of the target image can ensure that the target image moves quickly to the center of the display screen.

[0079] Furthermore, the field of view angle of the camera device may be acquired; and the moving speed may be determined according to the field of view angle and the dynamic distance.

[0080] The larger the field of view of the camera device, the larger the range of the captured image. Therefore, the larger the field of view of the camera device, the faster the moving speed of the target image can ensure that the target image moves quickly to the center of the display screen.

[0081] It should be understood that the moving speed of the target image can be determined based on at least one of the resolution of the camera device and the field of view of the camera device and the dynamic distance between the center coordinates of the target image and the target coordinates in the moving direction. The moving speed of the target image can also be determined based on the dynamic distance between the center coordinates of the target image and the target coordinates in the moving direction.

[0082] S330: Moving the target image according to the preset moving direction and the moving speed so that the center of the target image moves to the center of the display screen.

[0083] The first center coordinates of the target image can be directly moved along the first moving direction at the moving speed to the second center coordinates of the display screen; it can also be first moved horizontally along the second moving direction at the moving speed, and then moved vertically along the third moving direction at the moving speed; it can also be first moved vertically along the third moving direction at the moving speed, and then moved horizontally along the second moving direction at the moving speed.

[0084] It should be understood that the target image is within the field of view of the camera device. The higher the pixel of the camera device, the larger the field of view, and the larger the range of movement of the target image. When the target image is within the field of view of the camera device, the target image can be effectively moved to the center area of ​​the display screen without rotating the camera device.

[0085] Furthermore, if the target image is within the field of view of the camera device, but is located at the edge of the field of view, in order to avoid moving the first center coordinate of the target image to the second center coordinate of the display screen, resulting in a partial blank of the display screen, that is, the acquired image cannot completely fill the display screen, the acquired image can be appropriately enlarged to avoid the acquired image cannot completely fill the display screen. The enlargement threshold can be set according to the resolution of the display screen to ensure that the enlarged image does not affect the user experience.

[0086] Furthermore, if the acquired image still cannot completely fill the display screen after the magnification process, a reminder message may be sent through the display screen or through a voice device to remind the user that the image has deviated from the field of view.

[0087] Example 4

[0088] In this embodiment, see Figure 4 , showing that a smart following device 1 includes: an acquisition module 100, a calculation module 200 and a movement module 300.

[0089] An acquisition module 100 is used to acquire the first center coordinates of a target image in a current frame; a calculation module 200 is used to calculate the distance between the center of the target image and the center of the display screen based on the first center coordinates and the second center coordinates of the display screen; a moving module 300 is used to move the target image so that the center of the target image moves to the center of the display screen when the distance is greater than a preset distance threshold.

[0090] The acquisition module 100 includes: a recognition unit, which is used to perform face recognition on the target image in the current frame; and a determination unit, which is used to use the nose tip position in the face image as the first center coordinate of the target image if the target image includes a face image.

[0091] The smart following device 1 disclosed in this embodiment is used to execute the smart following method described in the above embodiment through the coordinated use of the acquisition module 100, the calculation module 200 and the movement module 300. The implementation scheme and beneficial effects involved in the above embodiment are also applicable to this embodiment and will not be repeated here.

[0092] An embodiment of the present application relates to a terminal device, including a memory, a processor and a camera device, wherein the memory is used to store a computer program, the processor runs the computer program so that the terminal device can execute the intelligent following method involved in the embodiment of the present application, and the camera device is used to obtain a target image.

[0093] Terminal devices include smart TVs, smart phones, computers and other video devices with video cameras and display screens.

[0094] An embodiment of the present application relates to a readable storage medium storing a computer program. When the computer program is run on a processor, the intelligent following method described in the embodiment of the present application is executed.

[0095] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or the flow diagram, and the combination of boxes in the structure diagram and / or the flow diagram, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0096] In addition, the functional modules or units in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0097] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

[0098] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. An intelligent following method, It is characterized in that The method includes: Get the first center coordinates of the target image in the current frame; Calculate the distance between the center of the target image and the center of the display screen according to the first center coordinates and the second center coordinates of the display screen; When the distance is greater than a preset distance threshold, moving the target image so that the center of the target image moves to the center of the display screen; The moving the target image so that the center of the target image moves to the center of the display screen includes: During the movement, real-time monitoring of the dynamic distance between the center of the target image and the center of the display screen in a preset moving direction; determining a moving speed according to the dynamic distance; Moving the target image according to the preset moving direction and the moving speed so that the center of the target image moves to the center of the display screen; The determining the moving speed according to the dynamic distance comprises: Obtaining the field of view angle of the camera device; The moving speed is determined according to the field of view angle and the dynamic distance, wherein the larger the field of view angle is, the larger the moving speed is.

2. The intelligent following method according to claim 1, It is characterized in that The obtaining of the first center coordinates of the target image in the current frame includes: Performing face recognition on the target image in the current frame; If the target image includes a face image, the face image is annotated using a face frame, and the center position of the face frame is used as the first center coordinate.

3. The intelligent following method according to claim 1, It is characterized in that The preset movement direction includes lateral movement and / or vertical movement. When the preset moving direction is lateral movement: During the movement, real-time monitoring of the horizontal dynamic distance between the central horizontal coordinate of the target image and the central horizontal coordinate of the display screen; determining a lateral movement speed according to the lateral dynamic distance; Moving the target image according to the lateral moving speed so that the center of the target image moves laterally to a position corresponding to the central horizontal coordinate of the display screen; When the preset moving direction is vertical movement: During the movement, real-time monitoring of the vertical dynamic distance between the central vertical coordinate of the target image and the central vertical coordinate of the display screen; determining a vertical moving speed according to the vertical dynamic distance; The target image is moved according to the vertical moving speed so that the center of the target image moves vertically to a position corresponding to the central longitudinal coordinate of the display screen.

4. An intelligent following device, It is characterized in that The device includes: An acquisition module, used for acquiring the first center coordinates of the target image in the current frame; a calculation module, configured to calculate the distance between the center of the target image and the center of the display screen according to the first center coordinates and the second center coordinates of the display screen; A moving module, configured to move the target image so that the center of the target image moves to the center of the display screen when the distance is greater than a preset distance threshold; The moving module is also used to monitor in real time during the movement process the dynamic distance between the center of the target image and the center of the display screen in a preset moving direction; determine the moving speed according to the dynamic distance; move the target image according to the preset moving direction and the moving speed so that the center of the target image moves to the center of the display screen; obtain the field of view angle of the camera device; determine the moving speed according to the field of view angle and the dynamic distance, wherein the larger the field of view angle, the greater the moving speed.

5. The smart following device according to claim 4, It is characterized in that The acquisition module comprises: A recognition unit, configured to perform face recognition on the target image in the current frame; The determination unit is configured to use, if the target image includes a face image, the position of the nose tip in the face image as the first center coordinate of the target image.

6. A terminal device, It is characterized in that The invention comprises a memory, a processor and a camera device, wherein the memory is used to store a computer program, and when the computer program is run on the processor, the intelligent following method according to any one of claims 1 to 3 is executed, and the camera device is used to acquire a target image.

7. A readable storage medium, It is characterized in that The device stores a computer program, which executes the intelligent following method according to any one of claims 1 to 3 when running on a processor.

Images