Tracking control method and device, computer equipment and storage medium

By detecting close-up areas and safe distances, the accurate switching and automatic recovery functions of intelligent target tracking technology are achieved, which solves the problems of inconvenient tracking and switching and insufficient automatic recovery capabilities in the existing technology, and is suitable for a variety of application scenarios.

CN119071637APending Publication Date: 2024-12-03REMO TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411123232.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-15
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

Existing intelligent target tracking technology is difficult to achieve accurate tracking and switching in multiple application scenarios, especially the function of automatically restoring tracking when the target moves to other areas.

Method used

By detecting whether the predetermined close-up area is in the current panoramic image and determining the safe distance of the tracking target in the panoramic image and the preset close-up panoramic image, the close-up tracking mode switching of the predetermined close-up area is realized to ensure that the target is in the lens and is not easy to escape.

Benefits of technology

It realizes accurate switching between panoramic tracking and close-up tracking, which is suitable for a variety of application scenarios, ensuring that the target is zoomed in when the close-up area enters the lens, and returning to normal tracking mode when the target leaves.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119071637A_ABST
    Figure CN119071637A_ABST
Patent Text Reader

Abstract

The invention discloses a tracking control method and device, computer equipment and a storage medium. The tracking control method comprises the following steps: detecting that a predetermined close-up area is located in a current panoramic image; determining that the tracking target is in a safe distance in the current panoramic image and the tracking target is in a safe distance in a preset close-up panoramic image; and entering a close-up tracking mode to carry out close-up display on the predetermined close-up area. Compared with the prior art, when the target is tracked, the target can enter the close-up tracking mode when the preset close-up area enters the lens and the target is in the safe distance in the current panoramic image and the preset close-up panoramic image of the preset close-up area and is not easy to escape, and the preset close-up area is amplified and displayed; precise switching between panoramic tracking and close-up tracking is achieved, and the method is suitable for more application scenes.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and more specifically to a tracking control method, device, computer equipment and storage medium. Background Art

[0002] With the rapid development of the Internet and the rapid growth of remote video services, intelligent target tracking solutions have been widely used in education, entertainment, live broadcast and other fields to track target persons, such as tracking teachers in the classroom, tracking performers on the stage, or tracking anchors in the live broadcast room.

[0003] Currently, when tracking and shooting a target, the target is usually placed in the center of the picture for continuous tracking. However, in real application scenarios, it is often necessary to stop tracking at certain positions and focus on a certain area, and automatically resume tracking when the target moves to other areas. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a tracking control method, device, computer equipment and storage medium to achieve tracking switching and adapt to various application scenarios.

[0005] In a first aspect, an embodiment of the present invention provides a tracking control method, which includes: detecting that a predetermined close-up area is located within a current panoramic image; determining that a tracking target is at a safe distance within the current panoramic image and that the tracking target is at a safe distance within a preset close-up panoramic picture; entering a close-up tracking mode to display a close-up of the predetermined close-up area.

[0006] In a second aspect, an embodiment of the present invention further provides a tracking control device, which includes a unit for executing the above method.

[0007] In a third aspect, an embodiment of the present invention further provides a computer device, which includes a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above method when executing the computer program.

[0008] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a computer program, wherein the computer program includes program instructions, and the program instructions can implement the above method when executed by a processor.

[0009] Compared with the prior art, the present invention can enter the close-up tracking mode to display a close-up of the predetermined close-up area after detecting that the predetermined close-up area is located in the current panoramic image and determining that the tracking target is at a safe distance in the current panoramic image and that the tracking target is at a safe distance in the preset close-up panoramic screen. It can be seen that when the present invention tracks the target, when the predetermined close-up area enters the lens and the target is at a safe distance in both the current panoramic image and the preset close-up panoramic screen of the predetermined close-up area and is not easy to escape, the close-up tracking mode can be entered to enlarge and display the predetermined close-up area, thereby realizing precise switching between panoramic tracking and close-up tracking, and being suitable for more application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 A schematic diagram of an application scenario of the tracking control method provided by an embodiment of the present invention;

[0011] Figure 2 A schematic flow chart of a tracking control method provided in the first embodiment of the present invention;

[0012] Figure 3 A schematic diagram of a sub-flow chart of a tracking control method provided in the first embodiment of the present invention;

[0013] Figure 4 A schematic diagram of another sub-flow chart of the tracking control method provided by the first embodiment of the present invention;

[0014] Figure 5 A schematic diagram of the position relationship of a target in a panoramic image when the tracking control method provided by the first embodiment of the present invention is applied;

[0015] Figure 6 A schematic diagram of a close-up picture output when the tracking control method provided by the first embodiment of the present invention is applied;

[0016] Figure 7 A schematic diagram of another sub-flow chart of the tracking control method provided by the first embodiment of the present invention;

[0017] Figure 8 A schematic flow chart of a tracking control method provided in a second embodiment of the present invention;

[0018] Fig. 9 A schematic block diagram of a tracking control device provided in a first embodiment of the present invention;

[0019] Fig.10 A schematic block diagram of a tracking control device provided for a second embodiment of the present invention; and

[0020] Fig.11 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0022] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0023] It should also be understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0024] See also Figure 1 and Figure 2 , Figure 1 A schematic diagram of an application scenario of the tracking control method provided by an embodiment of the present invention, Figure 2 A schematic flow chart of a tracking control method provided in the first embodiment of the present invention. The tracking control method is applied in a server 10, wherein the server 10 interacts with a client 20 for data exchange, and the server 10 processes image data to control the camera to switch between a normal tracking mode and a close-up tracking mode according to the camera's posture information and the position of a predetermined close-up area in a current panoramic image and a preset close-up panoramic image, and sends the panorama image captured by tracking and the close-up image output after final cropping to the client 20 for display.

[0025] It should be noted that Figure 1 Only one client 20 is illustrated in the figure. In actual operation, the server 10 can exchange data with multiple clients 20.

[0026] Figure 2 1 is a flow chart of a tracking control method provided by the first embodiment of the present invention. As shown in the figure, the method includes the following steps S110-130.

[0027] S110: Detect that a predetermined close-up area is located within the current panoramic image.

[0028] In the present invention, while tracking the target, it is detected whether any predetermined close-up area is located in the current panoramic image. Specifically, this step may include steps S111-112 (such as Figure 3 shown):

[0029] S111 . Obtain preset close-up ROI coordinates of any predetermined close-up area mapped to the current panoramic image according to the camera posture.

[0030] In the present invention, the close-up area is calibrated by the posture position coordinates of the camera, and the predetermined close-up area is mapped into the current panoramic image, thereby realizing the determination of the camera working mode switching.

[0031] Specifically, in some embodiments, Figure 4 As shown, the step S111 may specifically include steps S1111-S1113.

[0032] S1111. Obtain the posture position coordinates and the corresponding field of view angle of the target tracked by the camera in the current panoramic image.

[0033] In this step, the camera may be a gimbal camera. During tracking shooting, the gimbal in the gimbal camera drives the camera to rotate. In this embodiment, the posture position coordinates of the camera are defined as p={α, β, r}, where α is the horizontal rotation angle of the camera, β is the vertical rotation angle of the camera, and r is the zoom ratio of the camera.

[0034] like Figure 5 As shown, the current panoramic image coordinates are set to {0,0,1,1}, and the current position coordinates of the target in the current panoramic image are set to t = {x min ,y min ,x max ,y max}, understandably, {x min ,y min} and {x max ,y max} represents the coordinates of the upper left corner and lower right corner of the tracking target detection box t, and the four coordinate values ​​range from 0 to 1.

[0035] S1112. Obtain preset position center coordinates of the center of the predetermined close-up area in the current panoramic image according to the acquired posture position coordinates, the corresponding field of view angle, and the posture position coordinates of the predetermined close-up area.

[0036] In this embodiment, the area that needs a close-up is calibrated by the camera's posture position coordinates, and all the areas that need a close-up constitute a predetermined close-up area set P = {p i}, i = 1, 2, ...; where p i is the position coordinate of the camera in the i-th predetermined close-up area.

[0037] In this step, each predetermined close-up area is mapped to the current panoramic image to obtain the corresponding preset position center coordinates. Specifically, according to the posture position coordinates p currently obtained by the camera, c ={αc ,β c ,r c}、Corresponding field of view {FOV x ,FOV y} and the posture position coordinates p of the predetermined close-up area i ={α i ,β i ,r i}, using the formula Calculate and obtain the preset position center coordinates {x c ,y c}; where x o and o is the center coordinate of the current panoramic image. Since the coordinates of the current panoramic image are {0,0,1,1}, in this embodiment, x o and o Both are 0.5.

[0038] S1113 . Obtain preset close-up ROI coordinates of each corresponding preset close-up area according to the current panoramic image coordinates and the acquired preset position center coordinates.

[0039] In this step, the ROI coordinates of each predetermined close-up area after being mapped to the current panoramic image are obtained. Specifically, according to the current panoramic image coordinates and the obtained preset position center coordinates {x c ,y c}, using the formula Calculate and obtain the preset close-up ROI coordinates ROI(i) of each corresponding predetermined close-up area; where r i is the camera zoom ratio corresponding to the predetermined close-up area. In this embodiment, x o and o is 0.5. It can be understood that ROI(i) is centered on the center of the close-up area and the zoom factor is r i The coordinates at time (such as Figure 6 ), which is the coordinates of the close-up image finally output after digital cropping.

[0040] In the present invention, the positive and negative signs in the calculation formula of the above ROI(i) match the positive and negative directions of the pan-tilt rotation direction and the mirror / flip status of the camera image, that is, the positive and negative signs are determined based on whether the horizontal and vertical directions of the camera image are consistent with the direction of pan-tilt rotation. Because the positive and negative directions of the pan-tilt rotation direction are always constant, but the direction of the image screen will be reversed as the user sets the mirror / flip, etc., it is necessary to adjust the positive and negative signs in the coordinates of the above two technical formulas in combination with the mirror / flip status of the camera screen. For example, when the horizontal and vertical directions of the camera screen are consistent with the direction of pan-tilt rotation, When the horizontal and vertical directions of the camera image are inconsistent with the direction of the gimbal rotation,

[0041] S112. Obtain the proportion of each predetermined close-up area in the current panoramic image according to the preset close-up ROI coordinates of the predetermined close-up area and the current panoramic image coordinates, and detect whether the proportion exceeds a fifth preset threshold. If the proportion exceeds the fifth preset threshold, the predetermined close-up area is located in the current panoramic image.

[0042] In this step of the present embodiment, the preset close-up ROI coordinates ROI(i) of each predetermined close-up area are traversed, and the ratio thereof to the current panoramic image is detected. If the ratio exceeds the fifth preset threshold, it indicates that the predetermined close-up area is located in the current panoramic image. Preferably, the ratio of the predetermined close-up area i in the current panoramic image can be obtained according to the ratio of the preset close-up ROI coordinates in the current panoramic image coordinates {0,0,1,1}. In the present embodiment, the fifth preset threshold can be [0.2,0.5]. Understandably, the ratio can be calculated by boundary distance calculation, covariance calculation, or the overlap ratio of the predetermined close-up area and the current panoramic image.

[0043] In some other embodiments, this step can also determine whether the predetermined close-up area is located in the current panoramic image by detecting the distance between the center of the predetermined close-up area and the center of the current panoramic image. Specifically, it can be: calculating the distance between the center of the preset close-up ROI coordinates of each predetermined close-up area and the center of the current panoramic image, and detecting whether the distance is lower than a sixth preset threshold. If so, the predetermined close-up area is located in the current panoramic image.

[0044] S120: Determine whether the tracking target is at a safe distance in the current panoramic image and whether the tracking target is at a safe distance in a preset close-up panoramic picture.

[0045] In this step, when the predetermined close-up area is located in the current panoramic image, it is detected whether the current conditions for entering the close-up tracking mode are met, so that the predetermined close-up area is enlarged and displayed when the conditions are met, that is, when any predetermined close-up area appears in the current panoramic image, it is detected whether the tracking target is at a safe distance in the current panoramic image and the preset close-up panoramic picture, and when both are at a safe distance, the predetermined close-up area is enlarged and displayed.

[0046] In some embodiments, Figure 7 As shown, this step may specifically include the following steps S121-S122:

[0047] S121 . Obtain preset panoramic ROI coordinates of any predetermined close-up area mapped to the current panoramic image according to the camera posture.

[0048] The method of obtaining the preset panoramic ROI coordinates of the predetermined close-up area mapped to the current panoramic image according to the camera posture can obtain the corresponding preset panoramic ROI coordinates of each predetermined close-up area according to the preset position center coordinates of the center of the predetermined close-up area in the current panoramic image and the current panoramic image coordinates obtained in step S1112. Specifically, according to the current panoramic image coordinates and the obtained preset position center coordinates {x c ,y c}, using the formula Calculate and obtain the preset panoramic ROI coordinates ROIF(i) of each corresponding predetermined close-up area; in this embodiment, x o and o is 0.5. Understandably, the preset panoramic ROI coordinate ROIF(i) is the panoramic screen ROI coordinate when the center point of the close-up area is the center of the screen and the zoom ratio is 1x. ROI(i) is the zoom ratio of ROIF(i). i The coordinates of the magnification.

[0049] The positive and negative signs in the calculation formula of ROIF(i) also match the positive and negative directions of the pan / tilt rotation and the mirror / flip states of the camera image. It can be understood that the preset panoramic ROI coordinates ROIF(i) can be ROIF(i)={x c -x o ,y c -y o ,x c +x o ,y c +y o}, or ROIF(i)={x c +x o ,y c +y o ,x c -x o ,y c -y o}.

[0050] S122: Detect whether the tracking target is at a safe distance within the current panoramic image and a preset close-up panoramic picture formed by the preset panoramic ROI coordinates.

[0051] In this embodiment, the step may be: respectively detecting whether the boundary distance between the tracking target and the current panoramic image and the boundary distance between the tracking target and the preset close-up panoramic screen exceed the first preset threshold and the second preset threshold. If the boundary distance between the tracking target and the current panoramic image exceeds the first preset threshold, and the boundary distance between the tracking target and the preset close-up panoramic screen exceeds the second preset threshold, the tracking target is at a safe distance in the current panoramic image and the preset close-up panoramic screen, and the camera may be controlled to exit the normal tracking mode and switch to the close-up tracking mode. If not, it means that the tracking target is about to disappear from the current panoramic image and / or the preset close-up panoramic screen, and the camera maintains the normal tracking mode and does not enter the close-up tracking mode. That is, when entering the regional close-up, it is ensured that the current target is visible in the current panoramic image and when it moves to the preset close-up panoramic screen, so as to avoid the target disappearing from the camera field of view after the camera rotates to the predetermined close-up area, and the tracking switch can be controlled more accurately.

[0052] Specifically, the detection of whether the boundary distance between the tracking target and the preset close-up panoramic picture exceeds the second preset threshold specifically includes: calculating the distance (which may also be the Euclidean distance, etc.) between the boundary (left, right, upper and / or lower boundary) of the tracking target detection frame t and the corresponding boundary in the preset close-up panoramic picture, taking the minimum value therebetween, and detecting whether the calculated minimum distance exceeds the second preset threshold; or, calculating the boundary distance between the center of the tracking target detection frame t and each boundary in the preset close-up panoramic picture, taking the minimum value therebetween, and detecting whether the calculated minimum distance exceeds the second preset threshold. In the present invention, the second preset threshold may be [-0.25, +0.25]; preferably, in this embodiment, the second preset threshold may be 0.1, and in some other embodiments, it may be increased or decreased according to actual needs.

[0053] It is understandable that the implementation method of detecting whether the boundary distance between the tracking target and the current panoramic image exceeds the first preset threshold can be similar to the implementation method of detecting whether the boundary distance between the tracking target and the preset close-up panoramic image exceeds the second preset threshold. The distance between the boundaries can be calculated to determine whether it is at a safe distance, which will not be described in detail here. In the present invention, the value of the first preset threshold can be the same as or different from the second preset threshold.

[0054] It can be seen from the above that if the current tracking target is about to escape within the current panoramic image or the preset close-up panoramic image, the tracking control method of the present invention does not switch to the close-up tracking mode, and still adopts the normal tracking mode. When the distance between the tracking target and the left and right boundaries and / or the upper and lower boundaries of the current panoramic image and the preset close-up panoramic image is at a safe distance, that is, exceeds the first preset threshold and the second preset threshold respectively, the camera is controlled to switch and enter the close-up tracking mode.

[0055] Understandably, in some other embodiments, the detection of whether the tracking target is at a safe distance in the current panoramic image and the preset close-up panoramic screen formed by the preset panoramic ROI coordinates can also be: respectively calculating the Mahalanobis distance / Euclidean distance between the center of the tracking target and the center of the current panoramic image and the preset close-up panoramic screen, if the calculated Mahalanobis distance / Euclidean distance between the tracking target and the center of the current panoramic image is lower than the third preset threshold, and the Mahalanobis distance / Euclidean distance between the tracking target and the center of the preset close-up panoramic screen is lower than the fourth preset threshold, then the tracking target is at a safe distance in the current panoramic image and the preset close-up panoramic screen, and the camera is controlled to exit the normal tracking mode and switch to the close-up tracking mode. That is, in other embodiments, the distance measurement parameters such as the Mahalanobis distance or Euclidean distance between the center of the tracking target detection frame and the center of the current panoramic image and the preset close-up panoramic screen can also be used as the judgment standard, and when the distance measurement parameter between the current panoramic image and the preset close-up panoramic screen is lower than the third preset threshold and the distance measurement parameter between the preset close-up panoramic screen is lower than the fourth preset threshold, it is determined that the condition for entering the close-up tracking mode is triggered, and the predetermined close-up area is enlarged and displayed, wherein the third preset threshold and the fourth preset threshold can be set according to actual conditions, for example, they can also be values ​​such as 0.1.

[0056] S130 , entering a close-up tracking mode to display a close-up of the predetermined close-up area.

[0057] In this step, entering the close-up tracking mode can specifically include: controlling the rotation of the camera's gimbal according to the horizontal rotation angle and the vertical rotation angle in the posture position coordinates of the predetermined close-up area, and controlling the camera's focal length according to the zoom ratio in the posture position coordinates of the predetermined close-up area, so as to instantaneously locate the predetermined close-up area, focus the camera on the predetermined close-up area, and perform real-time close-up tracking shooting.

[0058] In summary, the tracking control method of the present invention can realize the switching control between the normal tracking mode and the close-up tracking mode using a gimbal camera according to the posture position coordinates of the predetermined close-up area, the posture position coordinates of the target tracked by the camera in the current panoramic image, the corresponding field of view angle and the position of the target in the current panoramic image and the preset close-up panoramic picture, etc. When switching to the close-up tracking mode, the camera is driven to rotate by controlling the rotation of the gimbal to perform close-up tracking, and at the same time, the zoom of the camera focal length is controlled to zoom in on the target, so that the locked target can be photographed in close-up in real time and accurately, which is more in line with the actual needs of users in tracking and shooting scenarios.

[0059] Figure 8 FIG. 1 is a flow chart of a tracking control method provided by the second embodiment of the present invention. Figure 8 As shown, the tracking control method of this embodiment adds steps S140-S150 on the basis of the above-mentioned first embodiment.

[0060] S140, detecting whether the tracking target is still at a safe distance within the current panoramic image and the preset close-up panoramic image, if so, maintaining the current close-up tracking mode; if not, exiting the close-up tracking mode and resuming the normal tracking mode.

[0061] Preferably, in this embodiment, when the camera works in close-up tracking mode, although the normal tracking mode control is turned off, the real-time ROI coordinates of the tracking target are still calculated and updated. Specifically, based on the initial ROI coordinates of the tracking target in the current panoramic image, the center of the close-up area for close-up tracking, and the camera zoom factor during close-up tracking, the real-time ROI coordinates of the tracking target in the panoramic image are calculated and updated and stored in the database.

[0062] It can be understood that when the camera works in the close-up tracking mode, the method of step S122 can be used to detect whether the tracking target is still at a safe distance in the current panoramic image and the preset close-up panoramic picture. For example, it is respectively detected whether the distance between the boundary of the detection frame formed by the real-time ROI coordinates of the tracking target and the boundary of the rectangular frame formed by the current panoramic image and the preset panoramic ROI coordinates ROIF(i) exceeds the first preset threshold and the second preset threshold. If both exceed, the close-up tracking mode is maintained; and if only one does not exceed, it means that the target is about to disappear from the camera's field of view after the camera rotates to the predetermined close-up area (after entering the close-up display). At this time, the camera is controlled to exit the close-up tracking mode and return to the normal tracking mode, that is, if the distance between the boundary of the detection frame of the current tracking target and the boundary of the current panoramic image and the preset close-up panoramic picture is all at a safe distance, the current close-up tracking mode is maintained; if it is not at a safe distance from the current panoramic image or the preset close-up panoramic picture, it means that the target currently displayed in close-up is about to disappear from the camera's field of view, then the close-up tracking mode is exited and the normal tracking mode is restored.

[0063] S150, detecting whether the horizontal rotation angle and the vertical rotation angle of the camera are respectively within their corresponding preset ranges according to the camera posture; if not, prohibiting tracking.

[0064] In this step, the horizontal rotation angle and the vertical rotation angle of the camera are detected based on the acquired camera posture position coordinates to see whether they are respectively within their corresponding preset ranges, so as to perform tracking control, so as to limit the tracking range by limiting the horizontal rotation angle and the vertical rotation angle of the camera. If they are within the preset range, the tracking control is performed normally, and if they are not within the preset range, the tracking control is prohibited to prevent the lens from tracking some areas that the audience is not expected to see.

[0065] In specific implementation, a set of preset parameters can be defined: x 0 ,x 1 ,y 0,y 1 To represent the left and right extreme positions in the horizontal direction, and the upper and lower extreme positions in the vertical direction; according to the formula Get the control gain g of the camera in the horizontal and vertical directions x and g y , where f function represents the gain mapping function, x and y represent the current horizontal rotation angle and vertical rotation angle of the camera respectively, [x 0 ,x 1 ] and [y 0 ,y 1 ] respectively represent the effective preset range of the horizontal rotation angle and the vertical rotation angle. The preset range can be set according to the actual application requirements. When x / y is within the effective preset range, gx / gy outputs 1 (normal tracking control), otherwise it outputs 0 (indicating that tracking control is prohibited).

[0066] As can be seen from the above, when the tracking control method of this embodiment is applied, the camera tracks and shoots the tracking target, and can stop the normal tracking mode and switch to the close-up tracking mode at the specified position, and focus on and magnify the predetermined close-up area. When the target leaves or is about to leave the camera's field of view, the normal tracking mode is restored, that is, when the target leaves the specified position, it can be restored to normal tracking. At the same time, during the entire tracking process, it can be ensured that the tracking will not exceed the preset range, meeting the actual needs of users in tracking and shooting scenarios.

[0067] Fig. 9 FIG. 3 is a schematic block diagram of a tracking control device 300 provided in the first embodiment of the present invention. Fig. 9 As shown, corresponding to the tracking control method of the first embodiment above, the present invention further provides a tracking control device 300. The tracking control device 300 includes a unit for executing the tracking control method of the first embodiment above, and the device can be configured in a server. Fig. 9 The tracking control device 300 includes a detection and determination unit 301 and a control unit 302.

[0068] The detection and determination unit 301 is used to detect whether the predetermined close-up area is located in the current panoramic image, and when the predetermined close-up area is located in the current panoramic image, determine whether the tracking target is at a safe distance in the current panoramic image and the preset close-up panoramic screen. In this embodiment, the detection and determination unit 301 is specifically used to obtain the preset close-up ROI coordinates and the preset panoramic ROI coordinates mapped to the current panoramic image by any predetermined close-up area according to the camera posture; obtain the proportion of the predetermined close-up area in the current panoramic image according to the preset close-up ROI coordinates of each predetermined close-up area and the current panoramic image coordinates, and detect whether the proportion exceeds the fifth preset threshold. If the proportion exceeds the fifth preset threshold, the predetermined close-up area is located in the current panoramic image; and detect whether the tracking target is at a safe distance in the current panoramic image and the preset close-up panoramic screen formed by the preset panoramic ROI coordinates. Specifically, the detection and determination unit 301 includes an acquisition unit 3011, a first calculation unit 3012, a second calculation unit 3013, and a detection and determination unit 3014.

[0069] The acquisition unit 3011 is used to acquire the posture position coordinates of the target being tracked by the camera in the current panoramic image and the corresponding field of view angle; preferably, the current panoramic image being captured is acquired, the ROI coordinates are set to {0,0,1,1}, and the current position coordinates of the target being tracked in the current panoramic image are obtained {x min ,y min ,x max ,y max}, and at the same time obtain the current camera posture information, the posture information includes the horizontal rotation angle α, the vertical rotation angle β, the zoom ratio r and the field of view {FOV x ,FOV y}, etc., the horizontal rotation angle α, the vertical rotation angle β and the zoom factor r in the posture information constitute the posture position coordinates p = {α, β, r}. The first calculation unit 3012 is used to obtain the preset position center coordinates of the center of the predetermined close-up area in the current panoramic image according to the acquired posture position coordinates, the corresponding field of view angle and the posture position coordinates of the predetermined close-up area; that is, each predetermined close-up area is mapped to the current panoramic image, and the preset position center coordinates corresponding to the center of the predetermined close-up area are obtained according to the posture information of the camera during tracking shooting of the predetermined close-up area and the posture information of the camera during tracking shooting of the current panoramic image. Specifically, according to the posture position coordinates p currently acquired by the camera c ={α c ,β c ,r c}、Corresponding field of view {FOV x ,FOV y} and the posture position coordinates p of the predetermined close-up area i ={αi ,β i ,r i}, using the formula Calculate and obtain the preset position center coordinates {x c ,y c}; where x o and o is the center coordinate of the current panoramic image. Since the coordinates of the current panoramic image are {0,0,1,1}, in this embodiment, x o and o The second calculation unit 3013 is used to obtain the preset close-up ROI coordinates and the preset panoramic ROI coordinates of each corresponding predetermined close-up area according to the current panoramic image coordinates and the obtained preset position center coordinates; specifically, according to the current panoramic image coordinates and the obtained preset position center coordinates {x c ,y c}, using the formula and formula Calculate and obtain the preset close-up ROI coordinates ROI(i) and the preset panoramic ROI coordinates ROIF(i) of each corresponding predetermined close-up area; wherein, r i is the camera zoom ratio corresponding to the predetermined close-up area. In this embodiment, x o and o The detection and determination unit 3014 is used to obtain the proportion of the predetermined close-up area in the current panoramic image according to the preset close-up ROI coordinates of each predetermined close-up area and the current panoramic image coordinates, and detect whether the proportion exceeds the fifth preset threshold. If the proportion exceeds the fifth preset threshold, the predetermined close-up area is located in the current panoramic image; that is, traverse the preset close-up ROI coordinates ROI(i) of each predetermined close-up area, detect its proportion with the current panoramic image, and if the proportion exceeds the fifth preset threshold, it means that the predetermined close-up area is located in the current panoramic image.

[0070] The detection and determination unit 3014 is also used to detect whether the tracking target is at a safe distance in the current panoramic image and the preset close-up panoramic picture formed by the preset panoramic ROI coordinates; the detection and determination unit 3014 detects whether the current condition for entering the close-up tracking mode is met when the predetermined close-up area is located in the current panoramic image, so as to enlarge and display the predetermined close-up area when the condition is met. In this embodiment, the condition for entering the close-up tracking mode is: when the predetermined close-up area is located in the current panoramic image, whether the boundary distance of the tracking target in the current panoramic image exceeds a first preset threshold, and whether the boundary distance of the tracking target in the preset close-up panoramic picture exceeds a second preset threshold, in the detection and determination unit When 3014 detects that the boundary distance of the tracking target in the current panoramic image exceeds a first preset threshold and the boundary distance in the preset close-up panoramic picture exceeds a second preset threshold, the control unit 302 is used to control the camera to enter the close-up tracking mode, that is, the control unit 302 is used to control the rotation of the camera's gimbal according to the horizontal rotation angle and the vertical rotation angle in the posture position coordinates of the predetermined close-up area when the tracking target is at a safe distance in both the current panoramic image and the preset close-up panoramic picture, and to control the focal length of the camera according to the zoom ratio in the posture position coordinates of the predetermined close-up area, so as to instantaneously locate the predetermined close-up area, so that the camera focuses on the predetermined close-up area, and performs real-time close-up tracking shooting.

[0071] Fig.10 FIG. 3 is a schematic block diagram of a tracking control device 300 provided in the second embodiment of the present invention. Fig.10As shown, the tracking control device 300 of this embodiment is different from the first device embodiment described above in that a tracking restriction unit 303 is added, and the specific functions of the detection and determination unit 3014 are different, and the functions of the other units are the same or similar. In this embodiment, the detection and determination unit 3014 is also used to detect whether the tracking target is still in a safe distance in the current panoramic image and the preset close-up panoramic screen in the close-up tracking mode. If so, the current close-up tracking mode is maintained; if not, the close-up tracking mode is exited and the normal tracking mode is restored. In the present invention, when the camera works in close-up tracking, although the normal tracking mode control is turned off, the real-time ROI coordinates of the tracking target are still calculated and updated. When it is detected that the detection frame boundary formed by the real-time ROI coordinates of the tracking target is not within the safe distance from the boundary of the current panoramic image and the preset close-up panoramic screen, that is, when the target currently displayed in close-up is about to disappear from the camera's field of view, the control unit 302 controls the camera to exit the close-up tracking mode and restore to the normal tracking mode. The tracking limiting unit 303 is used to detect whether the horizontal rotation angle and the vertical rotation angle of the camera are respectively within the corresponding preset ranges according to the acquired camera posture position coordinates. If not, tracking is prohibited. In this embodiment, the tracking range of the camera is limited, that is, the rotation range of the horizontal rotation angle and the vertical rotation angle of the camera is limited. If they are within the preset range, the tracking control is carried out normally, and if they are not within the preset range, the tracking control is prohibited to avoid the lens tracking to some areas that the audience is not willing to see.

[0072] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned tracking control device 300 and each unit can refer to the corresponding description in the aforementioned method embodiment, and for the convenience and brevity of description, it will not be repeated here.

[0073] The tracking control device 300 can be implemented in the form of a computer program. Fig.11 Runs on the computer device shown.

[0074] See also Fig.11 , Fig.11 1 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a server, which may be an independent server or a server cluster composed of multiple servers.

[0075] See also Fig.11 The computer device 500 includes a processor 502 , a memory and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0076] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. The computer program 5032 includes program instructions, and when the program instructions are executed, the processor 502 can execute a tracking control method.

[0077] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .

[0078] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a tracking control method.

[0079] The network interface 505 is used to communicate with other devices over the network. Fig.11 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0080] Among them, the processor 502 is used to run the computer program 5032 stored in the memory to implement the following steps: detecting that a predetermined close-up area is located within the current panoramic image; determining that the tracking target is at a safe distance within the current panoramic image and that the tracking target is at a safe distance within a preset close-up panoramic picture; entering a close-up tracking mode to display a close-up of the predetermined close-up area.

[0081] In one embodiment, determining that the tracking target is at a safe distance in the current panoramic image and that the tracking target is at a safe distance in a preset close-up panoramic picture specifically includes: obtaining preset panoramic ROI coordinates mapped to the current panoramic image from any predetermined close-up area according to the camera posture; and detecting whether the tracking target is at a safe distance in the current panoramic image and in a preset close-up panoramic picture formed by the preset panoramic ROI coordinates.

[0082] In one embodiment, the method of obtaining the preset panoramic ROI coordinates of any predetermined close-up area mapped to the current panoramic image according to the camera posture specifically includes: obtaining the posture position coordinates of the camera tracking the target in the current panoramic image and the corresponding field of view angle; obtaining the preset position center coordinates of the center of the predetermined close-up area in the current panoramic image according to the obtained posture position coordinates, the corresponding field of view angle and the posture position coordinates of the predetermined close-up area; and obtaining the corresponding preset panoramic ROI coordinates of each predetermined close-up area according to the current panoramic image coordinates and the obtained preset position center coordinates.

[0083] In one embodiment, the step of obtaining the preset position center coordinates of the center of the predetermined close-up area in the current panoramic image according to the acquired posture position coordinates, the corresponding field of view angle, and the posture position coordinates of the predetermined close-up area specifically includes: obtaining the preset position center coordinates of the center of the predetermined close-up area in the current panoramic image according to the posture position coordinates currently acquired by the camera c ={α c ,β c ,r c}、Corresponding field of view {FOV x ,FOV y} and the posture position coordinates p of the predetermined close-up area i ={α i ,β i ,r i}, using the formula Calculate and obtain the preset position center coordinates {x c ,y c}; where α is the horizontal rotation angle of the camera, β is the vertical rotation angle of the camera, r is the zoom ratio of the camera, and x o and o is the center coordinate of the current panoramic image.

[0084] In one embodiment, the step of obtaining the preset panoramic ROI coordinates of each corresponding predetermined close-up area according to the current panoramic image coordinates and the obtained preset position center coordinates specifically includes: obtaining the preset ROI coordinates of each corresponding predetermined close-up area according to the current panoramic image coordinates and the obtained preset position center coordinates {x c ,y c}, using the formula Calculate and obtain the preset panoramic ROI coordinates ROIF(i) of each corresponding predetermined close-up area; wherein the current panoramic image coordinates are set to {0,0,1,1}, x o and o is the center coordinate of the current panoramic image.

[0085] In one embodiment, the detecting whether the tracking target is at a safe distance within the current panoramic image and the preset close-up panoramic picture formed by the preset panoramic ROI coordinates specifically includes: respectively detecting whether the boundary distance between the tracking target and the current panoramic image and the boundary distance between the tracking target and the preset close-up panoramic picture exceed a first preset threshold and a second preset threshold; if the boundary distance between the tracking target and the current panoramic image exceeds the first preset threshold and the boundary distance between the tracking target and the preset close-up panoramic picture exceeds the second preset threshold, the tracking target is at a safe distance within the current panoramic image and the preset close-up panoramic picture.

[0086] In one embodiment, the detecting whether the tracking target is at a safe distance within the current panoramic image and the preset close-up panoramic picture formed by the preset panoramic ROI coordinates specifically includes: respectively calculating the Mahalanobis distance / Euclidean distance between the center of the tracking target and the center of the current panoramic image and the preset close-up panoramic picture; if the calculated Mahalanobis distance / Euclidean distance between the tracking target and the center of the current panoramic image is lower than a third preset threshold, and the Mahalanobis distance / Euclidean distance between the tracking target and the center of the preset close-up panoramic picture is lower than a fourth preset threshold, then the tracking target is at a safe distance within the current panoramic image and the preset close-up panoramic picture.

[0087] In one embodiment, after the processor 502 implements the entry into the close-up tracking mode to display a close-up of the predetermined close-up area, it also implements the following steps: detecting whether the tracking target is still at a safe distance within the current panoramic image and the preset close-up panoramic image, and if so, maintaining the current close-up tracking mode; if not, exiting the close-up tracking mode and resuming the normal tracking mode.

[0088] In one embodiment, the detecting that the predetermined close-up area is located in the current panoramic image specifically includes: obtaining preset close-up ROI coordinates of any predetermined close-up area mapped to the current panoramic image according to the camera posture; obtaining the proportion of the predetermined close-up area in the current panoramic image according to the preset close-up ROI coordinates of each predetermined close-up area and the current panoramic image coordinates, and detecting whether the proportion exceeds a fifth preset threshold. If the proportion exceeds the fifth preset threshold, the predetermined close-up area is located in the current panoramic image; or, calculating the distance between the center of the preset close-up ROI coordinates of each predetermined close-up area and the center of the current panoramic image, and detecting whether the distance is lower than a sixth preset threshold. If so, the predetermined close-up area is located in the current panoramic image.

[0089] In one embodiment, the processor 502 further implements the following steps: detecting whether the horizontal rotation angle and the vertical rotation angle of the camera are respectively within the corresponding preset ranges according to the camera posture, and if not, prohibiting tracking.

[0090] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0091] It can be understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program includes program instructions, and the computer program can be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0092] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program includes program instructions. When the program instructions are executed by a processor, the processor executes the following steps: detecting that a predetermined close-up area is located within a current panoramic image; determining that a tracking target is at a safe distance within the current panoramic image and that a tracking target is at a safe distance within a preset close-up panoramic screen; entering a close-up tracking mode to display a close-up of the predetermined close-up area.

[0093] It is understandable that in some embodiments, the processor may also execute other steps of the tracking control method described in the above embodiment when executing the program instructions stored in the storage medium. For details, please refer to the above embodiment and will not be repeated here. The storage medium may be a computer-readable storage medium that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk.

[0094] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0095] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0096] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0097] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a client, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0098] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A tracking control method, characterized in that: The tracking control method comprises: Detecting that the predetermined close-up area is located within the current panoramic image; Determining that the tracked target is at a safe distance within the current panoramic image and that the tracked target is at a safe distance within a preset close-up panoramic image; Entering a close-up tracking mode to display a close-up of the predetermined close-up area.

2. The tracking control method according to claim 1, characterized in that: The determining that the tracking target is at a safe distance in the current panoramic image and the tracking target is at a safe distance in a preset close-up panoramic picture specifically includes: Obtaining the preset panoramic ROI coordinates mapped to any predetermined close-up area in the current panoramic image according to the camera posture; It is detected whether the tracking target is at a safe distance within the current panoramic image and a preset close-up panoramic picture formed by the preset panoramic ROI coordinates.

3. The tracking control method according to claim 2, characterized in that: The step of obtaining, according to the camera posture, a preset panoramic ROI coordinate mapped to any predetermined close-up area in the current panoramic image specifically includes: Get the camera's tracking target's posture position coordinates and corresponding field of view angle in the current panoramic image; Obtaining the preset position center coordinates of the center of the predetermined close-up area in the current panoramic image according to the acquired posture position coordinates, the corresponding field of view angle, and the posture position coordinates of the predetermined close-up area; The preset panoramic ROI coordinates of each corresponding predetermined close-up area are obtained according to the current panoramic image coordinates and the acquired preset position center coordinates.

4. The tracking control method according to claim 3, characterized in that: The step of obtaining the preset position center coordinates of the center of the predetermined close-up area in the current panoramic image according to the acquired posture position coordinates, the corresponding field of view angle, and the posture position coordinates of the predetermined close-up area specifically includes: According to the current posture position coordinates p obtained by the camera c ={α c ,β c ,r c }、Corresponding field of view {FOV x ,FOV y } and the posture position coordinates p of the predetermined close-up area i ={α i ,β i ,r i }, using the formula Calculate and obtain the preset position center coordinates {x c ,y c }; where α is the horizontal rotation angle of the camera, β is the vertical rotation angle of the camera, r is the zoom ratio of the camera, and x o and o is the center coordinate of the current panoramic image.

5. The tracking control method according to claim 4, characterized in that: The step of obtaining the preset panoramic ROI coordinates of each corresponding predetermined close-up area according to the current panoramic image coordinates and the acquired preset position center coordinates specifically includes: According to the current panoramic image coordinates and the acquired preset position center coordinates {x c ,y c }, using the formula Calculate and obtain the preset panoramic ROI coordinates ROIF(i) of each corresponding predetermined close-up area; wherein the current panoramic image coordinates are set to {0,0,1,1}, x o and o is the center coordinate of the current panoramic image.

6. The tracking control method according to claim 2, characterized in that: The detecting whether the tracking target is at a safe distance within the current panoramic image and a preset close-up panoramic picture formed by the preset panoramic ROI coordinates specifically includes: It is respectively detected whether the boundary distance between the tracking target and the current panoramic image and the boundary distance between the tracking target and the preset close-up panoramic picture exceed a first preset threshold and a second preset threshold. If the boundary distance between the tracking target and the current panoramic image exceeds the first preset threshold and the boundary distance between the tracking target and the preset close-up panoramic picture exceeds the second preset threshold, the tracking target is at a safe distance within the current panoramic image and the preset close-up panoramic picture.

7. The tracking control method according to claim 2, characterized in that: The detecting whether the tracking target is at a safe distance within the current panoramic image and a preset close-up panoramic picture formed by the preset panoramic ROI coordinates specifically includes: The Mahalanobis distance / Euclidean distance between the center of the tracking target and the center of the current panoramic image and the preset close-up panoramic picture are calculated respectively. If the calculated Mahalanobis distance / Euclidean distance between the tracking target and the center of the current panoramic image is lower than the third preset threshold, and the Mahalanobis distance / Euclidean distance between the tracking target and the center of the preset close-up panoramic picture is lower than the fourth preset threshold, the tracking target is at a safe distance within the current panoramic image and the preset close-up panoramic picture.

8. The tracking control method according to claim 1, characterized in that: After entering the close-up tracking mode to display the predetermined close-up area in close-up, the method further includes: Check whether the tracking target is still at a safe distance within the current panoramic image and the preset close-up panoramic image. If so, maintain the current close-up tracking mode; if not, exit the close-up tracking mode and resume the normal tracking mode.

9. The tracking control method according to claim 1, characterized in that: The detecting that the predetermined close-up area is located in the current panoramic image specifically includes: Obtaining preset close-up ROI coordinates mapped to any predetermined close-up area in the current panoramic image according to the camera posture; According to the preset close-up ROI coordinates of each predetermined close-up area and the current panoramic image coordinates, the proportion of the predetermined close-up area in the current panoramic image is obtained, and it is detected whether the proportion exceeds the fifth preset threshold. If the proportion exceeds the fifth preset threshold, the predetermined close-up area is located in the current panoramic image; or, the distance between the center of the preset close-up ROI coordinates of each predetermined close-up area and the center of the current panoramic image is calculated, and it is detected whether the distance is lower than the sixth preset threshold. If so, the predetermined close-up area is located in the current panoramic image.

10. The tracking control method according to claim 1, characterized in that: The tracking control method further includes: detecting whether the horizontal rotation angle and the vertical rotation angle of the camera are respectively within the corresponding preset ranges according to the camera posture, and if not, prohibiting tracking.

11. A tracking control device, characterized in that: The method comprises a unit for executing the method according to any one of claims 1 to 10.

12. A computer device, characterized in that: The computer device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 10 when executing the computer program.

13. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the method according to any one of claims 1 to 10 can be implemented.