Object Tracking Method, Object Tracking System and Controller

Abstract

The present application discloses a target tracking method, a target tracking system and a controller. The method includes: acquiring a target image of a target; determining the pixel coordinates of the centroid point on the upper surface of the target and the overall feature image of the upper surface according to the target image; determining the three-dimensional physical coordinates of the centroid point on the upper surface of the target according to the pitch angle of the image acquisition device, the pixel coordinates of the centroid point on the upper surface of the target and the height; determining a first deflection angle and a second deflection angle of the target relative to the vertical line of the optical center of the image acquisition device according to the three-dimensional physical coordinates of the centroid point on the upper surface of the target; transforming the overall feature image of the upper surface of the target according to the first deflection angle and the second deflection angle to obtain a front view image of the target; and tracking the target according to the front view image. The present application can extract the upper surface features of the target at the front view angle under a large tilt angle at a long distance according to the deflection angle, and accurately track and locate the target by using the extracted features during the precise positioning process.

Description

Technical Field

[0001] This application relates to the technical field of target tracking, and particularly to a target tracking method, a target tracking system, and a controller. Background Art

[0002] In the field of construction machinery, an image acquisition device is usually used to acquire an image of the object being lifted to determine its position. The images captured by the image acquisition device at a long distance and a large tilt angle are the top surface and side images of the target object, and the image captured at a front view angle is the top surface image of the target object. Therefore, there are obvious differences in the images of the same object under two different viewing angles, and there are also obvious differences in the extracted image feature information. However, during the automatic hoisting process, the target object is often selected at a long distance and a large tilt angle, and the tracking and positioning of the target object are carried out at a front view angle. The obvious differences in the image feature information easily lead to the failure of the target object tracking and positioning, thus failing to meet the requirements of automatic hoisting. Summary of the Invention

[0003] The purpose of the embodiments of this application is to provide a target tracking method, a target tracking system, and a controller to solve the problem of difficult tracking and positioning of the target object at a long distance and a large tilt angle.

[0004] To achieve the above purpose, the first aspect of this application provides a target tracking method, which is applied to construction machinery. The construction machinery includes a boom and an image acquisition device, and the image acquisition device is arranged on the boom. The method includes:

[0005] Obtain a target image of the target object from the images collected by the image acquisition device;

[0006] Determine the pixel coordinates of the centroid point on the top surface of the target object and the overall feature image of the top surface of the target object according to the target image;

[0007] Determine the three-dimensional physical coordinates of the centroid point on the top surface of the target object according to the pitch angle of the image acquisition device, the pixel coordinates of the centroid point on the top surface of the target object, and the height of the target object;

[0008] Determine the first deflection angle and the second deflection angle of the target object relative to the vertical line of the optical center of the image acquisition device according to the three-dimensional physical coordinates of the centroid point on the top surface of the target object;

[0009] Transform the overall feature image of the top surface of the target object according to the first deflection angle and the second deflection angle to obtain a front view image of the target object;

[0010] Track the target object according to the front view image.

[0011] In the embodiments of this application, obtaining a target image of the target object from the images collected by the image acquisition device includes:

[0012] Obtain an initial image, where the initial image is an image captured by an image acquisition device;

[0013] In response to a bounding box input for a target object, determine a target image of the target object from the initial image.

[0014] In an embodiment of the present application, determining the pixel coordinates of the centroid point on the upper surface of the target object and the overall feature image of the upper surface of the target object based on the target image includes:

[0015] Extract multiple edge contour lines in the target image;

[0016] Extract multiple connected regions based on the multiple edge contour lines;

[0017] According to the pixel coordinates of the multiple centroid points corresponding to the multiple connected regions, determine the centroid point on the upper surface of the target object from the multiple centroid points;

[0018] Determine the overall feature image of the upper surface of the target object according to the connected region corresponding to the centroid point on the upper surface.

[0019] In an embodiment of the present application, the three-dimensional physical coordinates of the centroid point on the upper surface of the target object satisfy formula (1):

[0020]

[0021] Wherein, x object is the X-axis coordinate of the three-dimensional physical coordinates of the centroid point on the upper surface of the target object, y object is the Y-axis coordinate of the three-dimensional physical coordinates of the centroid point on the upper surface of the target object, h object is the Z-axis coordinate of the three-dimensional physical coordinates of the centroid point on the upper surface of the target object, h is the distance from the optical center point of the image acquisition device to the ground, θ is the pitch angle of the image acquisition device, β is the half field of view angle of the image acquisition device in the X-axis direction, O'A is the difference in pixel values on the X-axis between the center point on the upper surface of the target object and the center point of the initial image, O'D is half of the width of the initial image, O'A' is the difference in pixel values on the Y-axis between the center point on the upper surface of the target object and the center point of the initial image, O'E is half of the height of the initial image, γ is the half field of view angle of the image acquisition device in the Y-axis direction, H object is the height of the target object, H armhead is the height of the tip of the boom from the ground, H armcam is the distance from the tip of the boom to the optical center of the image acquisition device.

[0022] In an embodiment of the present application, the first deflection angle and the second deflection angle of the target object relative to the plumb line of the tip of the boom satisfy formula (2):

[0023]

[0024] Where, ω is the first deflection angle of the target object relative to the vertical line of the optical center of the image acquisition device, φ is the second deflection angle of the target object relative to the vertical line of the optical center of the image acquisition device, and x object is the X-axis coordinate of the three-dimensional physical coordinates of the centroid point on the upper surface of the target object, and y object is the Y-axis coordinate of the three-dimensional physical coordinates of the centroid point on the upper surface of the target object, and h object is the Z-axis coordinate of the three-dimensional physical coordinates of the centroid point on the upper surface of the target object.

[0025] In the embodiments of the present application, determining the front view image of the target object according to the first deflection angle, the second deflection angle, and the overall feature image of the upper surface of the target object includes:

[0026] Determining a first affine transformation matrix according to the overall feature image of the upper surface of the target object and the first deflection angle;

[0027] Transforming the overall feature image of the upper surface of the target object based on the first affine transformation matrix to obtain an initial front view image of the target object;

[0028] Determining a second affine transformation matrix according to the initial front view image of the target object and the second deflection angle;

[0029] Transforming the initial front view image based on the second affine transformation matrix to obtain the front view image of the target object.

[0030] In the embodiments of the present application, determining the first affine transformation matrix according to the overall feature image of the upper surface of the target object and the first deflection angle includes:

[0031] Obtaining the first initial coordinate values of the four vertices of the overall feature image of the upper surface of the target object;

[0032] Determining the first coordinate values after affine transformation of the four vertices of the overall feature image of the upper surface according to the first initial coordinate values of the four vertices and the first deflection angle;

[0033] Determining the first affine transformation matrix according to the first coordinate values after affine transformation of the four vertices.

[0034] In the embodiments of the present application, determining the second affine transformation matrix according to the initial front view image of the target object and the second deflection angle includes:

[0035] Obtaining the second initial coordinate values of the four vertices of the initial front view image of the target object;

[0036] Determining the second coordinate values after affine transformation of the four vertices of the initial front view image of the target object according to the second initial coordinate values of the four vertices of the initial front view image of the target object and the second deflection angle;

[0037] Determine the second affine transformation matrix according to the second coordinate values of the four vertices after affine transformation.

[0038] The second aspect of the present application provides a controller, including:

[0039] A memory configured to store instructions; and

[0040] A processor configured to call the instructions from the memory and capable of implementing the above-mentioned object tracking method when executing the instructions.

[0041] The third aspect of the present application provides an object tracking system, including:

[0042] An image acquisition device configured to acquire an image of an object; and

[0043] The above-mentioned controller.

[0044] The fourth aspect of the present application provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to cause a machine to execute the above-mentioned object tracking method.

[0045] Through the above technical solutions, obtain the target image of the object from the images collected by the image acquisition device; determine the pixel coordinates of the centroid point on the upper surface of the object and the overall feature image of the upper surface of the object according to the target image; determine the three-dimensional physical coordinates of the centroid point on the upper surface of the object according to the pitch angle of the image acquisition device, the pixel coordinates of the centroid point on the upper surface of the object, and the height of the object; determine the first deflection angle and the second deflection angle of the centroid point on the upper surface of the object relative to the vertical line of the optical center of the image acquisition device according to the three-dimensional physical coordinates of the centroid point on the upper surface of the object; transform the overall feature image of the upper surface of the object according to the first deflection angle and the second deflection angle to obtain the front view image of the object; track the object according to the front view image. The present application can extract the upper surface features of the object at the front view angle under a long-distance and large tilt angle according to the deflection angle, and accurately track and locate the object by using the extracted features during the accurate positioning process.

[0046] Other features and advantages of the embodiments of the present application will be described in detail in the subsequent specific implementation part. Description of the Drawings

[0047] The drawings are used to provide a further understanding of the embodiments of the present application, and constitute a part of the specification. Together with the following specific implementation, they are used to explain the embodiments of the present application, but do not constitute a limitation to the embodiments of the present application. In the drawings:

[0048] Figure 1 Schematically shows an application environment diagram of an object tracking method according to this embodiment;

[0049] Figure 2 Schematically shows a flowchart of an object tracking method according to an embodiment of the present application;

[0050] Figure 3 Schematically shows an extraction diagram of the overall feature image of the upper surface of an object according to an embodiment of the present application;

[0051] Figure 4 Schematically shows a conversion diagram from pixel coordinates to three-dimensional physical coordinates in the tilted state of an image acquisition device according to an embodiment of the present application;

[0052] Figure 5 Schematically shows a sectional view of the conversion diagram from pixel coordinates to three-dimensional physical coordinates in the tilted state of an image acquisition device according to an embodiment of the present application on the X-axis;

[0053] Figure 6 Schematically shows a sectional view of the conversion diagram from pixel coordinates to three-dimensional physical coordinates in the tilted state of an image acquisition device according to an embodiment of the present application on the Y-axis;

[0054] Figure 7 Schematically shows a horizontal and vertical affine transformation diagram according to an embodiment of the present application;

[0055] Figure 8 Schematically shows a structural block diagram of a controller according to an embodiment of the present application.

[0056] Description of reference numerals

[0057] 1 Image acquisition device 2 Controller

[0058] 3 Pan-tilt 41 Wireless bridge transmitter

[0059] 42 Wireless bridge receiver 5 Touch display screen Detailed implementation manners

[0060] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. It should be understood that the specific implementation manners described herein are only used to illustrate and explain the embodiments of the present application, and are not used to limit the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present application.

[0061] It should be noted that if there are directional indications (such as up, down, left, right, front, back...) involved in the embodiments of the present application, the directional indications are only used to explain the relative positional relationship, movement conditions, etc. between components in a specific posture (as shown in the drawings). If the specific posture changes, the directional indications will also change accordingly.

[0062] In addition, if there are descriptions such as "first", "second", etc. involved in the embodiments of the present application, the descriptions of "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one such feature. In addition, the technical solutions between various embodiments can be combined with each other, but it must be based on the fact that those skilled in the art can implement it. When the combination of technical solutions results in contradictions or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection required by the present application.

[0063] Figure 1 Schematically shows an application environment diagram of a target tracking method according to this embodiment. As Figure 1 shown, the target tracking method of the embodiments of the present application can be applied to an application environment as Figure 1 shown. In the embodiments of the present application, the target tracking method is applied to construction machinery, which can be a crane, including a boom and a target tracking system. The target tracking system can include an image acquisition device 1 and a controller 2. Among them, the image acquisition device 1 and the controller 2 are communicatively connected. As Figure 1 shown, the construction machinery can also include a pan-tilt 3, a wireless bridge, and a touch display screen 5. Among them, the wireless bridge includes a wireless bridge transmitter 41 and a wireless bridge receiver 42. The image acquisition device 1, the pan-tilt 3, and the wireless bridge transmitter 41 are all arranged at the top of the boom, and the wireless bridge receiver 42 is arranged on the side of the boom, keeping perpendicular to the wireless bridge transmitter 41 for line-of-sight transmission. The controller 2 and the touch display screen 5 are arranged in the cab of the construction machinery. The wireless bridge transmitter 41 and the wireless bridge receiver 42 can realize long-distance wireless transmission of images, enabling the image information of the image acquisition device 1 at the tip of the boom to be transmitted to the controller 2 of the vehicle body in real time, reliably, and stably.

[0064] In the embodiment of the present application, the pan-tilt 3 can rotate uniformly in the horizontal and vertical planes and can rotate to a specified position according to the specified horizontal angle and pitch angle values. By controlling the rotation of the pan-tilt 3 in the horizontal and vertical directions for the image acquisition device 1, compared with fixedly installing the image acquisition device 1, the field of view is larger and the working area of the construction machinery is wider. The initial state of the pan-tilt 3 remains vertically downward, and its axis is consistent with the body of the construction machinery, that is, the X-axis is perpendicular to the boom direction, and the Y-axis is parallel to the boom direction. During the target tracking process, the controller 2 can control the pan-tilt 3 to rotate in the horizontal and vertical planes so that the image acquisition device 1 roughly aligns with the target object, so that the target object appears in the video image of the image acquisition device 1. During the rotation of the boom and the image acquisition device 1, the controller 2 can adjust the focal length change of the image acquisition device 1 in real time according to the distance from the optical center point of the image acquisition device 1 along the optical axis of the image acquisition device 1 to the ground, so as to always keep the size of the target object in the image basically the same. The operator can frame the target object on the video image by touching the touch screen 5 in a touch screen manner, so as to obtain the target matrix frame, so as to subsequently extract the overall feature image of the upper surface of the target object from the rectangular frame, and track and accurately locate the target object according to the overall feature image of the upper surface of the target object. During the accurate positioning process, the deviation values of the center of the target object and the center of the hook on the X-axis and Y-axis can be calculated in real time according to the deviation between the center point of the positioning target object and the pixel value of the image center, and finally the unmanned accurate automatic hoisting can be realized.

[0065] Figure 2 Schematically shows a flowchart of a target object tracking method according to an embodiment of the present application. As Figure 2 shown, the embodiment of the present application provides a target object tracking method, which is applied to construction machinery. The construction machinery includes a boom and an image acquisition device, and the image acquisition device is arranged on the boom. The method may include the following steps.

[0066] Step 201: Obtain a target image of a target object from the image collected by the image acquisition device.

[0067] In the embodiment of the present application, in the field of construction machinery, an image acquisition device is usually used to acquire an image of the object to be lifted, and then the object to be lifted is tracked according to the image of the object to be lifted to determine its position. Therefore, in the embodiment of the present application, the pan-tilt can be controlled to rotate in the horizontal and vertical planes so that the image acquisition device roughly aligns with the target object, so as to collect a larger range of images containing the target object to be tracked through the image acquisition device, and then obtain the target image of the target object to be tracked from the image collected by the image acquisition device. Among them, the image acquisition device may be a monocular camera, and both the pan-tilt and the image acquisition device are arranged at the top of the boom.

[0068] Step 202: Determine the pixel coordinates of the centroid point of the upper surface of the target object and the overall feature image of the upper surface of the target object based on the target image.

[0069] In the embodiment of the present application, the images captured by the image acquisition device at a long distance and a large tilt angle are the images of the upper surface and the side surface of the target object, and the image captured at the front view angle is the image of the upper surface of the target object. To achieve accurate target tracking and positioning, it is required to be at the front view angle. There are obvious differences between the images of the same object at two different view angles, and there are also obvious differences in the extracted image feature information. The obvious difference features are likely to cause the failure of target tracking and positioning, unable to track the target object in real time, and thus unable to meet the requirements of automatic hoisting. Therefore, in the embodiment of the present application, the overall feature image of the upper surface of the target object can be extracted first, and the centroid of the overall feature image of the upper surface is used as the center point of the target image for tracking. Specifically, the pixel coordinates of the centroid point of the upper surface of the target object and the overall feature image of the upper surface of the target object can be determined according to the acquired target image. Among them, the pixel coordinates of the centroid point of the upper surface of the target object and the overall feature image of the upper surface of the target object can complement each other or be obtained independently. In one example, the pixel coordinates of the centroid point of the upper surface of the target object can be determined first according to the target image, and then the overall feature image of the upper surface of the target object can be determined according to the pixel coordinates of the centroid point of the upper surface of the target object. In another example, the pixel coordinates of the centroid point of the upper surface of the target object can be determined first according to the overall feature image of the upper surface of the target object. In yet another example, the overall feature image of the upper surface of the target object and the pixel coordinates of the centroid point of the upper surface of the target object can be determined respectively according to the target image.

[0070] Step 203: Determine the three-dimensional physical coordinates of the centroid point of the upper surface of the target object according to the pitch angle of the image acquisition device, the pixel coordinates of the centroid point of the upper surface of the target object, and the height of the target object.

[0071] In the embodiment of the present application, the centroid point of the overall feature image of the upper surface of the target object can be used as the center point of the target image to track the target object. Using the centroid point of the overall feature image of the upper surface of the target object as the center point of the target image to track the target object can convert the coordinates of the center point of the target image to the three-dimensional physical coordinate values in the camera coordinate system, that is, convert the pixel coordinates of the centroid point of the upper surface of the target object into three-dimensional physical coordinates. Specifically, first, the video image of the image acquisition device in the ground area and the projection point of the optical center point of the image acquisition device on the ground can be determined according to the pitch angle of the image acquisition device, and then the three-dimensional physical coordinates of the centroid point of the upper surface of the target object in the camera coordinate system can be determined in combination with the pixel coordinates of the centroid point of the upper surface of the target object and the height of the target object.

[0072] Step 204: Determine the first deflection angle and the second deflection angle of the target object relative to the plumb line of the optical center of the image acquisition device according to the three-dimensional physical coordinates of the centroid point of the upper surface of the target object.

[0073] In the embodiment of the present application, according to the three-dimensional physical coordinates of the centroid point on the upper surface of the target object, the deflection angles of the target object relative to the projection point of the arm tip on the X-axis and the Y-axis can be obtained, that is, the first deflection angle and the second deflection angle. According to the first deflection angle and the second deflection angle, the affine transformation matrices in the horizontal and vertical directions can be calculated respectively. After the affine transformation, the target object image under the front view angle can be obtained, and finally the feature extraction of the target object front view image under a large tilt angle at a long distance can be realized. Using the deflection angles of the target object relative to the projection point of the arm tip on the X-axis and the Y-axis as the basis for obtaining the target object image under the front view angle can make the subsequent target image feature extraction and tracking positioning more accurate.

[0074] Step 205: Transform the overall feature image of the upper surface of the target object according to the first deflection angle and the second deflection angle to obtain the front view image of the target object.

[0075] In the embodiment of the present application, by performing affine transformations on the overall feature image of the upper surface of the target object in the horizontal and vertical directions successively, the front view image of the target object can be obtained. It should be noted that the order of the two affine transformations can be horizontal first and then vertical, or vertical first and then horizontal. In the embodiment of the present application, the case of horizontal first and then vertical is taken as an example for illustration. Specifically, first, the overall feature image of the upper surface of the target object can be subjected to an affine transformation in the horizontal direction according to the deflection angle of the target object relative to the projection point of the arm tip on the X-axis, that is, the first deflection angle, and then the image after the horizontal affine transformation can be subjected to an affine transformation in the vertical direction according to the deflection angle of the target object relative to the projection point of the arm tip on the Y-axis, that is, the second deflection angle.

[0076] Step 206: Track the target object according to the front view image.

[0077] In the embodiment of the present application, after obtaining the front view image of the target object, the image features of the target object under the finally obtained front view angle can be extracted through the SURF (Speeded Up Robust Features) algorithm, that is, the features of the target object front view image under a large tilt angle are extracted, so that during the precise positioning process, the target object can be tracked and positioned in real time using the extracted image features to achieve unmanned precise automatic hoisting.

[0078] Through the above technical solution, a target image of a target object is obtained from the images collected by an image acquisition device; according to the target image, the pixel coordinates of the centroid point on the upper surface of the target object and the overall feature image of the upper surface of the target object are determined; according to the pitch angle of the image acquisition device, the pixel coordinates of the centroid point on the upper surface of the target object, and the height of the target object, the three-dimensional physical coordinates of the centroid point on the upper surface of the target object are determined; according to the three-dimensional physical coordinates of the centroid point on the upper surface of the target object, the first deflection angle and the second deflection angle of the target object relative to the vertical plumb line of the optical center of the image acquisition device are determined; according to the first deflection angle and the second deflection angle, the overall feature image of the upper surface of the target object is transformed to obtain a front view image of the target object; and the target object is tracked according to the front view image. The present application can frame the target and extract the upper surface features of the target at a front view angle under a large tilt angle at a long distance according to the deflection angle, and accurately track and locate the target object by using the extracted features during the accurate positioning process.

[0079] In an embodiment of the present application, step 201, obtaining a target image of a target object from the images collected by an image acquisition device may include:

[0080] An initial image is obtained, and the initial image is an image collected by the image acquisition device;

[0081] In response to a frame selection input for the target object, the target image of the target object is determined from the initial image.

[0082] In an embodiment of the present application, the image acquisition device can be controlled to rotate the pan-tilt head to collect an image of the target object. Wherein, the pan-tilt head is initially in a vertically downward state, and its axis is consistent with the vehicle body, that is, the X-axis is perpendicular to the boom direction, and the Y-axis is parallel to the boom direction. During the rotation of the boom and the image acquisition device, the focal length of the image acquisition device can be adjusted in real time according to the distance from the optical center point of the image acquisition device along the optical axis of the image acquisition device to the ground, so as to keep the size of the target object in the image basically the same at all times. Specifically, the controller can control the pan-tilt head to rotate in the horizontal and vertical planes to roughly align the image acquisition device with the target object first. At this time, the target object will appear in the video image of the image acquisition device. The operator frames the target object on the video image on the display screen in the cab in a touch-screen manner, so as to obtain the target image of the target object.

[0083] Figure 3 Schematically shows an extraction diagram of an overall feature image of the upper surface of a target object according to an embodiment of the present application. As Figure 3 shown, in an embodiment of the present application, step 202, determining the pixel coordinates of the centroid point on the upper surface of the target object and the overall feature image of the upper surface of the target object according to the target image may include:

[0084] Extract multiple edge contour lines in the target image;

[0085] Extract multiple connected regions based on multiple overall feature lines;

[0086] Based on the pixel coordinates of the multiple centroid points corresponding to the multiple connected regions, determine the centroid point of the upper surface of the target object from the multiple centroid points;

[0087] Determine the overall feature image of the upper surface of the target object according to the connected region corresponding to the centroid point of the upper surface.

[0088] In the embodiment of the present application, the controller can first grayscale and denoise the target image of the selected target object by Gaussian filtering to obtain the smoothed selected image, and then perform OTSU threshold segmentation and Canny edge extraction calculation on the smoothed image to obtain multiple edge contour lines. Multiple connected regions can be extracted based on the multiple edge contour lines. Calculate the pixel coordinates of the centroid points of the multiple connected regions respectively. From the multiple centroid points, the centroid point of the upper surface of the target object can be determined, and the overall feature image of the upper surface of the target object is determined according to the connected region corresponding to the centroid point of the upper surface. That is to say, according to the pixel coordinates of the multiple centroid points corresponding to the multiple connected regions, the side contour of the target object can be removed, and finally the overall feature image of the upper surface of the target object is obtained. As Figure 2 shown, in an example, the pixel coordinates of the centroid point of the upper surface contour of the target object are (x, y), and the pixel coordinates of the centroid points of the two side contours of the target object are (x1, y1), (x2, y2). Since the origin is located in the upper left corner of the image, the pixel coordinates of the centroid point of the upper surface of the target object are greater than those of the two side centroid points, satisfying x>x1 and x>x2. Therefore, the one with the largest X-axis coordinate value is taken as the centroid point of the upper surface of the target object, and thus the upper surface image of the target object is obtained.

[0089] Figure 4 Schematically shows a conversion diagram from pixel coordinates to three-dimensional physical coordinates in an inclined state of an image acquisition device according to an embodiment of the present application. As Figure 4 shown, in the embodiment of the present application, the three-dimensional physical coordinates of the centroid point of the upper surface of the target object satisfy formula (1):

[0090]

[0091] where x object is the X-axis coordinate of the three-dimensional physical coordinates of the centroid point of the upper surface of the target object, y object is the Y-axis coordinate of the three-dimensional physical coordinates of the centroid point of the upper surface of the target object, h objectis the Z - axis coordinate of the three - dimensional physical coordinates of the centroid point on the upper surface of the target object, h is the distance from the optical center point of the image acquisition device to the ground, θ is the pitch angle of the image acquisition device, β is the half - field - of - view angle of the image acquisition device in the X - axis direction, O'A is the difference in pixel values on the X - axis between the center point on the upper surface of the target object and the center point of the initial image, O'D is half of the width of the initial target image, O'A' is the difference in pixel values on the Y - axis between the center point on the upper surface of the target object and the center point of the initial image, O'E is half of the height of the initial target image, γ is the half - field - of - view angle of the image acquisition device in the Y - axis direction, H object is the height of the target object, H armhead is the height of the tip of the boom from the ground, H armcam is the distance from the tip of the boom to the optical center of the image acquisition device.

[0092] In the embodiment of the present application, according to the pitch angle of the image acquisition device and the pixel value coordinates of the centroid point on the upper surface of the target object, the three - dimensional physical coordinate values of the centroid of the target object with the optical center of the image acquisition device as the coordinate origin can be calculated. As Figure 3 shown, the shaded part is the video image of the ground area of the image acquisition device. O is the projection of the optical center point of the image acquisition device on the ground, O' is the center point of the video image in the tilted state, x' and y' are the coordinate values of the center point of the target object on the X and Y axes, and h is the distance from the optical center point of the image acquisition device to the ground.

[0093] Figure 5 Schematically shows a sectional view of the conversion of pixel coordinates to three - dimensional physical coordinates in the X - axis of an image acquisition device in a tilted state according to an embodiment of the present application. As Figure 5 shown, CD is the field - of - view range of the image acquisition device in the X - axis direction on the ground, O' is the center point of the initial image, A is the center point of the target object, θ is the pitch angle of the image acquisition device, β is the half - field - of - view angle of the image acquisition device in the X - axis direction, O'A is the difference in pixel values on the X - axis between the center point on the upper surface of the target object and the center point of the initial image, and O'D is half of the width of the initial target image. From Figure 5 it can be known that:

[0094]

[0095] Thus, the angle value can be calculated as:

[0096]

[0097] The final physical distance of y', that is, x object value is:

[0098]

[0099] Wherein, when O'A is located in the upper half of the image, x object is positive, and when O'A is located in the lower half of the image, x object is negative.

[0100] Figure 6 Schematically shows a sectional view of the Y-axis of the conversion diagram from pixel coordinates to three-dimensional physical coordinates in the tilted state of an image acquisition device according to an embodiment of the present application. As Figure 6 shown, O' is the center point of the initial image, A' is the center point of the target object, O'E is half of the field of view range of the image acquisition device in the Y-axis direction on the ground and half of the pixel value of the initial target image height, γ is the half field of view angle of the camera in the Y-axis direction, and O'A' is the difference in pixel values between the center point of the target object and the center point of the image on the Y-axis. From Figure 6 it can be known that:

[0101]

[0102] Thus, the angle value of can be calculated as:

[0103]

[0104] According to Figure 4 it can be obtained that:

[0105]

[0106] Therefore, the physical distance of x', that is, y object value is:

[0107]

[0108] In the embodiment of the present application, when the target object has a certain height, the height can be set to H object , then the actual physical distance calculation value of the corresponding target point A relative to the projection point of the image acquisition device is:

[0109]

[0110] According to the height H armhead of the crane arm tip from the ground, the distance H armcam from the arm tip to the camera optical center, and the self-height H object of the target object, the height h object from the target object to the camera optical center of the image acquisition device can be obtained. Among them, the self-height H object of the target object is a preset value set in advance. The height h object from the target object to the camera optical center of the image acquisition device satisfies the following formula:

[0111] h object = Harmhead -H armcam -H object

[0112] From the above formula, the three-dimensional coordinate value (x object , y object , h object ) of the centroid of the target object with the optical center of the camera as the coordinate origin can be calculated based on the centroid pixel coordinates of the target object.

[0113] As Figure 1 shown, in the embodiment of the present application, the first deflection angle and the second deflection angle of the target object relative to the plumb line of the tip of the boom satisfy formula (2):

[0114]

[0115] where ω is the first deflection angle of the target object relative to the plumb line of the optical center of the image acquisition device, φ is the second deflection angle of the target object relative to the plumb line of the optical center of the image acquisition device, x object is the X-axis coordinate of the three-dimensional physical coordinates of the centroid point on the upper surface of the target object, y object is the Y-axis coordinate of the three-dimensional physical coordinates of the centroid point on the upper surface of the target object, and h object is the Z-axis coordinate of the three-dimensional physical coordinates of the centroid point on the upper surface of the target object.

[0116] Figure 7 Schematically shows a horizontal and vertical affine transformation diagram according to an embodiment of the present application. As Figure 7 shown, in the embodiment of the present application, determining the front view image of the target object based on the first deflection angle, the second deflection angle, and the overall feature image of the upper surface of the target object may include:

[0117] Determining a first affine transformation matrix based on the overall feature image of the upper surface of the target object and the first deflection angle;

[0118] Transforming the overall feature image of the upper surface of the target object based on the first affine transformation matrix to obtain an initial front view image of the target object;

[0119] Determining a second affine transformation matrix based on the initial front view image of the target object and the second deflection angle;

[0120] Transforming the initial front view image based on the second affine transformation matrix to obtain the front view image of the target object.

[0121] In the embodiment of the present application, an orthographic image of the target object can be obtained by performing affine transformations on the overall feature image of the upper surface of the extracted target object in the horizontal direction and the vertical direction successively. It should be noted that the order of the two affine transformations can be horizontal direction first and then vertical direction, or vertical direction first and then horizontal direction. In the embodiment of the present application, the case of horizontal direction first and then vertical direction is taken as an example for illustration. Specifically, first, the first affine transformation matrix can be determined according to the deflection angle of the target object relative to the projection point of the arm tip on the X-axis, that is, the first deflection angle, and the overall feature image of the upper surface of the target object. Based on the first affine transformation matrix, a horizontal affine transformation is performed on the overall feature image of the upper surface of the target object to obtain an image after horizontal affine transformation, that is, the initial orthographic image of the target object. Then, the second affine transformation matrix is determined according to the initial orthographic image of the target object and the deflection angle of the target object relative to the projection point of the arm tip on the Y-axis, that is, the second deflection angle. Based on the second affine transformation matrix, a vertical affine transformation is performed on the image after horizontal affine transformation, that is, the initial orthographic image, to obtain the orthographic image of the target object.

[0122] In the embodiment of the present application, determining the first affine transformation matrix according to the overall feature image of the upper surface of the target object and the first deflection angle may include:

[0123] Obtain the first initial coordinate values of the four vertices of the overall feature image of the upper surface of the target object;

[0124] Determine the first coordinate values after affine transformation of the four vertices of the overall feature image of the upper surface according to the first initial coordinate values of the four vertices and the first deflection angle;

[0125] Determine the first affine transformation matrix according to the first coordinate values after affine transformation of the four vertices.

[0126] In the embodiment of the present application, the controller can first obtain the first initial coordinate values (0, 0), (W, 0), (0, H), (W, H) of the four vertices of the overall feature image of the upper surface of the target object. According to the first initial coordinate values of the four vertices and the first deflection angle, that is, the deflection angle ω in the horizontal direction, the coordinate values of the four vertices of the overall feature image of the upper surface of the target object after affine transformation can be obtained According to the four corresponding coordinate values, the first affine transformation matrix can be obtained, and thus the image after horizontal affine transformation, that is, the initial orthographic image of the target object, can be obtained.

[0127] In the embodiment of the present application, determining the second affine transformation matrix according to the initial orthographic image of the target object and the second deflection angle may include:

[0128] Obtain the second initial coordinate values of the four vertices of the initial orthographic image of the target object;

[0129] Determine the second coordinate values after affine transformation of the four vertices of the initial front-facing image of the target object according to the second initial coordinate values and the second deflection angle of the four vertices of the initial front-facing image of the target object;

[0130] Determine the second affine transformation matrix according to the second coordinate values after affine transformation of the four vertices.

[0131] In the embodiment of the present application, the controller can obtain the first initial coordinate values (0, 0), (W, W×tanφ), (0, H - W×tanφ), (W, H) of the four vertices of the initial front-facing image of the target object. According to the second initial coordinate values and the second deflection angle, i.e., the deflection angle φ in the vertical direction, of the four vertices, the second coordinate values after affine transformation of the four vertices of the initial front-facing image of the target object can be obtained as (0, 0), (W, 0), (0, H - W×tanφ), (W, H - W×tanφ). According to the four corresponding coordinate values, the second affine transformation matrix can be obtained, and finally the front-facing image of the target object can be obtained.

[0132] Figure 8 Schematically shows a structural block diagram of a controller according to an embodiment of the present application. As Figure 8 shown, the embodiment of the present application provides a controller, which may include:

[0133] A memory 810, configured to store instructions; and

[0134] A processor 820, configured to call instructions from the memory 810 and be able to implement the above-mentioned target object tracking method when executing the instructions.

[0135] Specifically, in the embodiment of the present application, the processor 820 may be configured to:

[0136] Obtain the target image of the target object from the images collected by the image acquisition device;

[0137] Determine the pixel coordinates of the centroid point on the upper surface of the target object and the overall feature image of the upper surface of the target object according to the target image;

[0138] Determine the three-dimensional physical coordinates of the centroid point on the upper surface of the target object according to the pitch angle of the image acquisition device, the pixel coordinates of the centroid point on the upper surface of the target object, and the height of the target object;

[0139] Determine the first deflection angle and the second deflection angle of the target object relative to the plumb line of the optical center of the image acquisition device according to the three-dimensional physical coordinates of the centroid point on the upper surface of the target object;

[0140] Transform the overall feature image of the upper surface of the target object according to the first deflection angle and the second deflection angle to obtain the front-facing image of the target object;

[0141] Track the target object based on the frontal image.

[0142] Further, the processor 820 may also be configured to:

[0143] Obtain an initial image, where the initial image is an image collected by the image acquisition device;

[0144] In response to the box selection input for the target object, determine the target image of the target object from the initial image.

[0145] Further, the processor 820 may also be configured to:

[0146] Extract multiple edge contour lines from the target image;

[0147] Extract multiple connected regions based on the multiple edge contour lines;

[0148] According to the pixel coordinates of the multiple centroid points corresponding to the multiple connected regions, determine the centroid point of the upper surface of the target object from the multiple centroid points;

[0149] Determine the overall feature image of the upper surface of the target object according to the connected region corresponding to the centroid point of the upper surface.

[0150] In the embodiments of the present application, the three-dimensional physical coordinates of the centroid point of the upper surface of the target object satisfy formula (1):

[0151]

[0152] where x object is the X-axis coordinate of the three-dimensional physical coordinates of the centroid point of the upper surface of the target object, y object is the Y-axis coordinate of the three-dimensional physical coordinates of the centroid point of the upper surface of the target object, h object is the Z-axis coordinate of the three-dimensional physical coordinates of the centroid point of the upper surface of the target object, h is the distance from the optical center point of the image acquisition device to the ground, θ is the pitch angle of the image acquisition device, β is the half field of view angle of the image acquisition device in the X-axis direction, O'A is the difference in pixel values on the X-axis between the center point of the upper surface of the target object and the center point of the initial image, O'D is half of the width of the initial image, O'A' is the difference in pixel values on the Y-axis between the center point of the upper surface of the target object and the center point of the initial image, O'E is half of the height of the initial image, γ is the half field of view angle of the image acquisition device in the Y-axis direction, H object is the height of the target object, H armhead is the height of the tip of the boom from the ground, H armcam is the distance from the tip of the boom to the optical center of the image acquisition device.

[0153] In the embodiments of the present application, the first deflection angle and the second deflection angle of the target object relative to the plumb line of the tip of the boom satisfy formula (2):

[0154]

[0155] Among them, ω is the first deflection angle of the target relative to the vertical line of the optical center of the image acquisition device, φ is the second deflection angle of the target relative to the vertical line of the optical center of the image acquisition device, and x object is the X-axis coordinate of the three-dimensional physical coordinates of the centroid point of the upper surface of the target, and y object is the Y-axis coordinate of the three-dimensional physical coordinates of the centroid point of the upper surface of the target, and h object is the Z-axis coordinate of the three-dimensional physical coordinates of the centroid point of the upper surface of the target.

[0156] Furthermore, the processor 820 can also be configured to:

[0157] Determine a first affine transformation matrix according to the overall feature image of the upper surface of the target and the first deflection angle;

[0158] Transform the overall feature image of the upper surface of the target based on the first affine transformation matrix to obtain an initial front view image of the target;

[0159] Determine a second affine transformation matrix according to the initial front view image of the target and the second deflection angle;

[0160] Transform the initial front view image based on the second affine transformation matrix to obtain a front view image of the target.

[0161] Furthermore, the processor 820 can also be configured to:

[0162] Obtain the first initial coordinate values of the four vertices of the overall feature image of the upper surface of the target;

[0163] Determine the first coordinate values after affine transformation of the four vertices of the overall feature image of the upper surface according to the first initial coordinate values of the four vertices and the first deflection angle;

[0164] Determine the first affine transformation matrix according to the first coordinate values after affine transformation of the four vertices.

[0165] Furthermore, the processor 820 can also be configured to:

[0166] Obtain the second initial coordinate values of the four vertices of the initial front view image of the target;

[0167] Determine the second coordinate values after affine transformation of the four vertices of the initial front view image of the target according to the second initial coordinate values of the four vertices of the initial front view image of the target and the second deflection angle;

[0168] Determine the second affine transformation matrix according to the second coordinate values after affine transformation of the four vertices.

[0169] Through the above technical solution, a target image of a target object is obtained from the image acquired by the image acquisition device; the pixel coordinates of the centroid point on the upper surface of the target object and the overall feature image of the upper surface of the target object are determined according to the target image; the three-dimensional physical coordinates of the centroid point on the upper surface of the target object are determined according to the pitch angle of the image acquisition device, the pixel coordinates of the centroid point on the upper surface of the target object, and the height of the target object; the first deflection angle and the second deflection angle of the target object relative to the plumb line of the optical center of the image acquisition device are determined according to the three-dimensional physical coordinates of the centroid point on the upper surface of the target object; the overall feature image of the upper surface of the target object is transformed according to the first deflection angle and the second deflection angle to obtain a front view image of the target object; and the target object is tracked according to the front view image. The present application can extract the upper surface features of the target object at the front view angle under a large tilt angle at a long distance according to the deflection angle. During the precise positioning process, the features extracted are used to accurately track and position the target object, and the deviation values of the center of the target object and the center of the hook on the X-axis and Y-axis are calculated in real time according to the deviation between the center point of the positioned target object and the pixel value of the image center, so as to achieve unmanned precise automatic hoisting.

[0170] As Figure 1 shown, an embodiment of the present application further provides a target object tracking system, which may include:

[0171] An image acquisition device 1 configured to acquire an image of a target object; and

[0172] The above-mentioned controller 2.

[0173] In an embodiment of the present application, the controller 2 controls the rotation of the pan-tilt so that the image acquisition device 1 can acquire an image including the target object. Through the wireless bridge, the controller 2 can receive the image acquired by the image acquisition device 1 and obtain the target image of the target object from the image acquired by the image acquisition device 1. According to the target image, the controller 2 determines the pixel coordinates of the centroid point on the upper surface of the target object and the overall feature image of the upper surface of the target object. The three-dimensional physical coordinates of the centroid point on the upper surface of the target object can be determined according to the pitch angle of the image acquisition device 1, the pixel coordinates of the centroid point on the upper surface of the target object, and the height of the target object. During the hoisting process, the controller 2 determines the first deflection angle and the second deflection angle of the target object relative to the plumb line of the optical center of the image acquisition device 1 in real time according to the three-dimensional physical coordinates of the centroid point on the upper surface of the target object, and transforms the overall feature image of the upper surface of the target object according to the first deflection angle and the second deflection angle to obtain a front view image of the target object, and finally tracks the target object according to the front view image to achieve unmanned precise automatic hoisting.

[0174] An embodiment of the present application further provides a machine-readable storage medium, on which instructions are stored, and the instructions are used to cause a machine to execute the above-mentioned target object tracking method.

[0175] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0176] The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for realizing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0177] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that realizes the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0178] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for realizing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0179] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and a memory.

[0180] The memory may include non-permanent memory in the computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.

[0181] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0182] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0183] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included within the scope of the claims of the present application.

Claims

1. A target tracking method, characterized in that, Applied to construction machinery, the construction machinery includes a boom and an image acquisition device, the image acquisition device is arranged on the boom, and the method includes: Obtain a target image of a target object from the images acquired by the image acquisition device; Determine the pixel coordinates of the centroid point of the upper surface of the target object and the overall feature image of the upper surface of the target object according to the target image; Determine the three-dimensional physical coordinates of the centroid point of the upper surface of the target object according to the pitch angle of the image acquisition device, the pixel coordinates of the centroid point of the upper surface of the target object, and the height of the target object; Determine a first deflection angle and a second deflection angle of the target object relative to the vertical line of the optical center of the image acquisition device according to the three-dimensional physical coordinates of the centroid point of the upper surface of the target object; Transform the overall feature image of the upper surface of the target object according to the first deflection angle and the second deflection angle to obtain a front view image of the target object; Track the target object according to the front view image.

2. The method according to claim 1, wherein Obtaining a target image of a target object from the images acquired by the image acquisition device includes: Obtain an initial image, where the initial image is the image acquired by the image acquisition device; In response to a bounding box input for the target object, determine the target image of the target object from the initial image.

3. The method according to claim 1, wherein The determining the pixel coordinates of the centroid point of the upper surface of the target object and the overall feature image of the upper surface of the target object according to the target image includes: Extract multiple edge contour lines in the target image; Extract multiple connected regions based on the multiple edge contour lines; According to the pixel coordinates of multiple centroid points corresponding to multiple connected regions, determine the centroid point of the upper surface of the target object from the multiple centroid points; Determine the overall feature image of the upper surface of the target object according to the connected region corresponding to the centroid point of the upper surface.

4. The method according to claim 1, wherein The three-dimensional physical coordinates of the centroid point of the upper surface of the target object satisfy formula (1): where x object is the X-axis coordinate of the three-dimensional physical coordinates of the centroid point of the upper surface of the target object, y object is the Y-axis coordinate of the three-dimensional physical coordinates of the centroid point of the upper surface of the target object, h object is the Z-axis coordinate of the three-dimensional physical coordinates of the centroid point of the upper surface of the target object, h is the distance from the optical center point of the image acquisition device to the ground, θ is the pitch angle of the image acquisition device, β is the half field of view angle of the image acquisition device in the X-axis direction, O'A is the difference in pixel values on the X-axis between the center point of the upper surface of the target object and the center point of the initial image, O'D is half of the width of the initial image, O'A' is the difference in pixel values on the Y-axis between the center point of the upper surface of the target object and the center point of the initial image, O'E is half of the height of the initial image, γ is the half field of view angle of the image acquisition device in the Y-axis direction, H object is the height of the target object, H armhead is the height of the boom tip from the ground, H armcam is the distance from the boom tip to the optical center of the image acquisition device.

5. The method according to claim 1, wherein The first deflection angle and the second deflection angle of the target object relative to the vertical line of the tip of the boom satisfy formula (2): Where, ω is the first deflection angle of the target object relative to the vertical line of the optical center of the image acquisition device, φ is the second deflection angle of the target object relative to the vertical line of the optical center of the image acquisition device, x object is the X-axis coordinate of the three-dimensional physical coordinates of the centroid point on the upper surface of the target object, y object is the Y-axis coordinate of the three-dimensional physical coordinates of the centroid point on the upper surface of the target object, h object is the Z-axis coordinate of the three-dimensional physical coordinates of the centroid point on the upper surface of the target object.

6. The method according to claim 1, wherein The determining the front view image of the target object according to the first deflection angle, the second deflection angle, and the overall feature image of the upper surface of the target object includes: Determine a first affine transformation matrix according to the overall feature image of the upper surface of the target object and the first deflection angle; Transform the overall feature image of the upper surface of the target object based on the first affine transformation matrix to obtain an initial front view image of the target object; Determine a second affine transformation matrix according to the initial front view image of the target object and the second deflection angle; Transform the initial front view image based on the second affine transformation matrix to obtain the front view image of the target object.

7. The method according to claim 6, wherein The determining the first affine transformation matrix according to the overall feature image of the upper surface of the target object and the first deflection angle includes: Obtain the first initial coordinate values of the four vertices of the overall feature image of the upper surface of the target object; Determine the first coordinate values after affine transformation of the four vertices of the overall feature image of the upper surface according to the first initial coordinate values of the four vertices and the first deflection angle; Determine the first affine transformation matrix according to the first coordinate values after affine transformation of the four vertices.

8. The method according to claim 6, wherein The determining the second affine transformation matrix according to the initial front view image of the target object and the second deflection angle includes: Obtain the second initial coordinate values of the four vertices of the initial front view image of the target object; Determine the second coordinate values after affine transformation of the four vertices of the initial front view image of the target object according to the second initial coordinate values of the four vertices of the initial front view image of the target object and the second deflection angle; Determine the second affine transformation matrix according to the second coordinate values after affine transformation of the four vertices.

9. A controller, characterized in that, Includes: A memory configured to store instructions; And A processor configured to call the instructions from the memory and capable of implementing the target object tracking method according to any one of claims 1 to 8 when executing the instructions.

10. A target tracking system, characterized in that, Includes: An image acquisition device configured to acquire an image of a target object; And A controller according to claim 9.

11. A machine-readable storage medium, characterized in that Instructions are stored on the machine-readable storage medium, and the instructions are used to cause the machine to execute the target object tracking method according to any one of claims 1 to 8.

Images