Target tracking method and system

Abstract

The embodiment of the invention relates to a target tracking method and system. The method comprises the following steps: acquiring a first image of a target object shot by a wide-angle lens; determining first position information of a target object in the first image; based on the first position information and a view position relation, whether a target object is located in the second view range or not is determined, a determination result is obtained, and the view position relation represents the position relation between the first view range and the second view range; under the condition that the determination result shows that the target object is located in the second visual field range, obtaining a second image of the target object shot by the telephoto lens; and performing target tracking on a target object based on the second image. Therefore, the accuracy of target tracking can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of monitoring, in particular to a target tracking method and system. BACKGROUND

[0002] With the continuous development of monitoring technology, PTZ cameras have been widely used in the field of security monitoring. For example, PTZ cameras realize all-around coverage of the monitoring area through rotating and tilting mechanisms, and target tracking technology enables the camera to automatically track moving targets, improving monitoring efficiency and quality.

[0003] Existing target tracking technology mainly relies on a single lens to capture targets and realizes tracking through image processing algorithms. However, the single lens system often has insufficient tracking accuracy when dealing with long-distance targets or high-speed moving targets. In addition, when the target moves at a high speed or suddenly changes direction, the single lens system has difficulty in quickly adjusting the focal length and viewing angle, which can easily lead to target loss, thereby reducing the accuracy of target tracking. SUMMARY

[0004] In view of this, to solve some or all of the above technical problems, the present application provides a target tracking method and system.

[0005] In a first aspect, the present application provides a target tracking method, which comprises:

[0006] obtaining a first image of a target object captured by a wide-angle lens; wherein a first field of view range of the wide-angle lens and a second field of view range of a long-focus lens have an intersection;

[0007] determining first position information of the target object in the first image;

[0008] determining whether the target object is located in the second field of view range based on the first position information and a field of view position relationship, to obtain a determination result, wherein the field of view position relationship represents a position relationship between the first field of view range and the second field of view range;

[0009] in a case where the determination result indicates that the target object is located in the second field of view range, obtaining a second image of the target object captured by the long-focus lens;

[0010] tracking the target object based on the second image.

[0011] In one possible implementation, after determining whether the target object is located in the second field of view range, the method further comprises:

[0012] In a case where the determination result indicates that the target object is not located in the second field of view range, the long-focus lens is controlled to move based on the first position information, so that the target object is located in the second field of view range.

[0013] In a case where the target object is located in the second field of view range, the long-focus lens is controlled to capture a second image of the target object.

[0014] In one possible implementation, the target tracking of the target object based on the second image includes:

[0015] Determining second position information of the target object in the second image based on a previous image of the second image, the previous image being an image captured before the second image, and the second position information indicating a predicted position of the target object in the second image.

[0016] Detecting third position information of the target object in the second image, the third position information indicating a measured position of the target object in the second image.

[0017] The target tracking of the target object based on the second position information and the third position information.

[0018] In one possible implementation, the second position information indicates a position of the target object in a first detection box in the previous image, and the third position information indicates a position of the target object in a second detection box in the second image.

[0019] The target tracking of the target object based on the second position information and the third position information includes:

[0020] Determining a target parameter of a first image region in the first detection box and a second image region in the second detection box based on the second position information and the third position information, the target parameter including an intersection over union or a similarity.

[0021] In a case where the target parameter is greater than or equal to a preset threshold, it is determined that the target object in the previous image and the target object in the second image represent a same object, and the wide-angle lens and / or the long-focus lens is controlled to track and capture the target object in the second image.

[0022] In one possible implementation, in a case where the target parameter is less than the preset threshold, the method further includes:

[0023] Determining that the target object in the previous image and the target object in the second image represent different objects.

[0024] determining corresponding speed information of the target object in the preceding image;

[0025] determining the corresponding speed information of the target object in the preceding image as corresponding speed information of the target object in the second image;

[0026] determining the third position information and the speed information as state information of the target object in the second image.

[0027] In one possible implementation, the determining, based on the preceding image of the second image, of the second position information of the target object in the second image comprises:

[0028] determining fifth position information of the target object in the preceding image of the second image, and corresponding speed information of the target object in the preceding image, wherein the speed information represents a speed of the target object moving from a shooting scene of the preceding image to a shooting scene of the second image;

[0029] determining the second position information of the target object in the second image based on the fifth position information and the speed information.

[0030] In one possible implementation, the field of view position relationship represents a conversion relationship between an image coordinate system of the wide-angle lens and an image coordinate system of the long-focus lens; and

[0031] The determining, based on the first position information and the field of view position relationship, of whether the target object is located in the second field of view range comprises:

[0032] determining an image coordinate representing the first position information, and determining an image coordinate of the target object after coordinate conversion according to the conversion relationship and the image coordinate of the first position information;

[0033] determining whether the target object is located in the second field of view range based on the image coordinate after coordinate conversion.

[0034] In one possible implementation, a first field of view center of the first field of view range and a second field of view center of the second field of view range represent a same position.

[0035] In a second aspect, an embodiment of the present application provides a target tracking device, and the device comprises:

[0036] an acquisition unit configured to acquire a first image of a target object captured by a wide-angle lens; wherein a first field of view range of the wide-angle lens and a second field of view range of a long-focus lens have an intersection;

[0037] The first determining unit is configured to determine first position information of the target object in the first image.

[0038] The second determining unit is configured to determine, based on the first position information and a field-of-view position relationship, whether the target object is located in the second field-of-view range, to obtain a determination result, wherein the field-of-view position relationship represents a position relationship between the first field-of-view range and the second field-of-view range.

[0039] The third determining unit is configured to acquire, in a case where the determination result indicates that the target object is located in the second field-of-view range, a second image of the target object captured by the long-focus lens.

[0040] The tracking unit is configured to perform target tracking on the target object based on the second image.

[0041] In one possible implementation, after determining whether the target object is located in the second field-of-view range, the apparatus further includes:

[0042] The first control unit is configured to, in a case where the determination result indicates that the target object is not located in the second field-of-view range, control the long-focus lens to move based on the first position information, so that the target object is located in the second field-of-view range.

[0043] The second control unit is configured to, in a case where the target object is located in the second field-of-view range, control the long-focus lens to capture a second image of the target object.

[0044] In one possible implementation, the performing target tracking on the target object based on the second image includes:

[0045] determining second position information of the target object in the second image based on a preceding image of the second image, wherein the preceding image is an image captured before the second image, and the second position information represents a predicted position of the target object in the second image.

[0046] detecting third position information of the target object in the second image, wherein the third position information represents an actually measured position of the target object in the second image.

[0047] performing target tracking on the target object based on the second position information and the third position information.

[0048] In one possible implementation, the second position information represents a position of the target object in a first detection frame in the preceding image, and the third position information represents a position of the target object in a second detection frame in the second image.

[0049] The target tracking of the target object based on the second position information and the third position information comprises:

[0050] The target parameter of the first image region in the first detection frame and the second image region in the second detection frame is determined based on the second position information and the third position information, wherein the target parameter comprises an intersection over union or a similarity;

[0051] In a case where the target parameter is greater than or equal to a preset threshold, it is determined that the target object in the previous image and the target object in the second image represent a same object, and the wide-angle lens and / or the long-focus lens is controlled to track and shoot the target object in the second image.

[0052] In one possible implementation, in a case where the target parameter is less than the preset threshold, the device further comprises:

[0053] A seventh determination unit is configured to determine that the target object in the previous image and the target object in the second image represent different objects.

[0054] A fourth determination unit is configured to determine speed information corresponding to the target object in the previous image.

[0055] A fifth determination unit is configured to determine the speed information corresponding to the target object in the previous image as speed information corresponding to the target object in the second image.

[0056] A sixth determination unit is configured to determine the third position information and the speed information as state information corresponding to the target object in the second image.

[0057] In one possible implementation, the determination of the second position information of the target object in the second image based on the previous image of the second image comprises:

[0058] The fifth position information of the target object in the previous image of the second image and speed information corresponding to the target object in the previous image are determined, wherein the speed information represents a speed of the target object moving from a shooting scene of the previous image to a shooting scene of the second image.

[0059] The second position information of the target object in the second image is determined based on the fifth position information and the speed information.

[0060] In one possible implementation, the field of view position relationship represents a conversion relationship between an image coordinate system of the wide-angle lens and an image coordinate system of the long-focus lens; and

[0061] determine whether the target object is located in the second field-of-view range based on the first position information and a field-of-view position relationship, the field-of-view position relationship representing a position relationship between the first field-of-view range and the second field-of-view range.

[0062] determine image coordinates representing the first position information, and determine image coordinates of the target object after coordinate conversion according to the conversion relationship and the image coordinates of the first position information;

[0063] determine whether the target object is located in the second field-of-view range based on the image coordinates of the target object after coordinate conversion.

[0064] In a possible implementation, a first field-of-view center of the first field-of-view range and a second field-of-view center of the second field-of-view range represent a same position.

[0065] In a third aspect, an embodiment of the present application provides a target tracking system, which includes a processing unit, a wide-angle lens and a long-focus lens, the processing unit being connected to the wide-angle lens and the long-focus lens respectively; wherein:

[0066] the wide-angle lens is configured to capture a first image of a target object; wherein a first field-of-view range of the wide-angle lens and a second field-of-view range of the long-focus lens have an intersection;

[0067] the processing unit is configured to: acquire the first image captured by the wide-angle lens; determine first position information of the target object in the first image; determine whether the target object is located in the second field-of-view range based on the first position information and a field-of-view position relationship, the field-of-view position relationship representing a position relationship between the first field-of-view range and the second field-of-view range, to obtain a determination result; in a case where the determination result indicates that the target object is located in the second field-of-view range, acquire a second image of the target object captured by the long-focus lens; and perform target tracking on the target object based on the second image.

[0068] In a possible implementation, the system further includes a control unit, the control unit being connected to the processing unit, the wide-angle lens and the long-focus lens respectively; and

[0069] the control unit is configured to:

[0070] in a case where the determination result indicates that the target object is not located in the second field-of-view range, control the long-focus lens to move so that the target object is located in the second field-of-view range based on the first position information;

[0071] in a case where the target object is located in the second field-of-view range, control the long-focus lens to capture a second image of the target object.

[0072] In a possible implementation, the system further comprises a control unit; the control unit is connected to the processing unit, the wide-angle lens and the long-focus lens respectively;

[0073] The processing unit is further configured to: determine second position information of the target object in the second image; and send the second position information to the control unit.

[0074] The control unit is configured to: based on the second position information, control the first field of view center of the wide-angle lens and the second field of view center of the long-focus lens to be displayed at a center position of a subsequent image of the second image; the first field of view center is a center of the first field of view range; the second field of view center is a center of the second field of view range; and the subsequent image represents a next frame image of the second image.

[0075] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising:

[0076] a memory configured to store a computer program;

[0077] a processor configured to execute the computer program stored in the memory, and when the computer program is executed, any embodiment of the target tracking method of the first aspect is implemented.

[0078] In a fifth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and when the computer program is executed by a processor, any embodiment of the target tracking method of the first aspect is implemented.

[0079] In a sixth aspect, an embodiment of the present application provides a computer program product, which comprises computer readable code, and when the computer readable code runs on a device, the processor in the device implements any embodiment of the target tracking method of the first aspect.

[0080] The target tracking method provided in the embodiments of the present application can acquire a first image of a target object captured by a wide-angle lens, wherein a first field of view range of the wide-angle lens intersects with a second field of view range of a long-focus lens, then first position information of the target object in the first image is determined, and then whether the target object is located in the second field of view range is determined based on the first position information and a field of view position relationship, to obtain a determination result, wherein the field of view position relationship represents a position relationship between the first field of view range and the second field of view range, subsequently, in a case where the determination result indicates that the target object is located in the second field of view range, a second image of the target object captured by the long-focus lens is acquired, and finally, target tracking is performed on the target object based on the second image. In this way, a more suitable lens can be selected from the long-focus lens and the wide-angle lens for image capturing based on whether the target object is located in the second field of view range, and then target tracking is performed by using the captured image, so that the accuracy of target tracking can be improved.

[0081] The target tracking system provided in the embodiments of the present application comprises a processing unit, a wide-angle lens and a long-focus lens, the processing unit is connected with the wide-angle lens and the long-focus lens respectively, wherein the wide-angle lens is configured to capture a first image of a target object, wherein a first field of view range of the wide-angle lens intersects with a second field of view range of the long-focus lens; the processing unit is configured to acquire the first image captured by the wide-angle lens, determine first position information of the target object in the first image, determine whether the target object is located in the second field of view range based on the first position information and a field of view position relationship, to obtain a determination result, wherein the field of view position relationship represents a position relationship between the first field of view range and the second field of view range, acquire a second image of the target object captured by the long-focus lens in a case where the determination result indicates that the target object is located in the second field of view range, and perform target tracking on the target object based on the second image. In this way, since the long-focus lens can capture more detailed information, target tracking performed by using the second image captured by the long-focus lens can improve the accuracy of target tracking. BRIEF DESCRIPTION OF DRAWINGS

[0082] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0083] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the accompanying drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows, and obviously, other drawings can also be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0084] One or more embodiments are illustrated by way of example in the drawings in which like reference numerals indicate similar elements, and as such, clearances and the sizes of parts between the figures can not be to scale. The figures in the drawings are not necessarily to scale, and the proportions of certain parts have been exaggerated to help to improve understanding of the embodiments.

[0085] Figure 1 A flowchart of a target tracking method provided by an embodiment of the present application;

[0086] Figure 2 A flowchart of another target tracking method provided by an embodiment of the present application;

[0087] Figure 3A An application scenario diagram of a target tracking method provided by an embodiment of the present application;

[0088] Figure 3B A flowchart of yet another target tracking method provided by an embodiment of the present application;

[0089] Figure 3C A flowchart of still another target tracking method provided by an embodiment of the present application;

[0090] Figure 4 A structural diagram of a target tracking device provided by an embodiment of the present application;

[0091] Figure 5 A structural diagram of a target tracking system provided by an embodiment of the present application;

[0092] Figure 6 A structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0093] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. It should be apparent that the described embodiments are only part of the embodiments of the present application and are not intended to limit the scope of the present application. It should be noted that the relative arrangement of the components and steps illustrated in these embodiments, numerical expressions, and numerical values are not intended to limit the scope of the present application unless otherwise specifically stated.

[0094] Those skilled in the art can understand that the terms "first", "second", etc. in the embodiments of the present application are only used to distinguish different steps, devices or modules, and do not represent any specific technical meaning, nor represent the logical order between them.

[0095] It should also be understood that in the present embodiment, "a plurality of" can mean two or more, and "at least one" can mean one, two or more.

[0096] It should also be understood that, whenever used in the present application, the terms "comprise", "comprising", "comprises" and / or "comprising" should be interpreted as referring to the parts, data and / or structures described in the application, but not precluding the presence of one or more other parts, data and / or structures.

[0097] In addition, the term "and / or" in the present application is merely a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " in the present application generally represents an "or" relationship between the front and rear associated objects.

[0098] It should also be understood that the description of the various embodiments of the present application focuses on the differences between the various embodiments, and the same or similar parts can be referred to each other, and for the sake of brevity, will not be repeated.

[0099] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the application or its application or uses.

[0100] Techniques, methods, and devices known to those of ordinary skill in the relevant art can not be discussed in detail, but should be considered part of the specification where appropriate.

[0101] It should be noted that similar reference numbers and letters refer to similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0102] It should be noted that the embodiments and features in the embodiments in the present application can be combined with each other without conflict. In order to understand the embodiments of the present application, the following will be described in detail with reference to the drawings and in combination with the embodiments. Obviously, the described embodiments are part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application.

[0103] In order to solve the technical problem of low accuracy of target tracking in the prior art, the present application provides a target tracking method and system, which can improve the accuracy of target tracking.

[0104] Figure 1A flowchart of a target tracking method provided in an embodiment of the present application is shown. The method can be applied to one or more electronic devices such as a chip, a smart phone, a notebook computer, a desktop computer, a portable computer, a server, and the like. In addition, the execution subject of the method can be hardware or software. When the execution subject is hardware, the execution subject can be one or more of the electronic devices. For example, a single electronic device can execute the method, or multiple electronic devices can cooperate with each other to execute the method. When the execution subject is software, the method can be implemented as one or more software or software modules. No specific limitation is made herein.

[0105] As shown in Figure 1 , the method specifically includes the following steps.

[0106] In step 101, a first image of a target object captured by a wide-angle lens is obtained. The first field of view range of the wide-angle lens and the second field of view range of a long-focus lens have an intersection.

[0107] In the embodiment, the wide-angle lens can be a lens with an angle of view of more than 90 degrees. The angle of view of the wide-angle lens can be greater than that of a standard lens. Compared with the standard lens, the wide-angle lens can capture a larger area of a scene within the same shooting distance range.

[0108] The long-focus lens can be a lens with a focal length greater than 80 millimeters. The long-focus lens can capture a subject at a long distance and magnify it into the picture.

[0109] The target object can be any object in the image captured by the wide-angle lens. For example, the target object can be a tracking object such as a person, an animal, or the like.

[0110] The first image can be an image of the target object captured by the wide-angle lens. The target object can be included in the first image.

[0111] The first field of view range can be the field of view range of the wide-angle lens.

[0112] The second field of view range can be the field of view range of the long-focus lens.

[0113] In some optional implementations of the embodiment, a first field of view center of the first field of view range and a second field of view center of the second field of view range represent the same position.

[0114] Here, the first field of view center of the first field of view range and the second field of view center of the second field of view range can be controlled to be the same position by fixing the relative positions of the long-focus lens and the wide-angle lens. For example, the long-focus lens and the wide-angle lens can be fixedly connected, or the long-focus lens and the wide-angle lens can be synchronously moved (for example, the long-focus lens and the wide-angle lens can be arranged on a gimbal).

[0115] It can be understood that, in the optional implementation manner above, the first field of view center of the first field of view range and the second field of view center of the second field of view range can represent the same position, so that the target object photographed by the long-focus lens and the wide-angle lens is located at the image center, and thus the accuracy of target tracking can be further improved.

[0116] In step 102, first position information of the target object in the first image is determined.

[0117] In this embodiment, the first position information can represent the position of the target object in the first image. For example, the first position information can represent the position of a detection box containing the target object in the first image, or can represent the position of a pixel point belonging to the target object in the first image.

[0118] In step 103, whether the target object is located in the second field of view range is determined based on the first position information and a field of view position relationship, to obtain a determination result, where the field of view position relationship represents a position relationship between the first field of view range and the second field of view range.

[0119] In this embodiment, the field of view position relationship can represent a conversion relationship for converting an image in the first field of view range into an image in the second field of view range, or can represent a conversion relationship for converting an image in the second field of view range into an image in the first field of view range. The field of view position relationship can be represented by a matrix, a vector, a formula, etc.

[0120] The determination result can represent whether the target object is located in the second field of view range.

[0121] In step 104, in a case where the determination result indicates that the target object is located in the second field of view range, a second image of the target object photographed by the long-focus lens is acquired.

[0122] In this embodiment, the second image can be an image containing the target object photographed by the long-focus lens.

[0123] In some optional implementation manners of this embodiment, after the determination of whether the target object is located in the second field of view range, the following step can be further performed:

[0124] In step one, in a case where the determination result indicates that the target object is not located in the second field of view range, the long-focus lens is controlled to move based on the first position information, so that the target object is located in the second field of view range.

[0125] As an example, the long-focus lens can be controlled to move based on the first position information and the field-of-view position relationship, so that the target object is located in the second field-of-view range.

[0126] As another example, the long-focus lens can also be controlled to move based on the first position information and the second field-of-view range, so that the target object is located in the second field-of-view range.

[0127] Step two, in the case where the target object is located in the second field-of-view range, the long-focus lens is controlled to capture a second image of the target object.

[0128] It can be understood that in the optional implementation described above, in the case where the target object is not located in the second field-of-view range, the long-focus lens can be controlled to move, so that a second image containing the target object is captured. Here, since the long-focus lens can capture more detailed information, controlling the long-focus lens to move based on the first position information of the target object in the first image and then using the second image captured by the long-focus lens to perform target tracking can improve the accuracy of target tracking.

[0129] Step 105, performing target tracking on the target object based on the second image.

[0130] In this embodiment, target tracking on the target object can be achieved by determining whether the target object in the second image represents the same object as the target object in the first image.

[0131] As an example, a machine learning algorithm can be used to perform target tracking on the target object based on the second image.

[0132] In some optional implementations of this embodiment, the field-of-view position relationship represents a conversion relationship between an image coordinate system of the wide-angle lens and an image coordinate system of the long-focus lens. For example, the field-of-view position relationship can be a matrix used to convert an image in the first field-of-view range to an image in the second field-of-view range.

[0133] On this basis, the first position information and the field-of-view position relationship can be used to determine whether the target object is located in the second field-of-view range in the following manner:

[0134] First, determine the image coordinates representing the first position information, and determine the image coordinates of the target object after coordinate conversion according to the conversion relationship and the image coordinates of the first position information.

[0135] The image coordinates of the first position information can be represented by a matrix. As an example, the image coordinates of the first position information can be (x1, y1, w1, h1). Wherein (x1, y1) represents the position coordinates of the upper left corner of the target object in the first image, which is the coordinate under the image coordinate system of the wide-angle lens, w1 represents the width of the detection box of the target object in the first image, and h1 represents the height of the detection box of the target object in the first image.

[0136] The conversion relationship can be a matrix representing the field of view position relationship. For example, the conversion relationship can be a matrix for converting an image in the first field of view range to an image in the second field of view range.

[0137] Secondly, whether the target object is located in the second field of view range is determined based on the image coordinates after the coordinate conversion.

[0138] As an example, the above first step can be represented by the following formula one:

[0139] Bbox1 = Bboxw * T

[0140] Formula one

[0141] Wherein, Bboxw represents the image coordinates (x1, y1, w1, h1) of the first position information, T represents the conversion relationship, * represents matrix multiplication operation, and Bbox1 represents the image coordinates (x2, y2, w2, h2) of the target object after the coordinate conversion. Wherein (x1, y1) represents the position coordinates of the upper left corner of the target object in the first image, which is the coordinate under the image coordinate system of the wide-angle lens, w1 represents the width of the detection box of the target object in the first image, and h1 represents the height of the detection box of the target object in the first image. (x2, y2) represents the position coordinates of the upper left corner of the target object in the second image, which is the coordinate under the image coordinate system of the long-focus lens, w2 represents the width of the detection box of the target object in the second image, and h2 represents the height of the detection box of the target object in the second image.

[0142] Further, if the resolution of the image captured by the long-focus lens is WxH, if any of the conditions x2>=0, y2>=0, w2

[0143] It can be understood that, in the optional implementation, the target object in the first image is converted into the target object in the second image based on the conversion relationship representing the field-of-view position relationship, and then it is determined whether the target object is located in the second field-of-view range, so that whether the target object is located in the second field-of-view range can be determined more accurately.

[0144] The target tracking method provided in the embodiments of the present application can obtain a first image of a target object captured by a wide-angle lens, wherein a first field-of-view range of the wide-angle lens and a second field-of-view range of a long-focus lens have an intersection, then first position information of the target object in the first image is determined, and then it is determined whether the target object is located in the second field-of-view range based on the first position information and a field-of-view position relationship, to obtain a determination result, wherein the field-of-view position relationship represents a position relationship between the first field-of-view range and the second field-of-view range, then a second image of the target object captured by the long-focus lens is obtained in a case where the determination result indicates that the target object is located in the second field-of-view range, and finally target tracking is performed on the target object based on the second image. Therefore, since the long-focus lens can capture more detailed information, target tracking performed based on the second image captured by the long-focus lens can improve the accuracy of target tracking.

[0145] Figure 2 A flowchart of another target tracking method provided in the embodiments of the present application is shown in FIG. 3. As shown in FIG. 3, the method specifically includes the following steps. Figure 2

[0146] In step 201, a first image of a target object captured by a wide-angle lens is obtained, wherein a first field-of-view range of the wide-angle lens and a second field-of-view range of a long-focus lens have an intersection.

[0147] In the embodiments, step 201 is basically the same as step 101 in the corresponding embodiments, and thus will not be described here again. Figure 1 In step 202, first position information of the target object in the first image is determined.

[0148] In the embodiments, step 202 is basically the same as step 102 in the corresponding embodiments, and thus will not be described here again.

[0149] Figure 1 In step 203, it is determined whether the target object is located in the second field-of-view range based on the first position information and a field-of-view position relationship, to obtain a determination result, wherein the field-of-view position relationship represents a position relationship between the first field-of-view range and the second field-of-view range.

[0150] In the embodiments, step 203 is basically the same as step 103 in the corresponding embodiments, and thus will not be described here again.

[0151] In the embodiments, step 203 is basically the same as step 103 in the corresponding embodiments, and thus will not be described here again. Figure 1 ​​The step 103 in the corresponding embodiment is basically the same, and thus will not be described here again.

[0152] At step 204, if the determination result indicates that the target object is located in the second field of view, a second image of the target object captured by the long-focus lens is acquired.

[0153] In this embodiment, the step 204 is basically the same as the step 104 in the corresponding embodiment, and thus will not be described here again. Figure 1 The step 104 in the corresponding embodiment is basically the same, and thus will not be described here again.

[0154] At step 205, second position information of the target object in the second image is determined based on a preceding image of the second image, wherein the preceding image is an image captured before the second image.

[0155] In this embodiment, the lens for capturing the preceding image of the second image can be the same as or different from the lens for capturing the second image. For example, the second image and the preceding image of the second image can both be captured via the long-focus lens. For another example, the preceding image of the second image can be captured via the wide-angle lens, and the second image can be captured via the long-focus lens.

[0156] The preceding image is an Nth image captured before the second image. N is a positive integer, for example, N can be 1, 2, or 3. When N is 1, the preceding image is the preceding image of the second image.

[0157] In other words, the frame extraction can be performed every N-1 frames, so as to obtain the preceding image of the second image, the second image, and the like.

[0158] The second position information can indicate a predicted position of the target object in the second image.

[0159] As an example, the position of the target object in the preceding image of the second image and the speed of the target object can be determined first. Thus, the second position information of the target object in the second image can be determined based on the above position and speed.

[0160] As another example, the preceding image of the second image can also be input into a pre-trained position prediction model, so as to determine the second position information of the target object in the second image.

[0161] The position prediction model can be a convolutional neural network model trained by using a deep learning algorithm.

[0162] At step 206, third position information of the target object in the second image is detected.

[0163] In the embodiment, the third position information represents a measured position of the target object in the second image. In other words, the third position information can represent a position of the target object in the second image detected (measured) by detection.

[0164] Here, a target detection algorithm can be used to detect the third position information of the target object in the second image.

[0165] In step 207, target tracking is performed on the target object based on the second position information and the third position information.

[0166] In the embodiment, whether the target object in the previous image and the target object in the second image are the same object can be determined by determining whether the position represented by the second position information and the position represented by the third position information match, thereby performing target tracking on the target object.

[0167] In some optional implementations of the embodiment, the second position information represents a position of the target object in a first detection box in the previous image. The third position information represents a position of the target object in a second detection box in the second image. The first detection box can be a detection box of the target object in the previous image. The second detection box can be a detection box of the target object in the second image.

[0168] On this basis, the target tracking can be performed on the target object based on the second position information and the third position information in the following manner:

[0169] First, a target parameter of a first image region in the first detection box and a second image region in the second detection box is determined based on the second position information and the third position information.

[0170] The target parameter includes an intersection-over-union or a similarity.

[0171] Second, in a case where the target parameter is greater than or equal to a preset threshold, it is determined that the target object in the previous image and the target object in the second image represent the same object, and the wide-angle lens and / or the long-focus lens is controlled to track and capture the target object in the second image.

[0172] It can be understood that in the above optional implementation, the similarity or the intersection-over-union of the first image region and the second image region can be used to more accurately determine whether the target object in the previous image and the target object in the second image are the same object, thereby further improving the accuracy of target object tracking.

[0173] Optionally, the target tracking can also be performed on the target object based on the second position information and the third position information in the following manner:

[0174] In a first step, an intersection-over-union of a first image region in the first bounding box and a second image region in the second bounding box is determined based on the second position information and the third position information.

[0175] In a second step, in a case where the target parameter is greater than or equal to a preset intersection-over-union threshold, it is determined that the target object in the previous image and the target object in the second image represent a same object, and the wide-angle lens and / or the long-focus lens are controlled to track and capture the target object in the second image; in a case where the target parameter is less than the preset intersection-over-union threshold, a similarity of the first image region in the first bounding box and the second image region in the second bounding box is determined based on the second position information and the third position information. In a case where the similarity is greater than or equal to a preset similarity threshold, it is determined that the target object in the previous image and the target object in the second image represent a same object, and the wide-angle lens and / or the long-focus lens are controlled to track and capture the target object in the second image; in a case where the similarity is less than the preset similarity threshold, it is determined that the target object in the previous image and the target object in the second image represent different objects.

[0176] It can be understood that, in the above scheme, the intersection-over-union can be used to determine whether the target object in the previous image of the second image and the target object in the second image are a same object, and only in a case where the result of the intersection-over-union determination indicates that the two objects are different objects, the similarity is used to further determine whether the two objects are a same object, so that the efficiency and accuracy of target tracking can be taken into account.

[0177] In some application scenarios of the above optional implementation, in a case where the target parameter is less than the preset threshold, the following steps can also be performed:

[0178] In a first step, it is determined that the target object in the previous image and the target object in the second image represent different objects.

[0179] In a second step, speed information corresponding to the target object in the previous image is determined.

[0180] The speed information can represent a speed of the target object in the previous image. Specifically, the speed information can include at least one of a speed of the target object in a horizontal coordinate direction, a speed of the target object in a vertical coordinate direction, a change speed of a width of a bounding box of the target object, and a change speed of a height of the bounding box of the target object.

[0181] In a third step, the speed information corresponding to the target object in the previous image is determined as speed information corresponding to the target object in the second image.

[0182] Fourthly, the third position information and the speed information are determined as the corresponding state information of the target object in the second image.

[0183] Here, the images other than the image of the first frame can have corresponding state information Mwt(Xt, Yt, Wt, Ht, VXt, VYt, VWt, VHt).

[0184] Wherein, (Xt, Yt, Wt, Ht) can represent the position of the target object in the previous image (denoted as the image of the tth frame) of the second image, and t is used to identify different image frames. Mwt represents the state information of the target object corresponding to the image of the tth frame. (Xt, Yt) represents the position coordinates of the top-left pixel of the target object in the image of the tth frame, Xt represents the horizontal coordinate, and Yt represents the vertical coordinate. Wt represents the width of the detection box of the target object in the image of the tth frame, and Ht represents the height of the detection box of the target object in the image of the tth frame. VXt represents the speed of the target object in the horizontal coordinate direction in the image of the tth frame, VYt represents the speed of the target object in the vertical coordinate direction in the image of the tth frame, VWt represents the speed of the target object in the width direction in the image of the tth frame, and VHt represents the speed of the target object in the height direction in the image of the tth frame.

[0185] Therefore, in the case that the target object in the image of the tth frame and the target object in the image of the t+1th frame represent different objects, it can be determined that the corresponding state information of the target object in the image of the t+1th frame is Mwt+1(Xt+1, Yt+1, Wt+1, Ht+1, VXt, VYt, VWt, VHt).

[0186] Wherein, (Xt+1, Yt+1, Wt+1, Ht+1) can represent the third position information, and (VXt, VYt, VWt, VHt) can represent the corresponding speed information of the target object in the previous image (i.e. the image of the tth frame).

[0187] It can be understood that in the above application scenario, in the case that the target object in the previous image of the image is not matched from the image, the third position information and the speed information can be determined as the corresponding state information of the target object in the second image.

[0188] In some optional implementations of the embodiment, the second position information of the target object in the second image can be determined based on the previous image of the second image in the following manner:

[0189] Firstly, fifth position information of the target object in the previous image of the second image and corresponding speed information of the target object in the previous image are determined.

[0190] The fifth position information can represent a measured position or a predicted position of the target object in a preceding image of the second image. The speed information represents a speed of the target object moving from a shooting scene of the preceding image to a shooting scene of the second image.

[0191] In a second step, the second position information of the target object in the second image is determined based on the fifth position information and the speed information.

[0192] Here, the state information of the target object in the preceding image of the second image can be Mwt(Xt, Yt, Wt, Ht, VXt, VYt, VWt, VHt).

[0193] Here, (Xt, Yt, Wt, Ht) can represent a position of the target object in the preceding image of the second image, and t is used to identify different image frames. Mwt represents state information of the target object corresponding to the t-th image frame. (Xt, Yt) represents a position coordinate of the top-left corner pixel of the target object in the t-th image frame, Xt represents a horizontal coordinate, and Yt represents a vertical coordinate. Wt represents a width of the detection box of the target object in the t-th image frame, and Ht represents a height of the detection box of the target object in the t-th image frame. VXt represents a speed of the target object in the t-th image frame in the horizontal coordinate direction, VYt represents a speed of the target object in the t-th image frame in the vertical coordinate direction, VWt represents a speed of the target object in the t-th image frame in the width direction, and VHt represents a speed of the target object in the t-th image frame in the height direction.

[0194] Thus, the position of the target object in the t+1-th image frame (represented by the second position information) can be predicted as Bbox_prediction(Xp, Yp, Wp, Hp).

[0195] Here, Xp=Xt+VXt, Yp=Yt+VYt, Wp=Wt+VWt, and Hp=Ht+VHt.

[0196] It can be understood that in the above optional implementation, the position of the target object in the preceding image of the second image and the corresponding speed of the target object in the preceding image can be used to predict the position of the target object in the second image, thereby improving the accuracy of determining the position of the target object in the second image, and further improving the accuracy of target tracking.

[0197] It should be noted that, in addition to the above-mentioned content, the present embodiment can also include Figure 1 the corresponding technical features described in the corresponding embodiments, and further achieve Figure 1 the technical effects of the target tracking method shown in the description, please refer to Figure 1For brevity, the relevant description is not repeated here.

[0198] The target tracking method provided by the embodiments of the present application realizes target tracking by the position of the target object in the second image predicted based on the preceding image of the second image and the position of the target object in the second image obtained by actually detecting the second image, thereby improving the accuracy of target tracking.

[0199] The embodiments of the present application are exemplarily described below, but it should be noted that the embodiments of the present application can have the features described below, but the following description does not constitute a limitation on the protection scope of the embodiments of the present application.

[0200] With the continuous development of monitoring technology, PTZ cameras have been widely used in security monitoring, video conferencing, live streaming and other fields. PTZ cameras realize omnidirectional coverage of the monitoring area through rotating and tilting mechanisms. Target tracking technology enables the camera to automatically track moving targets (i.e. the above-mentioned target objects), improving monitoring efficiency and quality.

[0201] Existing target tracking technology mainly relies on a single lens to capture targets and realizes tracking through image processing algorithms. However, single-lens systems often have problems such as insufficient tracking accuracy and slow response speed when dealing with long-distance targets or high-speed moving targets. In addition, when the target moves at high speed or suddenly changes direction, the single-lens system has difficulty in quickly adjusting the focal length and viewing angle, which can easily lead to target loss.

[0202] The dual-camera PTZ system integrates a long-focus lens and a wide-angle lens, which can theoretically improve tracking performance through the cooperation of long and short focal lengths. The wide-angle lens has a large field of view and is suitable for rough positioning of the target; while the long-focus lens can provide higher image resolution and is suitable for accurate tracking. However, in existing technologies, the control algorithm of the dual-camera PTZ often does not well combine the characteristics of the two lenses, resulting in the inability to maximize the advantages of the dual-camera system in actual application.

[0203] To solve the above problems, the present scheme combines the advantages of long-focus lenses and wide-angle lenses and realizes efficient target tracking in different scenarios through the cooperative work of the two. The long-focus lens is responsible for providing high-definition details of the target and is suitable for accurate tracking of long-distance targets; the wide-angle lens is responsible for monitoring a large field of view and ensures that the target can be quickly captured and repositioned when the target moves quickly or temporarily leaves the field of view of the long-focus lens. To address the problem of fast target object movement, the present scheme combines motion modeling and feature matching to compensate for the motion of the tracking target based on motion modeling tracking failure, thereby realizing accurate tracking of the target when the PTZ rotates quickly.

[0204] In summary, the dual-camera gimbal target tracking method provided in the scheme effectively solves the problems of insufficient tracking accuracy and slow response speed in the prior art by skillfully combining the advantages of wide-angle and long-focus lenses and optimizing the gimbal control algorithm.

[0205] The overall architecture of the camera includes a main chip (i.e., a processing unit), a long-focus lens, a wide-angle lens, and a dual-axis gimbal (i.e., a control unit). The field of view of the long-focus lens overlaps with that of the wide-angle lens, and the overlapping area is located at the center of the wide-angle lens, as shown in Figure 3A The wide-angle lens and the long-focus lens are calibrated to obtain the conversion relationship between the wide-angle image coordinate system and the long-focus image coordinate system, which is T, i.e., the above-mentioned field of view position relationship.

[0206] As shown in Figure 3A , the data of the long-focus lens and the wide-angle lens can be collected by the image acquisition module to obtain the long-focus image and the wide-angle image. The wide-angle image (i.e., the above-mentioned first image) is sent to the detection model to obtain the position of the target object in the wide-angle image (i.e., the above-mentioned first position information), which is recorded as Bboxw(x1, y1, w1, h1) in terms of the position of the top-left corner of the target object and the width and height, and the feature code Feature of the target object is also obtained. The position of the target object in the long-focus image coordinate system Bboxl(x2, y2, w2, h2) is calculated using formula 1-1 through the conversion relationship between the two lenses.

[0207] Formula 1-1

[0208] Wherein, Bboxw represents the image coordinates of the first position information, T represents the conversion relationship, * represents matrix multiplication, and Bboxl represents the image coordinates of the target object after coordinate conversion.

[0209] Let the resolution of the long-focus lens image be W*H. If any of the conditions x2>=0, y2>=0, w2<W, and h2<H is not met, the target object is not in the long-focus field of view (i.e., the above-mentioned second field of view range), in which case the image collected by the wide-angle lens can be continuously tracked, and the position information Bboxw of the target is fed back to the gimbal control system to correct the motion motor control parameters, moving the target to the center of the wide-angle lens, as shown in Figure 3B . At the same time, the position of the target in the long-focus lens Bboxl(x2, y2, w2, h2) is calculated through formula 1-1. When x2>=0, y2>0, w2<W, and h2<H are all met, the target is located in the long-focus field of view, and the tracking is switched to the long-focus image.

[0210] The target coordinate position (x, y, w, h) is used to model and track the speed and position of the target object. The state variables of the target in the wide-angle lens at frame t are denoted as Mwt(Xt, Yt, Wt, Ht, VXt, Vyt, VWt, VHt), which are the horizontal coordinate position of the upper left corner of the target, the vertical coordinate position of the upper left corner of the target, the width of the target object, the height of the target object, the speed of the target object in the horizontal coordinate direction, the speed of the target object in the vertical coordinate direction, the change speed of the target object in the width direction, and the change speed of the target object in the height direction. For the image at frame t+1, the target position (i.e., the second position information mentioned above) Bbox_prediction(Xp, Yp, Wp, Hp) of the current frame can be estimated based on Mt through formulas 1-2:

[0211] Xp = Xt + VXt

[0212] Yp = Yt + VYt

[0213] Wp = Wt + VWt

[0214] Hp = Ht + VHt

[0215] Formulas 1-2

[0216] The simultaneous detection model outputs the target position (i.e., the third position information mentioned above) Bbox_detection of the current frame detected.

[0217] If the intersection over union (IOU) of the target Bbox(Xd, Yd, Wd, Hd) found in Bbox_detection and Bbox_prediction is greater than a threshold, it is considered that the same target object is matched, and the state variables Mwt+1(Xt+1, Yt+1, Wt+1, Ht+1, VXt+1, Vyt+1, VWt+1, VHt+1) of the target are updated.

[0218] wherein Xt+1 = Xd, Yt+1 = Yd, Wt+1 = Wd, Ht+1 = Hd, VXt+1 = Xt+1 - Xt, VYt+1 = Yt+1 - Yt, VWt+1 = Wt+1 - Wt, and VHt+1 = Ht+1 - Ht. IOU = (Bbox1 ∩ Bbox2) / (Bbox1 ∪ Bbox2). Wherein Bbox1 represents the first image region mentioned above, and Bbox2 represents the second image region mentioned above.

[0219] For the target object that is not matched successfully (for example, the target object whose IOU is greater than the threshold value, or the target object that is not matched due to the fast moving speed of the target object), the features of the detection target object P1 and the tracking target object P2 that are not matched are calculated for similarity. For example, the cosine similarity can be used.

[0220] If the similarity is greater than the threshold value T1 of the matching success, it is determined that the target object is the same. At this time, the IOU of P1 and P2 is calculated. If the IOU is greater than the threshold value T2 of the matching success, the target tracking is performed. Otherwise, the position of the tracking target is corrected, and then the motion modeling is updated, that is, the third position information and the speed information are determined as the state information corresponding to the target object in the second image.

[0221] When the target object is switched from the wide-angle view to the long-focus view, the gimbal variable of the target in the wide-angle view is converted to the image Mlt in the long-focus lens through the two image coordinate system transformation relationship Mlt=Mwt*T.

[0222] Wherein, Mlt represents the image of the target object in the long-focus lens, Mwt represents the image of the target object in the wide-angle lens, and T represents the matrix for converting the image of the target object in the wide-angle lens to the image of the target object in the long-focus lens.

[0223] When the gimbal tracking is performed by using the long-focus image, the target position is corresponded from the long-focus coordinate system to the wide-angle coordinate system by using the inverse matrix of the two image coordinate system transformation relationship T, and the wide-angle image Bboxw=Bboxl*T-1 is obtained.

[0224] Wherein, Bboxl represents the image of the target object in the long-focus lens, Bboxw represents the image of the target object in the wide-angle lens, and T-1 represents the matrix for converting the image of the target object in the long-focus lens to the image of the target object in the wide-angle lens, that is, the inverse matrix of the above-mentioned matrix T.

[0225] Therefore, the gimbal control system can be controlled by using Bboxw to ensure that the target is kept in the center of the view.

[0226] When the target continuously moves and the gimbal moves to the limit position or the tracking algorithm loses the tracked target, the tracking is stopped, as shown in Figure 3C

[0227] The scheme can be applied to gimbal cameras, dual-camera cameras, ordinary cameras and the like. It can be used for locking the target for tracking, drawing the target trajectory, digital zooming and the like. It can be used in indoor, street, forest, farm and the like scenes.

[0228] ​It should be noted that in addition to the above, the present embodiment can also include the technical features described in the above embodiments, thereby achieving the technical effects of the above target tracking method. For details, please refer to the above description. For brevity, no further description is given here.

[0229] The target tracking method provided by the embodiments of the present application can ensure a longer tracking distance, improve the tracking accuracy and the stability of the gimbal control through the cooperation of the long-focus lens and the short-focus lens, and ensure the accuracy of the gimbal tracking through the motion compensation of the feature matching motion modeling module.

[0230] Figure 4 A structural schematic diagram of a target tracking device provided by the embodiments of the present application is shown. Specifically, it includes:

[0231] The acquisition unit 401 is configured to acquire a first image of a target object captured by a wide-angle lens; wherein a first field of view range of the wide-angle lens and a second field of view range of a long-focus lens have an intersection;

[0232] The first determination unit 402 is configured to determine first position information of the target object in the first image;

[0233] The second determination unit 403 is configured to determine whether the target object is located in the second field of view range based on the first position information and a field of view position relationship, to obtain a determination result, wherein the field of view position relationship represents a position relationship between the first field of view range and the second field of view range;

[0234] The third determination unit 404 is configured to acquire a second image of the target object captured by the long-focus lens in a case where the determination result indicates that the target object is located in the second field of view range;

[0235] The tracking unit 405 is configured to perform target tracking on the target object based on the second image.

[0236] After determining whether the target object is located in the second field of view range, the device further includes:

[0237] The first control unit (not shown in the figure) is configured to control the long-focus lens to move based on the first position information in a case where the determination result indicates that the target object is not located in the second field of view range, so as to make the target object located in the second field of view range.

[0238] The second control unit (not shown in the figure) is configured to control the long-focus lens to capture a second image of the target object in a case where the target object is located in the second field of view range.

[0239] In a possible implementation, the target tracking of the target object based on the second image comprises:

[0240] determining second position information of the target object in the second image based on a previous image of the second image, wherein the previous image is an image captured before the second image, and the second position information represents a predicted position of the target object in the second image;

[0241] detecting third position information of the target object in the second image, wherein the third position information represents a measured position of the target object in the second image;

[0242] tracking the target object based on the second position information and the third position information.

[0243] In a possible implementation, the second position information represents a position of the target object in a first detection box in the previous image, and the third position information represents a position of the target object in a second detection box in the second image; and

[0244] The target tracking of the target object based on the second position information and the third position information comprises:

[0245] determining a target parameter of a first image region in the first detection box and a second image region in the second detection box based on the second position information and the third position information, wherein the target parameter comprises an intersection over union or a similarity;

[0246] in a case where the target parameter is greater than or equal to a preset threshold, determining that the target object in the previous image and the target object in the second image represent a same object, and controlling the wide-angle lens and / or the long-focus lens to track and capture the target object in the second image.

[0247] In a possible implementation, in a case where the target parameter is less than the preset threshold, the apparatus further comprises:

[0248] a seventh determining unit (not shown in the figure) configured to determine that the target object in the previous image and the target object in the second image represent different objects;

[0249] a fourth determining unit (not shown in the figure) configured to determine corresponding speed information of the target object in the previous image;

[0250] a fifth determining unit (not shown in the figure) configured to determine the corresponding speed information of the target object in the previous image as corresponding speed information of the target object in the second image;

[0251] A sixth determining unit (not shown in the figure) is configured to determine the third position information and the speed information as state information of the target object in the second image.

[0252] In a possible implementation, the determining of the second position information of the target object in the second image based on the preceding image of the second image comprises:

[0253] determining fifth position information of the target object in the preceding image of the second image and corresponding speed information of the target object in the preceding image, wherein the speed information represents a speed of the target object moving from a shooting scene of the preceding image to a shooting scene of the second image;

[0254] determining the second position information of the target object in the second image based on the fifth position information and the speed information.

[0255] In a possible implementation, the field-of-view position relationship represents a conversion relationship between an image coordinate system of the wide-angle lens and an image coordinate system of the long-focus lens; and

[0256] The determining of whether the target object is located in the second field-of-view range based on the first position information and the field-of-view position relationship comprises:

[0257] determining an image coordinate representing the first position information, and determining an image coordinate of the target object after coordinate conversion according to the conversion relationship and the image coordinate of the first position information;

[0258] determining whether the target object is located in the second field-of-view range based on the image coordinate after coordinate conversion.

[0259] In a possible implementation, a first field-of-view center of the first field-of-view range and a second field-of-view center of the second field-of-view range represent the same position.

[0260] The target tracking device provided in this embodiment can be a target tracking device as shown in Figure 4 may perform all steps of each target tracking method described above, and further realize the technical effects of each target tracking method described above. For brevity, the related description is not repeated here.

[0261] Figure 5 FIG. 1 is a structural schematic diagram of a target tracking system provided in this embodiment.

[0262] As shown in Figure 5As shown, the target tracking system 10 comprises a processing unit 11 (for example, a main chip), a wide-angle lens 12 and a long-focus lens 13, the processing unit 11 is connected with the wide-angle lens 12 and the long-focus lens 13 respectively; wherein:

[0263] The wide-angle lens 12 is configured to: capture a first image of a target object; wherein a first field of view range of the wide-angle lens 12 and a second field of view range of the long-focus lens 13 have an intersection;

[0264] The processing unit 11 is configured to: acquire the first image captured by the wide-angle lens 12; determine first position information of the target object in the first image; determine whether the target object is located in the second field of view range based on the first position information and a field of view position relationship, to obtain a determination result, wherein the field of view position relationship represents a position relationship between the first field of view range and the second field of view range; in a case where the determination result indicates that the target object is located in the second field of view range, acquire a second image of the target object captured by the long-focus lens 13; and perform target tracking on the target object based on the second image.

[0265] In some optional implementation manners of the embodiment, the system further comprises a control unit (for example, a pan-tilt head); the control unit is connected with the processing unit 11, the wide-angle lens 12 and the long-focus lens 13 respectively; and

[0266] The control unit is configured to:

[0267] In a case where the determination result indicates that the target object is not located in the second field of view range, control the long-focus lens 13 to move so that the target object is located in the second field of view range based on the first position information and the field of view position relationship;

[0268] In a case where the target object is located in the second field of view range, control the long-focus lens 13 to capture a second image of the target object.

[0269] In some optional implementation manners of the embodiment, the target tracking system 10 further comprises a control unit (for example, a pan-tilt head); the control unit is connected with the processing unit 11, the wide-angle lens 12 and the long-focus lens 13 respectively; a first field of view center of the first field of view range and a second field of view center of the second field of view range represent a same position;

[0270] The processing unit 11 is further configured to: determine second position information of the target object in the second image; and send the second position information to the control unit;

[0271] The control unit is configured to: based on the second position information, control the first field of view center of the wide-angle lens 12 and the second field of view center of the telephoto lens 13 to be displayed at the center position of the subsequent image of the second image; the first field of view center is the center of the first field of view range; the second field of view center is the center of the second field of view range; and the subsequent image represents the next frame image of the second image.

[0272] It can be understood that, in the above optional implementation, by controlling the first field of view center of the wide-angle lens 12 and the second field of view center of the telephoto lens 13 to be displayed at the center position of the subsequent image of the second image, the target object photographed by the wide-angle lens and the telephoto lens is located at the center of the image, so that the accuracy of target tracking can be further improved.

[0273] Optionally, the target tracking of the target object based on the second image comprises:

[0274] determining second position information of the target object in the second image based on a previous image of the second image, wherein the previous image is an image photographed before the second image, and the second position information represents a predicted position of the target object in the second image;

[0275] detecting third position information of the target object in the second image, wherein the third position information represents a measured position of the target object in the second image;

[0276] tracking the target object based on the second position information and the third position information.

[0277] Optionally, the second position information represents a position of the target object in a first detection frame in the previous image, and the third position information represents a position of the target object in a second detection frame in the second image; and

[0278] The target tracking of the target object based on the second position information and the third position information comprises:

[0279] determining a target parameter of a first image region in the first detection frame and a second image region in the second detection frame based on the second position information and the third position information, wherein the target parameter comprises an intersection over union or a similarity;

[0280] in a case where the target parameter is greater than or equal to a preset threshold, determining that the target object in the previous image and the target object in the second image represent the same object, and controlling the wide-angle lens and / or the telephoto lens to track and photograph the target object in the second image.

[0281] Optionally, in the case that the target parameter is less than the preset threshold, the method further comprises:

[0282] determining that the target object in the previous image and the target object representation in the second image represent different objects;

[0283] determining the corresponding speed information of the target object in the previous image;

[0284] determining the corresponding speed information of the target object in the previous image as the corresponding speed information of the target object in the second image;

[0285] determining the third position information and the speed information as the corresponding state information of the target object in the second image.

[0286] Optionally, the determining, based on the previous image of the second image, of the second position information of the target object in the second image comprises:

[0287] determining fifth position information of the target object in the previous image of the second image and corresponding speed information of the target object in the previous image, wherein the speed information represents the speed of the target object moving from a shooting scene of the previous image to a shooting scene of the second image;

[0288] determining the second position information of the target object in the second image based on the fifth position information and the speed information.

[0289] Optionally, the field of view position relationship represents a conversion relationship between an image coordinate system of the wide-angle lens and an image coordinate system of the telephoto lens; and

[0290] the determining, based on the first position information and the field of view position relationship, of whether the target object is located in the second field of view range comprises:

[0291] determining an image coordinate representing the first position information, and determining an image coordinate of the target object after coordinate conversion according to the conversion relationship and the image coordinate of the first position information;

[0292] determining whether the target object is located in the second field of view range based on the image coordinate after coordinate conversion.

[0293] Optionally, a first field of view center of the first field of view range and a second field of view center of the second field of view range represent the same position.

[0294] The target tracking system provided by the embodiments of the present application comprises a processing unit, a wide-angle lens and a long-focus lens, the processing unit is connected with the wide-angle lens and the long-focus lens respectively; wherein: the wide-angle lens is configured to: capture a first image of a target object; wherein a first field of view range of the wide-angle lens and a second field of view range of the long-focus lens have an intersection; the processing unit is configured to: acquire the first image captured by the wide-angle lens; determine first position information of the target object in the first image; determine whether the target object is located in the second field of view range based on the first position information and a field of view position relationship, to obtain a determination result, wherein the field of view position relationship represents a position relationship between the first field of view range and the second field of view range; in a case where the determination result indicates that the target object is located in the second field of view range, acquire a second image of the target object captured by the long-focus lens; and perform target tracking on the target object based on the second image. Thus, a more suitable lens can be selected from the long-focus lens and the wide-angle lens for image capturing based on whether the target object is located in the second field of view range, and then the target tracking is performed by using the captured image, so that the accuracy of the target tracking can be improved.

[0295] Figure 6 A structural schematic diagram of an electronic device provided by the embodiments of the present application is shown in FIG. 1. Figure 6 The electronic device 500 shown in the figure comprises at least one processor 501, a memory 502, at least one network interface 504 and other user interfaces 503. The various components in the electronic device 500 are coupled together through a bus system 505. It can be understood that the bus system 505 is used to realize the connection communication between the components. In addition to the data bus, the bus system 505 also includes a power bus, a control bus and a status signal bus. However, for the purpose of clear illustration, all kinds of buses are marked as the bus system 505 in the figure. Figure 6

[0296] The user interface 503 can include a display, a keyboard or a clicking device (for example, a mouse, a trackball, a touchpad or a touch screen, etc.).

[0297] ​It is to be understood that the memory 502 in embodiments of the present application can be a volatile memory or a nonvolatile memory, or can include both volatile and nonvolatile memory. Among them, the nonvolatile memory can be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a Random Access Memory (RAM) used as an external cache. By way of example, and not limitation, many forms of RAM can be used, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synch link DRAM (SLDRAM), and Direct Rambus RAM (DRRAM). The memory 502 described herein is intended to include, without being limited to, these and any other suitable types of memory.

[0298] In some embodiments, the memory 502 stores the following elements, executable units or data structures, or a subset of them, or an extended set of them: an operating system 5021 and an application program 5022.

[0299] Among them, the operating system 5021 contains various system programs, such as framework layer, core library layer, driver layer, etc., for implementing various basic services and processing hardware-based tasks. The application program 5022 contains various application programs, such as Media Player, Browser, etc., for implementing various application services. The program for implementing the method of the embodiments of the present application can be contained in the application program 5022.

[0300] In the present embodiment, by calling the program or instruction stored in the memory 502, specifically, the program or instruction stored in the application program 5022, the processor 501 is used to execute the method steps provided by each method embodiment, for example, including:

[0301] acquire a first image of a target object captured by a wide-angle lens, wherein a first field of view range of the wide-angle lens and a second field of view range of a long-focus lens have an intersection;

[0302] determine first position information of the target object in the first image;

[0303] determine whether the target object is located in the second field of view range based on the first position information and a field of view position relationship, to obtain a determination result, wherein the field of view position relationship represents a position relationship between the first field of view range and the second field of view range;

[0304] acquire a second image of the target object captured by the long-focus lens in a case where the determination result indicates that the target object is located in the second field of view range;

[0305] perform target tracking on the target object based on the second image. The method disclosed in the embodiments of the present application can be applied to the processor 501 or implemented by the processor 501. The processor 501 can be an integrated circuit chip having a signal processing capability. In the implementation process, each step of the above method can be completed by integrated logic circuits or instruction forms of software in the processor 501. The processor 501 mentioned above can be a general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Each method, step and logic block diagram disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software units in the decoding processor for execution. The software unit can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register or other mature storage media in the art. The storage medium is located in the memory 502, and the processor 501 reads the information in the memory 502 and combines the hardware to complete the steps of the above method.

[0306] It can be understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing units can be implemented within one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSP Devices, DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general purpose processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described above, or a combination thereof.

[0307] For software implementation, the techniques described herein can be implemented with a processing unit executing program code embodied in software. The software is stored in a memory and executed by the processing unit. The memory can be implemented within the processing unit or external to the processing unit.

[0308] The electronic device provided by the embodiments can be an electronic device as shown in Figure 6 The electronic device provided by the embodiments can be an electronic device as shown in

[0309] The embodiments of the present application further provide a storage medium (computer readable storage medium). The storage medium stores one or more programs. The storage medium can include a volatile memory, such as a random access memory, and / or can include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk drive, or a solid-state drive. The storage medium can also include a combination of the above-mentioned memories.

[0310] The one or more programs stored in the storage medium can be executed by the one or more processors to implement the target tracking method described above.

[0311] The processor is configured to execute the target tracking program stored in the memory to implement the following steps of the target tracking method executed at the electronic device side:

[0312] Obtaining a first image of a target object captured by a wide-angle lens; wherein a first field of view range of the wide-angle lens and a second field of view range of a long-focus lens have an intersection;

[0313] Determining first position information of the target object in the first image;

[0314] determine whether the target object is located in the second field of view range based on the first position information and a field of view position relationship, to obtain a determination result, wherein the field of view position relationship represents a position relationship between the first field of view range and the second field of view range;

[0315] in a case where the determination result indicates that the target object is located in the second field of view range, acquire a second image of the target object captured by the long-focus lens;

[0316] perform target tracking on the target object based on the second image. It should also be further appreciated by the skilled in the art that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of the examples have been described in general terms above. Whether the functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. The skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0317] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, software executed by a processor, or a combination of both. The software modules can be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0318] It should be understood that the terms used herein are for the purpose of describing specific example embodiments only and are not intended to be limiting. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. The terms "comprises", "comprising", "includes", "including" and "has", "having" are inclusive and therefore specify the presence of stated features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof. The method steps, processes, and operations described herein are not to be interpreted as necessarily requiring their performance in the specific order indicated, unless explicitly stated otherwise. It should also be understood that additional or alternative steps can be employed.

[0319] The foregoing detailed description of the application has been presented for purposes of illustration and description. Various modifications and changes can be made to these embodiments without departing from the spirit and scope of the application. It is intended that the scope of the application should not be limited by the particular representative embodiments described above.

Claims

1. A target tracking method characterized by, The method comprises: obtaining a first image of a target object captured by a wide-angle lens; wherein a first field of view range of the wide-angle lens and a second field of view range of a long-focus lens have an intersection; determining first position information of the target object in the first image; based on the first position information and a field of view position relationship, determining whether the target object is located in the second field of view range, to obtain a determination result, wherein the field of view position relationship represents a position relationship between the first field of view range and the second field of view range; in a case where the determination result indicates that the target object is located in the second field of view range, obtaining a second image of the target object captured by the long-focus lens; based on the second image, performing target tracking on the target object.

2. The method of claim 1, wherein, After determining whether the target object is located in the second field of view range, the method further comprises: in a case where the determination result indicates that the target object is not located in the second field of view range, based on the first position information, controlling the long-focus lens to move so that the target object is located in the second field of view range; in a case where the target object is located in the second field of view range, controlling the long-focus lens to capture a second image of the target object.

3. The method of claim 1, wherein, The target tracking on the target object based on the second image comprises: based on a previous image of the second image, determining second position information of the target object in the second image, wherein the previous image is an image captured before the second image, and the second position information represents a predicted position of the target object in the second image; detecting third position information of the target object in the second image, wherein the third position information represents a measured position of the target object in the second image; based on the second position information and the third position information, performing target tracking on the target object.

4. The method of claim 3, wherein, The second position information represents a position of the target object in a first detection box in the previous image, and the third position information represents a position of the target object in a second detection box in the second image. And The target tracking on the target object based on the second position information and the third position information comprises: based on the second position information and the third position information, determining a target parameter of a first image region in the first detection box and a second image region in the second detection box, wherein the target parameter comprises an intersection over union or a similarity; in a case where the target parameter is greater than or equal to a preset threshold, determining that the target object in the previous image and the target object in the second image represent the same object, and controlling the wide-angle lens and / or the long-focus lens to track and capture the target object in the second image.

5. The method of claim 4, wherein, in a case where the target parameter is less than the preset threshold, the method further comprises: determining that the target object in the previous image and the target object in the second image represent different objects; determining corresponding speed information of the target object in the previous image; determining the corresponding speed information of the target object in the previous image as corresponding speed information of the target object in the second image; The third position information and the speed information are determined as corresponding state information of the target object in the second image.

6. The method of claim 3, wherein, The second position information of the target object in the second image is determined based on a preceding image of the second image, including: The fifth position information of the target object in the preceding image of the second image is determined, and corresponding speed information of the target object in the preceding image is determined; The second position information of the target object in the second image is determined based on the fifth position information and the speed information.

7. The method of claim 1, wherein, The field of view position relationship represents a conversion relationship between an image coordinate system of the wide-angle lens and an image coordinate system of the long-focus lens. And The target object is determined to be located in the second field of view range based on the first position information and the field of view position relationship, including: An image coordinate representing the first position information is determined, and an image coordinate of the target object after coordinate conversion is determined according to the conversion relationship and the image coordinate of the first position information; The target object is determined to be located in the second field of view range based on the image coordinate after coordinate conversion.

8. The method according to any one of claims 1 to 7, characterized in that, The first field of view center of the first field of view range and the second field of view center of the second field of view range represent the same position.

9. A target tracking system, characterized by The system includes a processing unit, a wide-angle lens, and a long-focus lens, and the processing unit is connected to the wide-angle lens and the long-focus lens; wherein: The wide-angle lens is used to: capture a first image of a target object; wherein the first field of view range of the wide-angle lens and the second field of view range of the long-focus lens have an intersection; The processing unit is used to: acquire the first image captured by the wide-angle lens; determine first position information of the target object in the first image; determine whether the target object is located in the second field of view range based on the first position information and a field of view position relationship, to obtain a determination result, wherein the field of view position relationship represents a position relationship between the first field of view range and the second field of view range; in a case where the determination result indicates that the target object is located in the second field of view range, acquire a second image of the target object captured by the long-focus lens; and perform target tracking on the target object based on the second image.

10. The system of claim 9, wherein, The system further includes a control unit; the control unit is connected to the processing unit, the wide-angle lens, and the long-focus lens; and The control unit is used to: In a case where the determination result indicates that the target object is not located in the second field of view range, control the long-focus lens to move so that the target object is located in the second field of view range based on the first position information; In a case where the target object is located in the second field of view range, control the long-focus lens to capture a second image of the target object.

Images