Target tracking method, device, road monitoring equipment, equipment and medium

Abstract

The present invention discloses a target tracking method, device, road monitoring equipment, equipment and medium. Since the method is applied to a controller of a road monitoring equipment, the road monitoring equipment includes a first lens and a second lens, wherein the first lens is a telephoto lens and the second lens is a wide-angle lens, a first image captured by the first lens is obtained, if the first image contains a target, the regional position of the target in the first image is determined, and the pan / tilt of the first lens is controlled to rotate to track the target, so that the regional position is always located at a set position of the first lens field of view; thereby, images of blind spots can be captured and characteristic information of targets in the blind spots can be accurately identified; until at any moment the target is located in the target area in the second image captured by the second lens, the first lens is stopped from tracking the target and the first lens is controlled to return to the initial position, thereby ensuring that the road monitoring equipment can obtain a large field of view angle and high-definition images of distant targets.

Description

Technical Field

[0001] The present invention relates to the technical field of camera equipment, and in particular to a target tracking method, device, road monitoring equipment, equipment and medium. Background Art

[0002] With the annual increase in the number of domestic vehicles, the pressure and complexity of urban traffic are also increasing. Intelligent traffic supervision systems and equipment can reduce the workload of traffic police, and can also accurately capture traffic violations, urge traffic participants to abide by traffic rules, and reduce the occurrence of illegal and irregular behaviors.

[0003] In order to monitor traffic violations on the road, the prior art usually uses a road monitoring device with only one lens to capture photos at the checkpoint, and the lens of the road monitoring device is usually a wide-angle lens or a telephoto lens. The wide-angle lens can be used to obtain images of targets containing traffic violations, but since the wide-angle lens has a larger field of view and a wider monitoring range, the distant target occupies a smaller proportion in the image, so it is impossible to obtain high-definition images of distant targets through the wide-angle lens. The telephoto lens can be used to obtain high-definition images of distant targets, but the field of view is small and the monitoring range is narrow. Therefore, a road monitoring device that can obtain high-definition images of distant targets and has a larger field of view is needed.

[0004] The prior art includes a dual-lens road monitoring device, which includes a wide-angle lens, a telephoto lens, a ball camera carrying component and a micro pan-tilt head. The wide-angle lens is fixedly connected to the ball camera carrying component of the road monitoring device, and the telephoto lens is connected to the ball camera carrying component through the micro pan-tilt head. The ball camera carrying component and the micro pan-tilt head can both rotate in the horizontal and vertical directions, so that the wide-angle lens and the telephoto lens can both rotate in the horizontal and vertical directions, so that the dual-lens road monitoring device can obtain a larger field of view while ensuring that high-definition images of distant targets can be obtained, and the wide-angle lens and the telephoto lens are independent of each other and do not interfere with each other.

[0005] However, since the wide-angle lens and telephoto lens work independently of each other, the acquired images are stored independently and not fused, which is equivalent to using two independent road monitoring devices. In addition, since the determination of lane lines will affect the determination of traffic violations when monitoring traffic violations, and the change of the field of view corresponding to the image will affect the determination of lane lines, the field of view needs to be fixed. Since the wide-angle lens and telephoto lens in the above dual-lens road monitoring device rotate in the horizontal and vertical directions, the field of view corresponding to the image will change, so the existing dual-lens road monitoring device cannot monitor road traffic violations.

[0006] In the common design of the prior art, the road monitoring equipment may also include a wide-angle lens and a telephoto lens, and both the wide-angle lens and the telephoto lens are fixed in the housing of the road monitoring equipment. The wide-angle lens has a larger field of view, and a high-definition image of a distant target can be obtained through the telephoto lens. The image corresponding to the telephoto lens is seamlessly integrated into the image corresponding to the wide-angle lens through algorithm processing. This solution can ensure that a high-definition image of a distant target can be obtained while obtaining a larger field of view, and overcomes the disadvantage that the wide-angle lens and the telephoto lens in the above dual-lens road monitoring equipment are not truly integrated. Moreover, since the field of view of the wide-angle lens corresponding to the integrated image will not change, it will not affect the judgment of traffic violations.

[0007] However, there is a blind spot between the best image acquisition area of ​​the wide-angle lens and the best image acquisition area of ​​the telephoto lens. The image acquired in the best image acquisition area is clear, and although the image containing the target can also be acquired by the wide-angle lens and the telephoto lens in the blind spot, the image containing the target is not clear, which may cause misjudgment of traffic violations or fail to accurately identify the characteristic information of the target; and because the field of view of the telephoto lens is very small and the field of view of the wide-angle lens is larger, there is a blind spot between the field of view of the telephoto lens and the field of view of the wide-angle lens, resulting in the inability to accurately identify the characteristic information of the target in the blind spot.

[0008] Therefore, how to accurately identify the characteristic information of targets in blind spots based on the road monitoring equipment's ability to obtain high-definition images of targets with a large field of view and at a distance becomes an urgent problem to be solved. Summary of the invention

[0009] The present invention provides a target tracking method, device, road monitoring equipment, equipment and medium to solve the problem of how to accurately identify the characteristic information of the target in the blind spot on the basis that the road monitoring equipment can obtain a large field of view and high-definition images of distant targets.

[0010] The present invention provides a target tracking method, the method comprising:

[0011] Acquire a first image captured by a first lens;

[0012] If the first image contains a target, determining a region position of the target in the first image, and controlling the pan / tilt of the first lens to track the target, so that the region position is always located at a set position of the field of view of the first lens;

[0013] If the target is located in the target area in the second image captured by the second lens at any time, the first lens is stopped from tracking the target and the first lens is controlled to return to an initial position.

[0014] Furthermore, the method further comprises:

[0015] Acquire a third image captured by the second lens at the same time as the first image;

[0016] According to the target in the first image, cropping is performed to obtain a first target area image of the first image;

[0017] The first target area image of the first image and the third image are fused to obtain a fused target image output.

[0018] Furthermore, the step of fusing the first target area image of the first image and the third image to obtain a fused target image output includes:

[0019] According to the second target area image of the target in the third image, scaling the first target area image to the same size as the second target area image;

[0020] The second target area image is replaced with the scaled first target area image, and the replaced third image is determined as the fused target image and outputted.

[0021] Furthermore, if the first image contains a target, determining the regional position of the target in the first image includes:

[0022] If the first image contains only one target, determining the center of the target in the first image as the regional position of the target;

[0023] If the first image includes multiple targets, a third target region including each target in the first image is determined, and a region center of the third target region is determined as the region position.

[0024] Accordingly, the present invention provides a road monitoring device based on any one of the above target tracking methods, the road monitoring device comprising: a first lens, a second lens, a pan / tilt of the first lens, and a controller, wherein the first lens is a telephoto lens, the second lens is a wide-angle lens, and the pan / tilt comprises a horizontal motor and a vertical motor;

[0025] The controller is used to execute any one of the above target tracking methods;

[0026] The first lens is used to capture the first image and send it to the controller;

[0027] The second lens is used to capture a second image and a third image at the same time as the first image and send them to the controller;

[0028] The pan / tilt head is used to receive control instructions from the controller to rotate.

[0029] Accordingly, the present invention provides a target tracking device, the device comprising:

[0030] An acquisition module, used to acquire a first image captured by a first lens;

[0031] A tracking module is used to determine the regional position of the target in the first image if the first image contains the target, control the pan / tilt rotation of the first lens to track the target, so that the regional position is always located at a set position in the field of view of the first lens; if the target is located in the target area in the second image captured by the second lens at any moment, stop the first lens from tracking the target and control the first lens to return to an initial position.

[0032] Furthermore, the acquisition module is further used to acquire a third image captured by the second lens at the same time as the first image;

[0033] The device also includes:

[0034] A cropping module, used for cropping the first image according to the target in the first image to obtain a first target area image of the first image;

[0035] The fusion module is used to fuse the first target area image of the first image and the third image to obtain a fused target image output.

[0036] Furthermore, the fusion module is specifically used to scale the first target area image to the same size as the second target area image of the target in the third image according to the second target area image of the target; replace the second target area image with the scaled first target area image, determine the replaced third image as the fused target image and output it.

[0037] Furthermore, the tracking module is specifically used to determine the center of the target in the first image as the regional position of the target if the first image contains only one target; if the first image contains multiple targets, determine a third target area in the first image containing each target, and determine the regional center of the third target area as the regional position.

[0038] Accordingly, an embodiment of the present invention provides a road monitoring device based on any one of the above target tracking methods, the road monitoring device comprising: a first lens, a second lens, a pan / tilt of the first lens, and a controller, wherein the first lens is a telephoto lens, the second lens is a wide-angle lens, and the pan / tilt comprises a horizontal motor and a vertical motor;

[0039] The controller is used to execute any one of the above target tracking methods;

[0040] The first lens is used to capture the first image and send it to the controller;

[0041] The second lens is used to capture a second image and a third image at the same time as the first image and send them to the controller;

[0042] The pan / tilt head is used to receive control instructions from the controller to rotate.

[0043] Accordingly, the present invention provides an electronic device, comprising: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0044] The memory stores a computer program, and when the program is executed by the processor, the processor implements the steps of any one of the above-mentioned target tracking methods.

[0045] Accordingly, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the methods described in the above-mentioned ground segmentation network model training method.

[0046] The present invention provides a target tracking method, device, road monitoring equipment, equipment and medium. Since the method is applied to a controller of a road monitoring equipment, the road monitoring equipment includes a first lens and a second lens, wherein the first lens is a telephoto lens and the second lens is a wide-angle lens. A first image captured by the first lens is obtained. If the first image contains a target, the regional position of the target in the first image is determined, and the pan / tilt of the first lens is controlled to rotate to track the target, so that the regional position is always located at a set position of the first lens field of view; thereby, through the rotation of the pan / tilt, the telephoto lens can capture the image of the blind area between the image capture areas of the fixed telephoto lens and the wide-angle lens in the prior art, and the blind area outside the field of view of the telephoto lens, so as to accurately identify the characteristic information of the target in the blind area; until at any moment, when the target is located in the target area in the second image captured by the second lens, the first lens is stopped from tracking the target and the first lens is controlled to return to the initial position, thereby ensuring that the road monitoring equipment can obtain a large field of view and a high-definition image of a distant target. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0048] Figure 1 A process diagram of a target tracking method provided by an embodiment of the present invention;

[0049] Figure 2 A schematic diagram of an image acquisition area at an initial position of a first lens and an image acquisition area of ​​a second lens provided by an embodiment of the present invention;

[0050] Figure 3 A schematic diagram of an image acquisition area after a pan / tilt of a first lens is vertically rotated according to an embodiment of the present invention;

[0051] Figure 4 A schematic diagram of an image acquisition area after a pan / tilt of a first lens is horizontally rotated according to an embodiment of the present invention;

[0052] Figure 5 A process diagram of a target tracking method provided by an embodiment of the present invention;

[0053] Figure 6 A schematic diagram of the structure of a road monitoring device provided by an embodiment of the present invention;

[0054] Figure 7 A schematic diagram of the structure of a binocular camera provided by an embodiment of the present invention;

[0055] Figure 8 A schematic diagram of the internal structure of a binocular camera provided by an embodiment of the present invention;

[0056] Fig. 9 A schematic diagram of the external structure of a binocular camera provided by an embodiment of the present invention;

[0057] Fig.10 A schematic diagram of the structure of a target tracking device provided by an embodiment of the present invention;

[0058] Fig.11 A schematic diagram of the structure of a target tracking device provided by an embodiment of the present invention;

[0059] Fig.12 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] In order to accurately identify the characteristic information of the target in the blind spot, an embodiment of the present invention provides a process schematic diagram of a target tracking method.

[0062] Embodiment 1:

[0063] Figure 1 A process diagram of a target tracking method provided by an embodiment of the present invention, the process includes the following steps:

[0064] S101: Acquire a first image captured by a first lens.

[0065] A target tracking method provided by an embodiment of the present invention is applied to a controller of a road monitoring device, wherein the road monitoring device is a dual-lens road monitoring device, including a first lens and a second lens, the first lens is a telephoto lens, and the second lens is a wide-angle lens, that is, the focal length of the first lens is longer than that of the second lens, and the field of view of the second lens is larger than that of the first lens.

[0066] Figure 2 A schematic diagram of an image acquisition area at an initial position of a first lens and an image acquisition area of ​​a second lens provided in an embodiment of the present invention, such as Figure 2 As shown in the figure, the area within the range of ray OA and ray OB is the image acquisition area of ​​the second lens, the trapezoid EFGH is the best image acquisition area of ​​the second lens, and the angle AOB is the field of view of the second lens; the area within the range of ray OC and ray OD is the image acquisition area of ​​the first lens, the trapezoid KLMN is the best image acquisition area of ​​the first lens, and the angle COD is the field of view of the first lens. There is a blind spot between the best image acquisition area EFGH of the second lens and the best image acquisition area KLMN of the first lens, and there is also a blind spot between the field of view angle AOB of the second lens and the field of view angle COD of the first lens.

[0067] In order to accurately identify the characteristic information of the target in the blind spot, if the target is a vehicle, the characteristic information of the vehicle is the license plate of the vehicle, and if the target is a pedestrian, the characteristic information of the pedestrian is the facial information of the pedestrian. An embodiment of the present invention provides a target tracking method, that is, accurately identifying the characteristic information of the target in the blind spot by tracking the target entering the blind spot.

[0068] In order to track the target, the controller first obtains the first image captured by the first lens. Since the focal length of the first lens is longer than that of the second lens, the first image captured by the first lens is a distant image in the image acquisition area.

[0069] S102: If the first image contains a target, determine a region position of the target in the first image, and control the pan / tilt of the first lens to rotate to track the target, so that the region position is always located at a set position of the field of view of the first lens.

[0070] After acquiring the first image, the controller recognizes the first image to determine whether the first image contains a target. Specifically, the recognition can be performed using an existing model recognition method or an existing frame difference method.

[0071] If the first image contains a target, in order to track the target, the regional position of the target in the first image is first determined, wherein the region of the target can be a circular region or a rectangular region, and the regional position can be the position of the center of the region or the position of the corner point of the region. Specifically, the embodiment of the present invention does not impose any limitation on this.

[0072] After determining the area position of the target in the first image, the pan / tilt of the first lens is controlled to rotate to track the target, wherein the pan / tilt of the first lens may rotate in a horizontal direction or in a vertical direction.

[0073] Figure 3 A schematic diagram of an image acquisition area after a pan / tilt of a first lens is vertically rotated according to an embodiment of the present invention, such as Figure 3 As shown, there is an overlap between the best image acquisition area KLMN of the first lens and the best image acquisition area EFGH of the second lens. Figure 2 The blind area between the best image acquisition area KLMN of the first lens and the best image acquisition area EFGH of the second lens, Figure 3 Will Figure 2 Blind area removal between optimal image acquisition areas in .

[0074] Figure 4 A schematic diagram of an image acquisition area after a pan / tilt of a first lens is horizontally rotated according to an embodiment of the present invention, as shown in FIG4 , the overlap range of the field of view COD of the first lens and the field of view AOB of the second lens is located at the left ( Figure 4 The left and right side areas, Figure 2 The overlap range of the field of view COD of the first lens and the field of view OAB of the second lens is located in the middle ( Figure 4 left, middle and right) areas, compared to Figure 2The blind area of ​​the first lens's field of view COD and the second lens's field of view AOB, Figure 4 Will Figure 2 The left ( Figure 2 The left and right blind areas are removed.

[0075] By controlling the pan / tilt of the first lens to rotate, the image acquisition area of ​​the first lens will also change, and the corresponding area position of the target area in the first image will also change, so that the area position is always located at a set position of the field of view of the first lens, wherein the set position of the first image can be any point position in the first image, preferably, the set position of the first image is the middle area position of the first image.

[0076] For example, the pan / tilt of the first lens is controlled to rotate to track the target, so that the target area is always located in the middle area of ​​the first image. Figure 2 The best image acquisition area in KLMN is the lower ( Figure 2 The position of the upper and lower areas in the image is controlled to control the gimbal of the first lens to rotate vertically downward, and the image is obtained. Figure 3 The optimal image acquisition area KLMN of the first lens in the original Figure 2 The position of the lower area in the optimal image acquisition area KLMN becomes Figure 3 The middle area position in the optimal image acquisition area KLMN is determined so that the area position of the target is located in the middle area position of the first image.

[0077] If the area position of the target area in the first image corresponds to Figure 2 The best image acquisition area in KLMN is the left ( Figure 2 The left and right side areas of the image are located in the center of the image, and the gimbal of the first lens is controlled to rotate horizontally to the left. Figure 4 The optimal image acquisition area KLMN in the original Figure 2 The position of the left area in the optimal image acquisition area KLMN becomes Figure 4 The middle area position in the optimal image acquisition area KLMN is determined so that the area position of the target is located in the middle area position of the first image.

[0078] S103: If at any moment the target is located in the target area in the second image captured by the second lens, stop the first lens from tracking the target and control the first lens to return to an initial position.

[0079] In order to determine whether the tracking of the target is completed, in an embodiment of the present invention, the image acquisition area of ​​the second lens includes an optimal image acquisition area, and the target area corresponding to the optimal image acquisition area in the second image captured by the second lens is determined. If it is determined that the target is located in the target area at any moment, it is determined that the tracking of the target is completed.

[0080] Specifically, the controller continues to acquire the first image captured by the first lens at a subsequent moment. Since the image capture area of ​​the second lens includes the image capture area of ​​the first lens, the first image captured by the first lens can determine the corresponding area in the second image captured by the second lens, and therefore can also determine the target position corresponding to the set position of the first image in the second image. Since the target in the first image is always located at the set position, the target position is the corresponding position of the target in the second image.

[0081] According to the acquisition that the target in the first image captured by the first lens is located at the corresponding target position in the second image and the target area in the second image at the subsequent moment, if the target in the first image captured by the first lens at any moment is located at the corresponding target position in the second image and is located in the target area, it is determined that the tracking of the target is completed, so the tracking of the target is stopped and the first lens is controlled to return to the initial position.

[0082] Since the method is applied to a controller of a road monitoring device in an embodiment of the present invention, the road monitoring device includes a first lens and a second lens, wherein the first lens is a telephoto lens and the second lens is a wide-angle lens, a first image captured by the first lens is obtained, and if the first image contains a target, the regional position of the target in the first image is determined, and the pan / tilt of the first lens is controlled to rotate to track the target, so that the regional position is always located at a set position of the field of view of the first lens; thereby, through the rotation of the pan / tilt, the telephoto lens can capture images of the blind area between the image capture areas of the fixed telephoto lens and the wide-angle lens in the prior art, as well as the blind area outside the field of view of the telephoto lens, so as to accurately identify the characteristic information of the target in the blind area; until at any moment when the target is located in the target area in the second image captured by the second lens, the first lens is stopped from tracking the target and the first lens is controlled to return to the initial position, thereby ensuring that the road monitoring device can obtain a large field of view and high-definition images of distant targets.

[0083] Embodiment 2:

[0084] In order to obtain the fused target image, based on the above embodiment, in an embodiment of the present invention, the method further includes:

[0085] Acquire a third image captured by the second lens at the same time as the first image;

[0086] According to the target in the first image, cropping is performed to obtain a first target area image of the first image;

[0087] The first target area image of the first image and the third image are fused to obtain a fused target image output.

[0088] The controller of the road monitoring device also obtains a third image captured by the second lens at the same time as the first image.

[0089] Since the image acquisition area of ​​the second lens includes the image acquisition area of ​​the first lens, and the best image acquisition area of ​​the first lens is located farther away in the image acquisition area of ​​the second lens, for the best image acquisition area of ​​the first lens, the regional image in the first image captured by the first lens is clearer than the third image captured by the second lens. In order to facilitate the monitoring of traffic violations, the first image and the third image are fused to obtain a fused target image.

[0090] Preferably, in order to reduce the amount of computation for image fusion, the first image is cropped according to the target area in the first image to obtain a first target area image in the first image, and the first target area image is fused with the third image to obtain a fused target image and output it. Specifically, an existing seamless fusion algorithm can be used to perform fusion to obtain the fused target image.

[0091] As a possible implementation, after determining the fused target image, the target's feature information can also be extracted based on the existing feature extraction model. If the target is a vehicle, the vehicle's license plate, model, and body color and other feature information are extracted. If the target is a pedestrian, the pedestrian's face, hairstyle, clothing and other feature information are extracted, and the image corresponding to the traffic violation is determined and stored. The method of extracting the target's feature information belongs to the prior art, and the embodiment of the present invention does not limit this.

[0092] In order to obtain a fused target image, in an embodiment of the present invention, the step of fusing the first target area image of the first image and the third image to obtain a fused target image output includes:

[0093] According to the second target area image of the target area in the third image, scaling the first target area image to the same size as the second target area image;

[0094] The second target area image is replaced with the scaled first target area image, and the replaced third image is determined as the fused target image and outputted.

[0095] In the area image corresponding to the best image acquisition area of ​​the first lens in the third image, a second target area image of the target area is determined, and according to the area size of the second target area image, the first target area image is scaled to the same size as the second target area image.

[0096] The scaled first target area image is used to replace the second target area image in the fourth image. The replaced third image is the fused target image, and the target image is output.

[0097] Embodiment 3:

[0098] In order to determine the regional position of the target area in the first image, based on the above embodiments, in an embodiment of the present invention, if the first image contains the target, determining the regional position of the target in the first image includes:

[0099] If the first image contains only one target, determining the center of the target in the first image as the regional position of the target;

[0100] If the first image includes multiple targets, a third target region including each target in the first image is determined, and a region center of the third target region is determined as the region position.

[0101] In order to determine the regional position of the target in the first image, in the embodiment of the present invention, the first image may contain only one target or may contain multiple targets.

[0102] If the first image contains only one target, the center of the target in the first image is determined, and the center of the target is determined as the target region position. As a possible implementation, the smallest rectangular region containing the target may be determined, and the region center of the rectangular region may be determined as the target center.

[0103] If the first image contains multiple targets, a third target area containing each target in the first image is determined, wherein the third target area may be a rectangular area, and the area center of the third target area is determined as the area position of the target.

[0104] Embodiment 4:

[0105] The target tracking method provided by the present invention is described below through a specific embodiment. Figure 5 A process diagram of a target tracking method provided by an embodiment of the present invention is shown in FIG. Figure 5 As shown, the method comprises the following steps:

[0106] S501: Acquire a first image captured by a first lens.

[0107] S502: If the first image contains only one target, the center of the target in the first image is determined as the regional position of the target; if the first image contains multiple targets, a third target area containing each target in the first image is determined, and the regional center of the third target area is determined as the regional position; the pan / tilt rotation of the first lens is controlled to track the target, so that the regional position is always located at the set position of the field of view of the first lens.

[0108] S503: If at any moment the target is located in the target area in the second image captured by the second lens, stop the first lens from tracking the target and control the first lens to return to an initial position.

[0109] S504: Acquire a third image captured by the second lens at the same time as the first image; crop the first image according to the target in the first image to obtain a first target area image; scale the first target area image to the same size as the second target area image according to the second target area image of the target in the third image; replace the second target area image with the scaled first target area image, determine the replaced third image as the fused target image, and output it.

[0110] Embodiment 5:

[0111] Figure 6 A schematic diagram of the structure of a road monitoring device provided by an embodiment of the present invention is shown in FIG. Figure 6 As shown, the road monitoring device includes: a first lens 601, a second lens 602, a pan / tilt 603 of the first lens 601, and a controller 604, wherein the first lens 601 is a telephoto lens, the second lens 602 is a wide-angle lens, and the pan / tilt 603 includes a horizontal motor and a vertical motor;

[0112] The controller 604 is used to execute the target tracking method described in any one of the above embodiments 1-4;

[0113] The first lens 601 is used to capture the first image and the third image and send them to the controller 604;

[0114] The second lens 602 is used to capture a fourth image at the same time as the first image and a second image at the same time as the third image and send them to the controller 604;

[0115] The pan / tilt platform 603 is used to receive control instructions from the controller 604 to rotate.

[0116] Figure 7 A schematic diagram of the structure of a binocular camera provided by an embodiment of the present invention is shown in FIG. Figure 7As shown, the road monitoring device includes a wide-angle camera unit, an intelligent recognition and operation unit, a light sensing unit and a detail tracking unit. The wide-angle camera unit includes a wide-angle lens 701 and a first sensor 702. The intelligent recognition and operation unit includes a first intelligent main control (Systemon Chip, Soc) 703. The detail tracking unit includes a telephoto lens 704, a second sensor 705, a second intelligent main control Soc706, a pan-tilt microcontroller unit (Microcontroller Unit, MCU) 707, a horizontal motor 708 and a vertical motor 709. The light sensing unit includes a fill light 710.

[0117] The first intelligent system on chip (Soc) 703 and the second intelligent system on chip Soc 706 are equivalent to the controller 604 in the embodiment of the present invention, and are used to implement the target tracking method described in any one of the above embodiments 1-4.

[0118] The telephoto lens 704 and the second sensor 705 are equivalent to the first lens 601 in the embodiment of the present invention, and are used to capture the first image and the third image.

[0119] The wide-angle lens 701 and the first sensor 702 are equivalent to the second lens 602 in the embodiment of the present invention, and are used to capture the fourth image at the same time as the first image, and the second image at the same time as the third image.

[0120] The pan-tilt microcontroller unit (MCU) 707, the horizontal motor 708 and the vertical motor 709 are equivalent to the pan-tilt 603 in the embodiment of the present invention.

[0121] Figure 8 A schematic diagram of the internal structure of a binocular camera provided by an embodiment of the present invention is shown in FIG. Figure 8 As shown, the telephoto lens 801 and the wide-angle lens 802 are located inside the camera housing, and the camera housing is in a fixed state after installation and debugging.

[0122] The wide-angle lens 802 is fixedly mounted inside the camera housing, and the field of view of the wide-angle lens 802 is also fixed. The telephoto lens 801 is mounted on a micro-pan head 803, which can rotate in the horizontal and vertical directions, and the telephoto lens 801 can rotate with the micro-pan head 803.

[0123] The rotation angle of the micro-pan-tilt 803 is limited to ensure that the field of view of the telephoto lens 801 is always within the field of view of the wide-angle lens 802. If the wide-angle lens 802 cannot obtain the details of the target, the intelligent main control adjusts the micro-pan-tilt 803 to make the telephoto lens 801 track the target, and cuts out the regional image of the target area from the image captured by the telephoto lens 801 and sends it to the intelligent main control for seamless fusion processing with the image captured by the wide-angle lens 802 to obtain the fused target image.

[0124] Fig. 9 A schematic diagram of the external structure of a binocular camera provided by an embodiment of the present invention, such as Fig. 9 As shown, the telephoto lens 901 is located above (upper and lower in the figure) the wide-angle lens 902 , and fill lights 903 are located on both sides of the wide-angle lens 902 .

[0125] Embodiment 6:

[0126] Fig.10 The present invention provides a schematic diagram of a target tracking device according to an embodiment of the present invention. Based on the above embodiments, the present invention further provides a target tracking device, the device comprising:

[0127] An acquisition module 1001 is used to acquire a first image captured by a first lens;

[0128] The tracking module 1002 is used to determine the regional position of the target in the first image if the target is included in the first image, control the pan / tilt rotation of the first lens to track the target, so that the regional position is always located at the set position of the field of view of the first lens; if the target is located in the target area in the second image captured by the second lens at any time, stop the first lens from tracking the target and control the first lens to return to the initial position.

[0129] Furthermore, the acquisition module 1001 is further used to acquire a third image captured by the second lens at the same time as the first image;

[0130] The device further includes: a cropping module 1003, configured to crop, according to the target in the first image, a first target area image of the first image;

[0131] The fusion module 1004 is used to fuse the first target area image of the first image and the third image to obtain a fused target image output.

[0132] Furthermore, the fusion module 1004 is specifically used to scale the first target area image to the same size as the second target area image of the target in the third image according to the second target area image of the target; replace the second target area image with the scaled first target area image, determine the replaced third image as the fused target image and output it.

[0133] Furthermore, the tracking module 1002 is specifically used to determine, if the first image contains only one target, the center of the target in the first image as the regional position of the target; if the first image contains multiple targets, determine a third target area in the first image containing each target, and determine the regional center of the third target area as the regional position.

[0134] Fig.11 A schematic diagram of a target tracking device provided by an embodiment of the present invention is shown in FIG. Fig.11 As shown, the target tracking device includes: an image acquisition module 1101, an intelligent recognition module 1102, a detail tracking module 1103 and an image fusion module 1104.

[0135] The image acquisition module 1101 is equivalent to the acquisition module 1001 in the embodiment of the present invention, and is used to acquire the first image acquired by the first lens, and the second image and the third image acquired by the second lens.

[0136] The detail tracking module 1103 is equivalent to the tracking module 1002 in the embodiment of the invention, and is used to determine the regional position of the target in the first image, control the pan / tilt rotation of the first lens to track the target, so that the regional position is always located at the set position of the field of view of the first lens; when the target is located in the target area in the second image captured by the second lens at any time, stop the first lens from tracking the target and control the first lens to return to the initial position.

[0137] The image fusion module 1104 is equivalent to the cropping module 1003 and the fusion module 1004 in the embodiment of the present invention, and is used to crop the first target area image of the first image according to the target in the first image, and fuse the first target area image of the first image and the third image to obtain a fused target image output.

[0138] The intelligent recognition module 1102 is used to continue analyzing the target image based on the fused target image, extract the characteristic information of the target, and identify the traffic violation of the target.

[0139] Embodiment 7:

[0140] Fig.12A schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. On the basis of the above embodiments, an electronic device is further provided in an embodiment of the present invention, including a processor 1201, a communication interface 1202, a memory 1203 and a communication bus 1204, wherein the processor 1201, the communication interface 1202, and the memory 1203 communicate with each other through the communication bus 1204;

[0141] The memory 1203 stores a computer program. When the program is executed by the processor 1201, the processor 1201 performs the following steps:

[0142] Acquire a first image captured by a first lens;

[0143] If the first image contains a target, determining a region position of the target in the first image, and controlling the pan / tilt of the first lens to track the target, so that the region position is always located at a set position of the field of view of the first lens;

[0144] If the target is located in the target area in the second image captured by the second lens at any time, the first lens is stopped from tracking the target and the first lens is controlled to return to an initial position.

[0145] Furthermore, the processor 1201 is further configured to obtain a third image captured by the second lens at the same time as the first image;

[0146] According to the target in the first image, cropping is performed to obtain a first target area image of the first image;

[0147] The first target area image of the first image and the third image are fused to obtain a fused target image output.

[0148] Further, the processor 1201 is specifically configured to fuse the first target area image of the first image and the third image to obtain a fused target image output, which includes:

[0149] According to the second target area image of the target in the third image, scaling the first target area image to the same size as the second target area image;

[0150] The second target area image is replaced with the scaled first target area image, and the replaced third image is determined as the fused target image and outputted.

[0151] Further, the processor 1201 is specifically configured to, if the first image contains a target, determine the regional position of the target in the first image, including:

[0152] If the first image contains only one target, determining the center of the target in the first image as the regional position of the target;

[0153] If the first image includes multiple targets, a third target region including each target in the first image is determined, and a region center of the third target region is determined as the region position.

[0154] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0155] The communication interface 1202 is used for communication between the above electronic device and other devices.

[0156] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0157] The above-mentioned processor can be a general-purpose processor, including a central processing unit, a network processor (Network Processor, NP), etc.; it can also be a digital signal processing processor (Digital Signal Processing, DSP), an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc.

[0158] Embodiment 8:

[0159] On the basis of the above embodiments, an embodiment of the present invention further provides a computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to perform the following steps:

[0160] Acquire a first image captured by a first lens;

[0161] If the first image contains a target, determining a region position of the target in the first image, and controlling the pan / tilt of the first lens to track the target, so that the region position is always located at a set position of the field of view of the first lens;

[0162] If the target is located in the target area in the second image captured by the second lens at any time, the first lens is stopped from tracking the target and the first lens is controlled to return to an initial position.

[0163] Furthermore, the method further comprises:

[0164] Acquire a third image captured by the second lens at the same time as the first image;

[0165] According to the target in the first image, cropping is performed to obtain a first target area image of the first image;

[0166] The first target area image of the first image and the third image are fused to obtain a fused target image output.

[0167] Furthermore, the step of fusing the first target area image of the first image and the third image to obtain a fused target image output includes:

[0168] According to the second target area image of the target in the third image, scaling the first target area image to the same size as the second target area image;

[0169] The second target area image is replaced with the scaled first target area image, and the replaced third image is determined as the fused target image and outputted.

[0170] Furthermore, if the first image contains a target, determining the regional position of the target in the first image includes:

[0171] If the first image contains only one target, determining the center of the target in the first image as the regional position of the target;

[0172] If the first image includes multiple targets, a third target region including each target in the first image is determined, and a region center of the third target region is determined as the region position.

[0173] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0174] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0175] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0176] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0177] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A target tracking method, applied to a controller of a road monitoring device, wherein the road monitoring device comprises a first lens and a second lens, wherein the first lens is a telephoto lens and the second lens is a wide-angle lens, characterized in that: The method comprises: Acquire a first image captured by a first lens; If the first image contains a target, determining a region position of the target in the first image, and controlling the pan / tilt of the first lens to rotate to track the target, so that the region position is always located at a set position of the field of view of the first lens; If at any moment the target is located in the target area in the second image captured by the second lens, stop the first lens from tracking the target and control the first lens to return to an initial position; The field of view angle of the wide-angle lens is fixed, and the field of view angle of the telephoto lens is included in the field of view angle range of the wide-angle lens.

2. The method according to claim 1, characterized in that The method further comprises: Acquire a third image captured by the second lens at the same time as the first image; According to the target in the first image, cropping is performed to obtain a first target area image of the first image; The first target area image of the first image and the third image are fused to obtain a fused target image output.

3. The method according to claim 2, characterized in that The step of fusing the first target area image of the first image and the third image to obtain a fused target image output comprises: According to the second target area image of the target in the third image, scaling the first target area image to the same size as the second target area image; The second target area image is replaced with the scaled first target area image, and the replaced third image is determined as the fused target image and outputted.

4. The method according to claim 1, characterized in that If the first image contains a target, determining the area position of the target in the first image includes: If the first image contains only one target, determining the center of the target in the first image as the regional position of the target; If the first image includes multiple targets, a third target region including each target in the first image is determined, and a region center of the third target region is determined as the region position.

5. A road monitoring device based on the target tracking method according to any one of claims 1 to 4, characterized in that: The road monitoring device comprises: a first lens, a second lens, a pan / tilt of the first lens and a controller, wherein the first lens is a telephoto lens, the second lens is a wide-angle lens, and the pan / tilt comprises a horizontal motor and a vertical motor; The controller is used to execute the target tracking method according to any one of claims 1 to 4; The first lens is used to capture the first image and send it to the controller; The second lens is used to capture a second image and a third image at the same time as the first image and send them to the controller; The pan / tilt head is used to receive control instructions from the controller to rotate; The field of view angle of the wide-angle lens is fixed, and the field of view angle of the telephoto lens is included in the field of view angle range of the wide-angle lens.

6. A target tracking device, characterized in that: The device comprises: An acquisition module, configured to acquire a first image captured by a first lens, wherein the first lens is a telephoto lens; A tracking module, configured to determine the regional position of the target in the first image if the first image contains the target, control the pan / tilt of the first lens to track the target, so that the regional position is always located at a set position of the field of view of the first lens; if the target is located in the target area in the second image captured by the second lens at any time, stop the first lens from tracking the target and control the first lens to return to an initial position, wherein the second lens is a wide-angle lens; The field of view angle of the wide-angle lens is fixed, and the field of view angle of the telephoto lens is included in the field of view angle range of the wide-angle lens.

7. The device according to claim 6, characterized in that The acquisition module is further used to acquire a third image captured by the second lens at the same time as the first image; The device also includes: A cropping module, used for cropping the first image according to the target in the first image to obtain a first target area image of the first image; The fusion module is used to fuse the first target area image of the first image and the third image to obtain a fused target image output.

8. The device according to claim 7, characterized in that The fusion module is specifically used to scale the first target area image to the same size as the second target area image according to the second target area image of the target in the third image; replace the second target area image with the scaled first target area image, determine the replaced third image as the fused target image and output it.

9. An electronic device, characterized in that: include: A processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus; A computer program is stored in the memory, and when the program is executed by the processor, the processor executes the method according to any one of claims 1 to 4.

10. A computer-readable storage medium, characterized in that: The computer program executable by a processor is stored therein, and when the program runs on the processor, the processor is enabled to execute the method according to any one of claims 1 to 4.

Images