Vehicle-mounted tracking and pointing system and method for multi-source signal assisted positioning

Abstract

The invention provides a multi-source signal assisted positioning vehicle-mounted tracking and pointing system and method, and belongs to the field of photoelectric tracking. The system comprises a laser range finder, a vehicle-mounted pan-tilt camera, a first encoder, an interface communication module, a comprehensive control computer, a display, a data collector, a servo controller, a second encoder, a servo pan-tilt and a laser aiming confirmation device. According to the invention, a vehicle-mounted pan-tilt camera is used for identifying and capturing a target, laser ranging is used for assisting in positioning after the tracking target is confirmed, and a servo controller receives a positioning resolving signal after positioning resolving by combining encoder data on the vehicle-mounted pan-tilt camera, so that the center of a servo pan-tilt aiming optical axis is aligned to the center of the target. The rotary table is provided with a laser aiming confirmation device, the matching of a laser spot and a target is confirmed through a vehicle-mounted pan-tilt camera, and a confirmation signal is fed back to carry out a next target task. The feedback mechanism ensures the self-adaptive capability of the whole system, so that the system can be continuously adjusted in a dynamic environment to realize accurate aiming.

Description

Technical Field

[0001] The present invention belongs to the technical field of photoelectric tracking, and proposes a tracking and aiming system and method for global target recognition and vehicle-mounted tracking and aiming problems, and in particular relates to a vehicle-mounted tracking and aiming system and method for multi-source signal assisted positioning. Background Art

[0002] The optoelectronic tracking system is a highly integrated system device, which includes optics, mechanics, control, pattern recognition, signal processing and other fields. It uses photoelectric detectors as sensors and electromechanical control to ensure line of sight alignment and track the target, while maintaining line of sight stability during the process.

[0003] In the tracking operation of the optoelectronic tracking system, the limited field of view is a significant problem. The design and configuration of the optical sensor of the traditional optoelectronic tracking system determines the angle range it can cover. Due to the limitations of hardware and design principles, this field of view may not meet the tracking needs in some complex environments. For example, in military applications, if the target suddenly makes a large maneuver and exceeds the field of view of the tracking system, the system may lose the target, resulting in the failure of subsequent tracking tasks. Moreover, in the aviation monitoring scenario, when the target aircraft is in a fast-moving state or flying in a large area of ​​airspace, the limited field of view may not be able to continuously capture the target.

[0004] At the same time, the inability to provide feedback on the aiming effect is also a major drawback. During operation, the optoelectronic tracking system needs to accurately aim at the target to obtain high-quality data. However, existing systems often lack an effective feedback mechanism. Operators cannot accurately know the accuracy of the aiming and whether it is actually aimed at the key parts of the target. This may lead to inaccurate data collected. For some application scenarios that require precise aiming, such as target aiming of high-power microwave weapons and laser weapons, this inability to provide feedback on the aiming effect will seriously affect the effectiveness of the system. This lack of feedback may be due to a variety of factors, such as unreasonable sensor layout, the lack of corresponding evaluation mechanism in the data processing link, or the inability of the display interface to intuitively present the aiming status.

[0005] Attempts to improve the system in the market or in the research field include adding auxiliary sensors to expand the field of view, such as adding small wide-angle sensors around the main photoelectric sensor, and using the data from these auxiliary sensors to guide the main sensor to adjust its direction when the target approaches the edge of the field of view. However, this method increases the complexity and cost of the system, and there are certain technical difficulties in fusing the data of the auxiliary sensors with the main sensor.

[0006] In order to solve the problem of aiming effect feedback, some studies have proposed adding an aiming evaluation algorithm to the system to judge the aiming effect by analyzing the image features and tracking trajectory of the target. However, the accuracy of these algorithms will be greatly reduced in complex environments, such as when there are interference objects or the target features are not obvious, and the real-time performance is also difficult to guarantee. Summary of the invention

[0007] The present invention aims to solve the problems of field of view limitation of video target tracking and inability to provide feedback on aiming effect, and proposes a vehicle-mounted tracking and aiming system with multi-source signal assisted positioning in combination with multi-data source assisted positioning.

[0008] To achieve the above purpose, the specific technical solutions of the present invention are as follows:

[0009] A vehicle-mounted tracking and aiming system for multi-source signal assisted positioning, the system comprising a laser rangefinder, a vehicle-mounted pan-tilt camera, a first encoder, an interface communication module, an integrated control computer, a display, a data collector, a servo controller, a second encoder, a servo pan-tilt, and a laser aiming confirmer; the vehicle-mounted pan-tilt camera is connected to the laser rangefinder and the first encoder, the laser rangefinder, the vehicle-mounted pan-tilt camera and the first encoder are connected to the data collector, the data collector is connected to the interface communication module, the interface communication module is connected to the integrated control computer, the integrated control computer is simultaneously connected to the servo controller and the display, the servo controller is also connected to the vehicle-mounted pan-tilt camera and the servo pan-tilt, the servo pan-tilt is connected to the second encoder, the second encoder is connected to the data collector, and the laser aiming confirmer is connected to the servo pan-tilt.

[0010] Furthermore, the first encoder is installed on the vehicle-mounted pan-tilt camera to measure the posture information of the vehicle-mounted pan-tilt camera at two angles, namely, pitch and azimuth; the second encoder is installed on the servo pan-tilt to measure the posture information of the servo pan-tilt at two angles, namely, pitch and azimuth; each vehicle-mounted pan-tilt camera has two first encoders, which are respectively used to measure the posture information of the vehicle-mounted pan-tilt camera at two angles, namely, pitch and azimuth; the servo pan-tilt has two second encoders, which are respectively used to measure the posture information of the servo cloud at two angles, namely, pitch and azimuth.

[0011] Furthermore, there are two vehicle-mounted pan-tilt cameras, which are distributed on both sides of the roof. The vehicle-mounted pan-tilt cameras have two degrees of freedom, namely, pitch and azimuth, and can shoot video signals in different directions.

[0012] The present invention also proposes a vehicle-mounted tracking and aiming method for multi-source signal assisted positioning, the method comprising the following steps:

[0013] Step S1: The vehicle-mounted PTZ cameras are mounted on both ends of the vehicle roof to collect image information of surrounding targets, which is transmitted to the integrated control computer through the data acquisition device and the interface communication module, and the surrounding status is displayed on the display.

[0014] Step S2: The integrated control computer identifies and tracks the target in the image by improving the Yolo series target recognition algorithm. After selecting the target, the integrated control computer calculates the deviation between the center of the selected target and the center of the image taken by the vehicle-mounted pan-tilt camera, and controls the vehicle-mounted pan-tilt camera through the servo controller so that the aiming center of the vehicle-mounted pan-tilt camera is aligned with the target center.

[0015] Step S3: The laser rangefinder detects the distance between the vehicle-mounted pan-tilt camera and the target center and outputs a distance signal; the two first encoders respectively measure the posture information of the vehicle-mounted pan-tilt camera at two angles, namely, pitch and azimuth; the distance signal and the posture information of the vehicle-mounted pan-tilt camera are received by the integrated control computer through the data acquisition device via the interface communication module.

[0016] Step S4: The integrated control computer calculates the deviation between the servo platform aiming visual axis and the target visual axis according to the geometric relationship between the vehicle-mounted pan / tilt camera, the servo platform, and the target center, the distance signal, the vehicle-mounted pan / tilt camera attitude information, and the servo platform attitude information.

[0017] Step S5: According to the deviation between the aiming axis of the servo platform and the target axis, the servo controller controls the servo platform to align the center of its aiming axis with the target.

[0018] Step S6: The laser aiming confirmer emits a laser beam to illuminate the target, and uses the vehicle-mounted gimbal camera to confirm whether the image positions of the target and the laser spot coincide with each other to determine whether the target is locked. If there is a deviation, the position of the servo gimbal is adjusted according to the deviation to ultimately achieve the coincidence of the servo gimbal aiming visual axis and the target visual axis.

[0019] Furthermore, the deviation between the center of the selected target and the center of the image captured by the vehicle-mounted PTZ camera is calculated by the following formula:

[0020]

[0021] in, is the horizontal deviation between the center of the selected target and the center of the image captured by the vehicle-mounted PTZ camera. It is the vertical deviation between the center of the selected target and the center of the image captured by the vehicle-mounted PTZ camera; Indicates the pixel position of the target center in the horizontal direction, Indicates the pixel position of the target center in the vertical direction; Indicates the horizontal pixel position of the center of the image captured by the vehicle-mounted PTZ camera. Indicates the horizontal pixel position of the center of the image captured by the vehicle-mounted PTZ camera.

[0022] Furthermore, the aiming center of the vehicle-mounted pan / tilt camera is aligned with the target center in the following manner.

[0023] In the initial state, the vehicle-mounted pan-tilt camera moves with an adjustment amount of 2°. If the deviation between the center of the image taken by the vehicle-mounted pan-tilt camera and the target center is less than 10 pixels, the servo controller controls the vehicle-mounted pan-tilt camera to adjust the pitch and azimuth of the vehicle-mounted pan-tilt camera by 0.5°; if it is less than 5 pixels, the servo controller controls the vehicle-mounted pan-tilt camera to adjust the pitch and azimuth of the vehicle-mounted pan-tilt camera by 0.1°, until the deviation between the center of the image taken by the vehicle-mounted pan-tilt camera and the target center is less than 3 pixels, it is determined that the aiming center of the vehicle-mounted pan-tilt camera is aligned with the target center.

[0024] Furthermore, the deviation between the servo gimbal aiming axis and the target axis is calculated as follows:

[0025]

[0026] Then the aiming deviation of the servo gimbal relative to the initial state is for:

[0027]

[0028] in, Indicates the distance between the servo PTZ and the vehicle-mounted PTZ camera; Indicates the distance between the laser rangefinder and the center of the target. It indicates the pitch angle of the vehicle-mounted PTZ camera after the aiming center is aligned with the target center. Indicates the azimuth angle of the vehicle-mounted PTZ camera after the aiming center is aligned with the target center; Indicates the initial pitch angle of the servo gimbal. Indicates the initial azimuth of the servo gimbal; Indicates the center line and horizontal angle between the servo PTZ and the vehicle-mounted PTZ camera; It is the height difference between the aiming center of the vehicle-mounted PTZ camera and the aiming boresight center of the servo PTZ; Indicates the servo gimbal pitch deviation. Indicates the servo gimbal azimuth deviation. The servo gimbal pitch deviation and servo gimbal azimuth deviation refer to the deviation between the servo gimbal aiming axis and the target axis.

[0029] Furthermore, the servo controller converts the solved servo gimbal pitch deviation and servo gimbal azimuth deviation into the required PWM pulse, and sends a control signal to make the servo gimbal aiming axis continuously approach the target axis.

[0030] Furthermore, when the difference between the pitch deviation and azimuth deviation of the servo gimbal is less than 0.1 mrad, the laser aiming transmitter emits a light beam to illuminate the target, and the light spot and target position are matched through the meanshift algorithm. If the positions coincide, it means that the aiming axis of the servo gimbal is aligned with the target axis, and subsequent actions are performed; if the light spot position and the target position do not coincide, it means that the aiming axis of the servo gimbal is not aligned with the target axis. By confirming whether the aiming center of the vehicle-mounted gimbal camera is aligned with the target, the corresponding deviation of the servo gimbal is recalculated, and the servo gimbal is continued to be controlled by the servo controller until the light spot and the target coincide.

[0031] The invention proposes a vehicle-mounted tracking and aiming system with multi-source signal assisted positioning, which has the following beneficial effects:

[0032] 1) The vehicle-mounted PTZ camera of the present invention can obtain the surrounding environment information in all directions and efficiently capture the surrounding targets. The vehicle-mounted PTZ camera PTZ assists in positioning and calibration, and solves the deviation between the servo PTZ aiming axis and the target axis through geometric relationships. , quickly align the servo gimbal aiming the center of the visual axis with the target, emit a light beam through the laser aiming confirmer, and confirm whether the target is aimed at by combining the relative positions of the light spot and the target in the vehicle-mounted gimbal camera. The laser aiming confirmer emits a light beam and combines the relative positions of the target and the light spot in the vehicle-mounted gimbal camera to make an aiming judgment, that is, feedback on the aiming effect. This feedback mechanism ensures the adaptive ability of the entire system, enabling it to continuously adjust in a dynamic environment to achieve precise aiming.

[0033] 2) The present invention integrates the vehicle-mounted pan-tilt camera, laser rangefinder and encoder, and utilizes the relative position solution of the servo pan-tilt to achieve efficient target recognition and precise tracking, thereby significantly improving the performance and application capabilities of the vehicle-mounted tracking and aiming equipment. By integrating advanced image processing algorithms, real-time posture measurement of geometric solutions and laser confirmation mechanisms, the present invention optimizes the target capture and data processing process to ensure real-time feedback of the aiming effect. This innovation not only promotes the technological progress of the tracking and aiming system, but also provides strong support for the realization of more intelligent and efficient vehicle-mounted tracking and aiming system solutions, and has a far-reaching impact on promoting the widespread deployment of vehicle-mounted tracking and aiming systems in various applications such as active denial devices and high-power microwave weapons. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0035] Figure 1 This is a system block diagram of a vehicle-mounted tracking and aiming system for multi-source signal assisted positioning proposed by the present invention.

[0036] Figure 2 , Figure 3 The present invention is a geometrical schematic diagram of the deviation between the aiming visual axis of the servo pan-tilt platform and the target visual axis solved by the cooperation of the vehicle-mounted pan-tilt camera and the servo pan-tilt platform of the vehicle-mounted tracking and aiming system with multi-source signal auxiliary positioning of the present invention.

[0037] Figure 4 This is a roof distribution diagram of a servo pan / tilt and a vehicle-mounted pan / tilt camera of a vehicle-mounted tracking and aiming system for multi-source signal assisted positioning according to the present invention. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] The present invention provides a vehicle-mounted tracking and aiming system with multi-source signal assisted positioning, such as Figure 1 As shown, the system includes a laser rangefinder, a vehicle-mounted pan-tilt camera, a first encoder, an interface communication module, an integrated control computer, a display, a data collector, a servo controller, a second encoder, a servo pan-tilt, and a laser aiming confirmer.

[0040] The vehicle-mounted pan-tilt camera is connected to the laser rangefinder and the first encoder, the laser rangefinder, the vehicle-mounted pan-tilt camera and the first encoder are connected to the data collector, the data collector is connected to the interface communication module, the interface communication module is connected to the integrated control computer, the integrated control computer is simultaneously connected to the servo controller and the display, the servo controller is also connected to the vehicle-mounted pan-tilt camera and the servo pan-tilt, the servo pan-tilt is connected to the second encoder, the second encoder is connected to the data collector, and the laser aiming confirmer is connected to the servo pan-tilt.

[0041] The first encoder is installed on the vehicle-mounted PTZ camera to measure the attitude information of the vehicle-mounted PTZ camera at two angles, namely, pitch and azimuth. The second encoder is installed on the servo PTZ to measure the attitude information of the servo PTZ at two angles, namely, pitch and azimuth. Therefore, each vehicle-mounted PTZ camera has two first encoders, which are used to measure the attitude information of the vehicle-mounted PTZ camera at two angles, namely, pitch and azimuth; the servo PTZ has two second encoders, which are used to measure the attitude information of the servo PTZ at two angles, namely, pitch and azimuth.

[0042] There are two vehicle-mounted PTZ cameras, distributed on both sides of the roof. Figure 4 As shown, it ensures rapid capture of targets at all viewing angles. The vehicle-mounted PTZ camera has two degrees of freedom, pitch and azimuth, and can capture video signals in different directions.

[0043] The servo controller shown is connected to the vehicle-mounted pan-tilt camera and the servo pan-tilt, and is used to control the motors of the servo pan-tilt and the vehicle-mounted pan-tilt camera.

[0044] The laser rangefinder is used to detect the distance between the vehicle-mounted pan / tilt camera and the target center, and output a distance signal.

[0045] The data collector is used to collect, process and transmit data. In the present invention, the data collector is connected to the laser rangefinder, the vehicle-mounted PTZ camera, the first encoder and the second encoder to collect the distance signal output by the laser rangefinder, the image and video information captured by the vehicle-mounted PTZ camera, and the posture information measured by the first encoder and the second encoder; the data collector processes and converts these signals and then transmits them to the interface communication module to ensure protocol matching and transmission efficiency.

[0046] The laser aiming confirmer is used to emit a laser beam to illuminate a target.

[0047] The interface communication module is an optical fiber communication module, a serial port communication module or a WIFI communication module.

[0048] Based on the above-mentioned vehicle-mounted tracking and aiming system for multi-source signal assisted positioning, the present invention also proposes a vehicle-mounted tracking and aiming method for multi-source signal assisted positioning, the method comprising the following steps:

[0049] Step S1: The vehicle-mounted PTZ cameras are mounted on both ends of the vehicle roof to collect image information of surrounding targets, which is transmitted to the integrated control computer through the data acquisition device and the interface communication module, and the surrounding status is displayed on the display.

[0050] The vehicle-mounted pan-tilt camera collects surrounding target and environmental information through the data collector and then transmits it to the integrated control computer through the interface communication module. The integrated control computer adopts the Yolov3 network with Resnet18 as the backbone, and uses the coco dataset and VOT dataset for training. The trained model is added to the dataset produced after the vehicle-mounted pan-tilt camera collects targets and annotates them for transfer learning to obtain the recognition network.

[0051] Step S2: The integrated control computer identifies and tracks the target in the image by improving the Yolo series target recognition algorithm. After selecting the target, the integrated control computer calculates the deviation between the center of the selected target and the center of the image taken by the vehicle-mounted pan-tilt camera, and controls the vehicle-mounted pan-tilt camera through the servo controller so that the aiming center of the vehicle-mounted pan-tilt camera is aligned with the target center.

[0052] Select the target to track through the display operation interface connected to the integrated control computer. The vehicle-mounted PTZ camera is in the right-hand coordinate system, and the pixel coordinates of the target center are , Indicates the pixel position of the target center in the horizontal direction, Indicates the pixel position of the target center in the vertical direction. The pixel coordinates of the center of the image captured by the vehicle-mounted PTZ camera are , Indicates the horizontal pixel position of the center of the image captured by the vehicle-mounted PTZ camera. Indicates the horizontal pixel position of the center of the image captured by the vehicle-mounted PTZ camera.

[0053] The deviation between the center of the selected target and the center of the image captured by the vehicle-mounted PTZ camera is calculated using the following formula:

[0054]

[0055] in, is the horizontal deviation between the center of the selected target and the center of the image captured by the vehicle-mounted PTZ camera. It is the vertical deviation between the center of the selected target and the center of the image taken by the vehicle-mounted pan-tilt camera.

[0056] In the initial state, the vehicle-mounted pan-tilt camera moves with an adjustment amount of 2°. If the deviation between the center of the image taken by the vehicle-mounted pan-tilt camera and the target center is less than 10 pixels, the servo controller controls the vehicle-mounted pan-tilt camera to adjust the pitch and azimuth of the vehicle-mounted pan-tilt camera by 0.5°; if it is less than 5 pixels, the servo controller controls the vehicle-mounted pan-tilt camera to adjust the pitch and azimuth of the vehicle-mounted pan-tilt camera by 0.1°, until the deviation between the center of the image taken by the vehicle-mounted pan-tilt camera and the target center is less than 3 pixels, it is determined that the aiming center of the vehicle-mounted pan-tilt camera is aligned with the target center.

[0057] Step S3: The laser rangefinder detects the distance between the vehicle-mounted pan-tilt camera and the target center and outputs a distance signal; the two first encoders respectively measure the posture information of the vehicle-mounted pan-tilt camera at two angles, namely, pitch and azimuth; the distance signal and the posture information of the vehicle-mounted pan-tilt camera are received by the integrated control computer through the data acquisition device via the interface communication module.

[0058] Step S4: The integrated control computer calculates the deviation between the servo platform aiming visual axis and the target visual axis according to the geometric relationship between the vehicle-mounted pan / tilt camera, the servo platform, and the target center, the distance signal, the vehicle-mounted pan / tilt camera attitude information, and the servo platform attitude information.

[0059] Combined with the target center orientation, the vehicle-mounted pan / tilt camera close to the target center is selected to calculate the deviation between the servo pan / tilt aiming axis and the target axis, and cooperate with the servo pan / tilt tracking and aiming.

[0060] The two second encoders measure the attitude information of the servo pan-tilt platform at two angles, namely, pitch and azimuth, respectively. The attitude information of the servo pan-tilt platform is transmitted to the integrated control computer through the data acquisition device via the interface communication module.

[0061] like Figure 2 As shown in the figure, point A represents the center of the vehicle-mounted pan-tilt camera (the intersection of the extension lines of the pitch-azimuth axis), and point D represents the center of the servo pan-tilt (the intersection of the extension lines of the pitch-azimuth axis). The coordinate system of the vehicle-mounted pan-tilt camera is established with A and D as the origin. And the servo gimbal coordinate system , point B represents the equal height position of the vehicle-mounted pan-tilt camera in the coordinate system of the servo pan-tilt, point C represents the target center, point E represents the height position of the target center in the servo pan-tilt coordinate system, and point F represents the height position of the target center in the coordinate system of the vehicle-mounted pan-tilt camera. Figure 3 The figure shows the relative position of the vehicle-mounted PTZ camera and the servo PTZ from a bird's-eye view. Point A' represents the relative position of point A to The projection in the plane direction, D' represents the D point relative to The projection in the plane direction, point C' represents point C relative to The projection in the plane direction. The posture of the vehicle-mounted pan-tilt camera when the aiming center is aimed at the target center is , It indicates the pitch angle of the vehicle-mounted PTZ camera after the aiming center is aligned with the target center. It indicates the azimuth angle after the aiming center of the vehicle-mounted PTZ camera is aligned with the target center. When the aiming center of the vehicle-mounted PTZ camera is aligned with the target center, the distance AC between the vehicle-mounted PTZ camera and the target center detected by the laser rangefinder is , the distance between the servo PTZ and the vehicle-mounted PTZ camera ( )for , the center line of the servo pan / tilt and the vehicle-mounted pan / tilt camera and the horizontal angle are , the height difference between the aiming center of the vehicle-mounted PTZ camera and the aiming axis center of the servo PTZ is , assuming that the servo gimbal posture after aiming at the target is , Indicates the pitch angle of the servo gimbal after aiming at the target. Indicates the azimuth of the servo gimbal after aiming at the target. The initial state of the servo gimbal is , Indicates the initial pitch angle of the servo gimbal. It represents the initial azimuth of the servo gimbal. By using the triangle cosine theorem and sine theorem, the attitude of the servo gimbal after aiming at the target can be solved. :

[0062]

[0063] Then the aiming deviation of the servo gimbal relative to the initial state is for:

[0064]

[0065] in, Indicates the servo gimbal pitch deviation. Indicates the servo gimbal azimuth deviation. The servo gimbal pitch deviation and servo gimbal azimuth deviation refer to the deviation between the servo gimbal aiming axis and the target axis.

[0066] Step S5: According to the deviation between the aiming axis of the servo platform and the target axis, the servo controller controls the servo platform to align the center of its aiming axis with the target.

[0067] The servo controller converts the solved servo gimbal pitch deviation and servo gimbal azimuth deviation into the required PWM pulse, and sends a control signal to make the servo gimbal aiming axis continuously approach the target axis. The servo control can use optimized PID control method or model predictive control to improve response speed and stability and reduce overshoot.

[0068] Step S6: The laser aiming confirmer emits a laser beam to illuminate the target, and uses the vehicle-mounted gimbal camera to confirm whether the image positions of the target and the laser spot coincide with each other to determine whether the target is locked. If there is a deviation, the position of the servo gimbal is adjusted according to the deviation to ultimately achieve the coincidence of the servo gimbal aiming visual axis and the target visual axis.

[0069] When the difference between the pitch deviation and azimuth deviation of the servo gimbal is less than 0.1 mrad, the laser aiming transmitter emits a light beam to illuminate the target, and the light spot and target position are matched through the meanshift algorithm. If the positions coincide, it means that the aiming axis of the servo gimbal is aligned with the target axis, and subsequent actions are performed, such as microwave or laser irradiation. If the light spot position and the target position do not coincide, it means that the aiming axis of the servo gimbal is not aligned with the target axis. By confirming whether the aiming center of the vehicle-mounted gimbal camera is aligned with the target, the corresponding deviation of the servo gimbal is recalculated, and the servo gimbal is continued to be controlled by the servo controller until the light spot and the target coincide. The laser aiming confirmer emits a light beam and combines the vehicle-mounted gimbal camera to make an aiming judgment on the relative position of the target and the light spot, which is the feedback on the aiming effect. This feedback mechanism ensures the adaptability of the entire system, enabling it to continuously adjust in a dynamic environment to achieve precise aiming.

[0070] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. A vehicle-mounted tracking and aiming system with multi-source signal assisted positioning, characterized in that: The system includes a laser rangefinder, a vehicle-mounted pan-tilt camera, a first encoder, an interface communication module, an integrated control computer, a display, a data collector, a servo controller, a second encoder, a servo pan-tilt, and a laser aiming confirmer; the vehicle-mounted pan-tilt camera is connected to the laser rangefinder and the first encoder, the laser rangefinder, the vehicle-mounted pan-tilt camera and the first encoder are connected to the data collector, the data collector is connected to the interface communication module, the interface communication module is connected to the integrated control computer, the integrated control computer is simultaneously connected to the servo controller and the display, the servo controller is also connected to the vehicle-mounted pan-tilt camera and the servo pan-tilt, the servo pan-tilt is connected to the second encoder, the second encoder is connected to the data collector, and the laser aiming confirmer is connected to the servo pan-tilt.

2. According to the multi-source signal assisted positioning vehicle-mounted tracking and aiming system of claim 1, it is characterized in that: The first encoder is installed on the vehicle-mounted pan-tilt camera to measure the attitude information of the vehicle-mounted pan-tilt camera at two angles, namely, pitch and azimuth; the second encoder is installed on the servo pan-tilt to measure the attitude information of the servo pan-tilt at two angles, namely, pitch and azimuth; each vehicle-mounted pan-tilt camera has two first encoders, which are used to measure the attitude information of the vehicle-mounted pan-tilt camera at two angles, namely, pitch and azimuth; the servo pan-tilt has two second encoders, which are used to measure the attitude information of the servo cloud at two angles, namely, pitch and azimuth.

3. The vehicle-mounted tracking and aiming system for multi-source signal assisted positioning according to claim 2 is characterized in that: There are two vehicle-mounted pan-tilt cameras, which are distributed on both sides of the vehicle roof. The vehicle-mounted pan-tilt cameras have two degrees of freedom, namely, pitch and azimuth, and can shoot video signals in different directions.

4. A vehicle-mounted tracking and aiming method for multi-source signal-assisted positioning, characterized in that: The method comprises the following steps: Step S1: The vehicle-mounted PTZ cameras are mounted on both ends of the vehicle roof to collect surrounding target image information, which is transmitted to the integrated control computer through the data acquisition device and the interface communication module, and the surrounding status is displayed on the display; Step S2: The integrated control computer recognizes and tracks the target in the image by improving the Yolo series target recognition algorithm. After selecting the target, the integrated control computer calculates the deviation between the center of the selected target and the center of the image taken by the vehicle-mounted pan-tilt camera, and controls the vehicle-mounted pan-tilt camera through the servo controller so that the aiming center of the vehicle-mounted pan-tilt camera is aligned with the target center. Step S3: The laser rangefinder detects the distance between the vehicle-mounted PTZ camera and the target center and outputs a distance signal; the two first encoders respectively measure the attitude information of the vehicle-mounted PTZ camera at two angles, namely, pitch and azimuth; the distance signal and the attitude information of the vehicle-mounted PTZ camera are received by the integrated control computer through the data acquisition device and the interface communication module; Step S4: the integrated control computer calculates the deviation between the servo platform aiming visual axis and the target visual axis according to the geometric relationship between the vehicle-mounted PTZ camera, the servo platform, and the target center, the distance signal, the vehicle-mounted PTZ camera attitude information, and the servo platform attitude information; Step S5: According to the deviation between the aiming visual axis of the servo platform and the target visual axis, the servo controller controls the servo platform to align the center of its aiming visual axis with the target; Step S6: The laser aiming confirmer emits a laser beam to illuminate the target, and uses the vehicle-mounted gimbal camera to confirm whether the image positions of the target and the laser spot coincide with each other to determine whether the target is locked. If there is a deviation, the position of the servo gimbal is adjusted according to the deviation to ultimately achieve the coincidence of the servo gimbal aiming visual axis and the target visual axis.

5. The vehicle-mounted tracking and aiming method for multi-source signal assisted positioning according to claim 4 is characterized in that: The deviation between the center of the selected target and the center of the image captured by the vehicle-mounted PTZ camera is calculated using the following formula: in, is the horizontal deviation between the center of the selected target and the center of the image captured by the vehicle-mounted PTZ camera. It is the vertical deviation between the center of the selected target and the center of the image captured by the vehicle-mounted PTZ camera; Indicates the pixel position of the target center in the horizontal direction, Indicates the pixel position of the target center in the vertical direction; Indicates the horizontal pixel position of the center of the image captured by the vehicle-mounted PTZ camera. Indicates the horizontal pixel position of the center of the image captured by the vehicle-mounted PTZ camera.

6. The vehicle-mounted tracking and aiming method for multi-source signal assisted positioning according to claim 5 is characterized in that: The aiming center of the vehicle-mounted PTZ camera is aligned to the target center in the following ways: In the initial state, the vehicle-mounted pan-tilt camera moves with an adjustment amount of 2°. If the deviation between the center of the image taken by the vehicle-mounted pan-tilt camera and the target center is less than 10 pixels, the servo controller controls the vehicle-mounted pan-tilt camera to adjust the pitch and azimuth of the vehicle-mounted pan-tilt camera by 0.5°; if it is less than 5 pixels, the servo controller controls the vehicle-mounted pan-tilt camera to adjust the pitch and azimuth of the vehicle-mounted pan-tilt camera by 0.1°, until the deviation between the center of the image taken by the vehicle-mounted pan-tilt camera and the target center is less than 3 pixels, it is determined that the aiming center of the vehicle-mounted pan-tilt camera is aligned with the target center.

7. The vehicle-mounted tracking and aiming method for multi-source signal assisted positioning according to claim 4 is characterized in that: The deviation between the servo gimbal's aiming boresight and the target boresight is calculated as follows: Then the aiming deviation of the servo gimbal relative to the initial state is for, in, Indicates the distance between the servo PTZ and the vehicle-mounted PTZ camera; Indicates the distance between the laser rangefinder and the center of the target. It indicates the pitch angle of the vehicle-mounted PTZ camera after the aiming center is aligned with the target center. Indicates the azimuth angle of the vehicle-mounted PTZ camera after the aiming center is aligned with the target center; Indicates the initial pitch angle of the servo gimbal. Indicates the initial azimuth of the servo gimbal; Indicates the center line and horizontal angle between the servo PTZ and the vehicle-mounted PTZ camera; It is the height difference between the aiming center of the vehicle-mounted PTZ camera and the aiming boresight center of the servo PTZ; Indicates the servo gimbal pitch deviation. Indicates the servo gimbal azimuth deviation. The servo gimbal pitch deviation and servo gimbal azimuth deviation refer to the deviation between the servo gimbal aiming axis and the target axis.

8. The vehicle-mounted tracking and aiming method for multi-source signal assisted positioning according to claim 7 is characterized in that: The servo controller converts the solved servo gimbal pitch deviation and servo gimbal azimuth deviation into the required PWM pulse, and sends a control signal to make the servo gimbal aiming axis continuously approach the target axis.

9. The vehicle-mounted tracking and aiming method for multi-source signal assisted positioning according to claim 8, characterized in that: When the difference between the pitch deviation and azimuth deviation of the servo gimbal is less than 0.1mrad, the laser aiming transmitter emits a light beam to illuminate the target, and the light spot and target position are matched through the meanshift algorithm. If the positions coincide, it means that the aiming axis of the servo gimbal is aligned with the target axis, and subsequent actions are performed; if the light spot position and the target position do not coincide, it means that the aiming axis of the servo gimbal is not aligned with the target axis. By confirming whether the aiming center of the vehicle-mounted gimbal camera is aligned with the target, the corresponding deviation of the servo gimbal is recalculated, and the servo gimbal is continued to be controlled by the servo controller until the light spot and the target coincide.

Images