Panoramic target search and monocular linkage tracking methods, devices and computer equipment

Abstract

This invention relates to a panoramic target search and monocular linkage tracking method, apparatus, and computer device. The method includes acquiring target information; identifying the target in a panoramic image using a target detection algorithm; determining the target monocular camera corresponding to the target from the panoramic image; tracking the target using the target monocular camera according to a preset method, the preset method including cyclically switching between a monocular tracking mode of a first preset duration and a panoramic shooting mode of a second preset duration; when the target monocular camera switches to the panoramic shooting mode, re-synthesizing an updated panoramic image; determining whether the target has been lost from the field of view of the target monocular camera in monocular tracking mode; if so, re-identifying the target from the updated panoramic image. The linkage mechanism between monocular and panoramic tracking in this invention allows for flexible adjustment of the tracking strategy, maintaining tracking accuracy and stability, and enhancing tracking robustness in complex scenes.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a panoramic target search and monocular linkage tracking method, apparatus, and computer equipment. Background Technology

[0002] In the field of computer vision and image processing, target tracking technology has always been a research hotspot and a challenge. With the increasing demands of surveillance systems, intelligent security, and autonomous driving, higher requirements are being placed on the real-time performance, accuracy, and continuity of target tracking. Traditional target tracking methods mostly rely on a single camera, which has a limited field of view and is prone to losing the target when it moves to the edge of the field of view, leading to tracking failure. Furthermore, a single camera often exhibits unstable tracking performance in complex scenes (such as occlusion, changes in lighting, and background interference), making it difficult to meet the needs of practical applications.

[0003] Based on the above problems, there is an urgent need to propose a new target tracking method. Summary of the Invention

[0004] The main objective of this invention is to provide a panoramic target search and monocular linkage tracking method, apparatus, and computer equipment to solve the problem of target loss in the prior art.

[0005] To achieve the above objectives, the first aspect of the present invention provides a panoramic target search and monocular linkage tracking method, comprising:

[0006] Obtain information about the target being tracked;

[0007] The target is identified and tracked in the panoramic image by a target detection algorithm. The panoramic image is stitched together from images captured by multiple monocular cameras at the same time in panoramic shooting mode.

[0008] Identify the target monocular camera corresponding to the tracking target from the panoramic image;

[0009] The target is tracked using the target monocular camera according to a preset method, the preset method including a cyclic switching between a monocular tracking mode of a first preset duration and a panoramic shooting mode of a second preset duration;

[0010] When the target monocular camera switches to the panoramic shooting mode, it re-synthesizes an updated panoramic image. The updated panoramic image is stitched together from the latest images captured by multiple monocular cameras at the same moment in the panoramic shooting mode.

[0011] Determine whether the tracked target has been lost from the field of view of the target monocular camera in the monocular tracking mode;

[0012] If so, the tracking target is re-identified from the updated panoramic image.

[0013] Furthermore, the step of identifying the tracking target in the panoramic image using a target detection algorithm, wherein the panoramic image is stitched together from images captured simultaneously by multiple monocular cameras in panoramic shooting mode, includes:

[0014] Images captured simultaneously by each of the monocular cameras in a preset panoramic shooting mode;

[0015] The images captured simultaneously by each of the monocular cameras are stitched together using a preset image stitching algorithm to generate the panoramic image;

[0016] The target detection algorithm is used to traverse the panoramic image to search for whether there is a target in the panoramic image that is similar to the tracking target information;

[0017] If so, then a target similar to the tracking target information will be used as the tracking target.

[0018] Further, the step of using the target monocular camera to track the target according to a preset method, wherein the preset method includes a step of cyclically switching between a monocular tracking mode of a first preset duration and a panoramic shooting mode of a second preset duration, including:

[0019] Set the current shooting mode of the target monocular camera to monocular tracking mode;

[0020] The target is tracked using the monocular tracking mode, and the first tracking time starting point is recorded.

[0021] When the tracking duration from the first tracking time start point reaches the first preset duration, the monocular tracking mode is switched to the panoramic shooting mode, and the second tracking time start point of switching to the panoramic shooting mode is recorded. The field of view of the panoramic shooting mode is greater than or equal to the field of view of the monocular tracking mode.

[0022] When the tracking duration from the second tracking time start point reaches the second preset duration, the panoramic shooting mode is switched to the monocular tracking mode.

[0023] Furthermore, the step of re-identifying the tracking target from the updated panoramic image includes:

[0024] Calculate the speed of the tracked target;

[0025] Adjust the values ​​of the first preset duration and / or the second preset duration according to the preset speed-duration correspondence.

[0026] Further, before the step of calculating the speed of the tracked target, the following steps are included:

[0027] Obtain the current position of the tracked target in the updated panoramic image;

[0028] The current location and the time point at which the updated panoramic image was acquired are associated and added to the tracking target's movement path.

[0029] Further, the step of re-identifying the tracking target from the updated panoramic image includes:

[0030] Based on the movement path of the tracking target, a preset tracking algorithm is used to estimate the location of the tracking target to obtain the estimated location area;

[0031] The target is re-identified and tracked starting from the estimated location region of the updated panoramic image.

[0032] Further, the step of estimating the location of the target based on its movement path using a preset tracking algorithm to obtain the estimated location area includes:

[0033] Extract the N closest time points and their corresponding location information from the tracking target's movement path, where N is a preset positive integer;

[0034] Using the N time points and their corresponding location information as input, the preset tracking algorithm is used to estimate the location of the tracking target at the current time point or a preset future time point, and the coordinate value of the estimated location is obtained.

[0035] Centered on the coordinates of the estimated location, the estimated location area is determined according to the preset search range radius.

[0036] A second aspect of the present invention provides a panoramic target search and monocular linkage tracking device, comprising:

[0037] The acquisition module is used to acquire information about the tracking target.

[0038] The panoramic search module is used to identify and track targets in panoramic images using a target detection algorithm. The panoramic images are stitched together from images captured by multiple monocular cameras at the same time in panoramic shooting mode.

[0039] A camera determination module is used to determine, from the panoramic image, the target monocular camera corresponding to the tracking target;

[0040] A monocular tracking module is used to track the target using the target monocular camera according to a preset method, wherein the preset method includes a cyclic switching between a monocular tracking mode of a first preset duration and a panoramic shooting mode of a second preset duration.

[0041] A panoramic image generation module is used to re-synthesize an updated panoramic image when the target monocular camera switches to the panoramic shooting mode. The updated panoramic image is stitched together from the latest images captured by multiple monocular cameras at the same moment in the panoramic shooting mode.

[0042] The judgment module is used to determine whether the tracked target has been lost from the field of view of the target monocular camera in the monocular tracking mode;

[0043] A re-identification module is used to re-identify the tracked target from the updated panoramic image if it is lost.

[0044] Furthermore, the monocular tracking module includes:

[0045] The image acquisition unit is used to acquire images captured by each of the monocular cameras at the same time in a preset panoramic shooting mode;

[0046] The image stitching unit is used to stitch together the images captured by each of the monocular cameras at the same time using a preset image stitching algorithm to generate the panoramic image;

[0047] The target search unit is used to traverse the panoramic image using the target detection algorithm to search for whether there is a target in the panoramic image that is similar to the tracking target information;

[0048] The tracking target determination unit is used to identify a target similar to the tracking target information as the tracking target if such a target exists.

[0049] A third aspect of the present invention provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method for panoramic target search and monocular linkage tracking.

[0050] The present invention provides a panoramic target search and monocular linkage tracking method, apparatus, and computer equipment, which have the following beneficial effects:

[0051] By fusing the fields of view of multiple monocular cameras into a panoramic image, the tracking system's field of view is greatly expanded, effectively avoiding the target loss problem caused by the limited field of view of a single camera. Even if the target moves to the edge of the field of view of a single camera, the panoramic image can ensure the target's continued visibility, thus maintaining the continuity of tracking. In addition, after identifying the target in the panoramic image, the system can quickly determine and switch to a monocular camera near the target's location for fine tracking. This linkage mechanism between monocular and panoramic cameras allows for flexible adjustment of tracking strategies when facing complex scenes such as occlusion, changes in lighting, and background interference, maintaining the accuracy and stability of tracking and enhancing the robustness of tracking in complex scenes. At the same time, through periodic or conditionally triggered panoramic shooting modes, the system can continuously update its understanding of the environment, further improving its adaptability to complex scenes. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating a panoramic target search and monocular linkage tracking method in one embodiment of the present invention;

[0053] Figure 2 This is a schematic diagram of the structure of a panoramic target search and monocular linkage tracking device in one embodiment of the present invention;

[0054] Figure 3 This is a schematic block diagram of the structure of a computer device according to an embodiment of the present invention.

[0055] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0057] The embodiments of the present invention can be executed by a processor, which can be a cloud server or a local central processing unit.

[0058] Reference Figure 1 This embodiment illustrates a panoramic target search and monocular linkage tracking method, including:

[0059] S1. Obtain information about the target being tracked;

[0060] S2. Identify the tracking target in the panoramic image using a target detection algorithm. The panoramic image is stitched together from images captured by multiple monocular cameras at the same time in panoramic shooting mode.

[0061] S3. Determine the target monocular camera corresponding to the tracking target from the panoramic image;

[0062] S4. The target is tracked using the target monocular camera in a preset manner, the preset manner including a cyclic switching between a monocular tracking mode of a first preset duration and a panoramic shooting mode of a second preset duration;

[0063] S5. When the target monocular camera switches to the panoramic shooting mode, a new panoramic image is re-synthesized. The new panoramic image is stitched together from the latest images captured by multiple monocular cameras at the same moment in the panoramic shooting mode.

[0064] S6. Determine whether the tracked target has been lost from the field of view of the target monocular camera in the monocular tracking mode;

[0065] S7. If so, the tracking target is re-identified from the updated panoramic image.

[0066] In this embodiment, in step S1 above, initial information about the target to be tracked can be obtained through user input (such as clicking the screen to select a target), preset rules (such as tracking the largest or a specific color moving object), or automatic detection algorithms (such as target recognition based on features such as shape and texture). The automatic detection algorithm can employ feasible algorithms from the prior art, which will not be elaborated upon here.

[0067] In step S2 above, a panoramic image refers to a single image with a wide field of view formed by stitching together images captured by multiple monocular cameras from different angles at the same time.

[0068] Multiple monocular cameras are pre-positioned, for example, around the monitored area or the potential movement area of ​​the tracked target, to ensure image information is captured from different angles and orientations. The spacing and height between the monocular cameras should be adjusted according to the specific environment to ensure sufficient field-of-view overlap for subsequent image stitching. In some applications, such as indoor monitoring or tracking of specific areas, multiple monocular cameras can be mounted on walls, ceilings, or fixed brackets to form a certain encirclement or coverage network. For example, in autonomous driving monitoring, multiple monocular cameras can be mounted around the vehicle. The resolution, focal length, and viewing angle of the multiple monocular cameras should be set according to actual needs to ensure that the captured image quality meets the requirements of subsequent processing. All monocular cameras participating in panoramic image stitching should have synchronous triggering capabilities to ensure that images are captured at the same time to eliminate image misalignment caused by time differences.

[0069] The aforementioned target detection algorithm is a computer vision algorithm used to automatically identify and locate the tracked target in an image. For example, it could be an R-CNN algorithm. When the target detection algorithm is applied to a panoramic image, the algorithm searches for regions in the image that match the information of the tracked target and outputs the position and size information of the tracked target. This step is used to quickly and accurately locate the tracked target in the panoramic image.

[0070] In step S3 above, based on the position of the target in the panoramic image, and combined with the field of view and position information of each monocular camera, the monocular camera closest to the target is calculated and determined as the target monocular camera, thereby ensuring that the tracking task is undertaken by the most suitable monocular camera, improving tracking efficiency and accuracy.

[0071] In step S4 above, monocular tracking mode refers to a mode in which a single camera continuously tracks the target. In this mode, the camera continuously focuses on the target and adjusts parameters to maintain tracking. Panoramic shooting mode refers to all cameras participating in stitching the panoramic image simultaneously capturing images within their respective fields of view for subsequent panoramic image updates. The target monocular camera activates monocular tracking mode to continuously track the target. Simultaneously, it cycles between monocular tracking mode and panoramic shooting mode according to a preset timetable or condition (such as a decrease in tracking confidence). By cyclically switching modes, the continuity of tracking is ensured, and the target position can be reassessed through panoramic shooting mode when necessary, improving the stability and robustness of tracking.

[0072] In step S5 above, in panoramic shooting mode, all cameras participating in the stitching simultaneously capture images within their respective fields of view and transmit these images to the processor. The processor uses an image stitching algorithm to re-synthesize the updated panoramic image. By periodically updating the panoramic image, the real-time perception capability of the environment is ensured.

[0073] In step S6 above, "loss within field of view" refers to the inability to effectively identify or locate the target within the current field of view of the monocular camera. This is usually caused by the target moving out of the field of view, being obstructed, or the tracking algorithm failing. In monocular tracking mode, it is detected in real time whether the target is still within the field of view and being effectively tracked. If the target moves out of the field of view, is obstructed for too long, or the tracking algorithm fails and cannot locate the target, it is determined that the target is lost.

[0074] In step S7 above, once the target is determined to be lost, the target detection algorithm is immediately restarted to search the updated panoramic image, relocate and identify the tracking target. Then, based on the new target location information, a new target monocular camera is determined and tracking continues. By re-identifying the target from the panoramic image and resuming tracking, the problem of tracking interruption caused by target loss is effectively solved, improving the reliability and continuity of the tracking system.

[0075] This embodiment greatly expands the field of view of the tracking system by fusing the fields of view of multiple monocular cameras into a panoramic image. This effectively avoids the problem of target loss caused by the limited field of view of a single camera. Even if the target moves to the edge of the field of view of a single camera, the panoramic image can ensure the continuous visibility of the target, thereby maintaining the continuity of tracking. In addition, after identifying the target in the panoramic image, it can quickly determine and switch to a monocular camera near the target's location for fine tracking. This linkage mechanism between monocular and panoramic cameras allows for flexible adjustment of tracking strategies when facing complex scenes such as occlusion, changes in lighting, and background interference, maintaining the accuracy and stability of tracking and enhancing the robustness of tracking in complex scenes. At the same time, through periodic or conditionally triggered panoramic shooting modes, the system can continuously update its understanding of the environment, further improving its adaptability to complex scenes.

[0076] In one embodiment, step S2, which involves identifying the tracking target in a panoramic image using a target detection algorithm, wherein the panoramic image is stitched together from images captured simultaneously by multiple monocular cameras in panoramic shooting mode, includes:

[0077] S201. Acquire images captured by each of the monocular cameras at the same time in a preset panoramic shooting mode;

[0078] S202. The images captured by each of the monocular cameras at the same time are stitched together using a preset image stitching algorithm to generate the panoramic image;

[0079] S203. Using the target detection algorithm, traverse the panoramic image to search for whether there is a target in the panoramic image that is similar to the tracking target information;

[0080] S204. If so, then a target similar to the tracking target information shall be taken as the tracking target.

[0081] In this embodiment, in step S201 above, a panoramic shooting command is sent to all monocular cameras participating in panoramic image construction to ensure that they start shooting at the same time. Each monocular camera captures image data within its own field of view and transmits these images to the processor.

[0082] In step S202 above, after receiving image data from each camera, the images are processed using a preset image stitching algorithm. This preset image stitching algorithm is used to stitch images captured by multiple monocular cameras into a single panoramic image. The algorithm needs to consider factors such as the positional relationship between the cameras, overlapping areas of the images, and differences in lighting conditions to achieve high-quality image stitching. For example, image stitching can be performed using the following methods: Preprocessing operations such as denoising and distortion correction are performed on the images captured by each camera to improve image quality and reduce stitching errors; key points and their descriptors are extracted from the images using feature detection algorithms (such as SIFT, SURF, etc.); matching feature point pairs are found in the images of adjacent cameras using feature matching algorithms (such as FLANN, BFMatcher, etc.); based on the matching feature point pairs, a transformation matrix (such as affine transformation, perspective transformation, or homography matrix) is calculated between adjacent images, and the transformation matrix is ​​applied to transform the images so that adjacent images are aligned in the stitching area; image fusion algorithms such as weighted averaging and multi-band fusion are used in the overlapping area to eliminate stitching artifacts and improve the naturalness of the image; all processed images are stitched together according to a preset layout and order to form the final panoramic image. Using the above methods, images captured by multiple monocular cameras can be effectively stitched into a single panoramic image.

[0083] In step S203 above, the target detection algorithm is applied to the generated panoramic image. The algorithm traverses each region of the image, matching the tracking target information with preset target features (such as color, shape, texture, etc.) to find whether similar targets exist. This achieves rapid identification and localization of the tracking target in the panoramic image. An example can be taken using the R-CNN (Regions with Convolutional Neural Networks) algorithm to illustrate the specific process of the object detection algorithm: The input image is segmented and merged using a selective search algorithm to generate multiple candidate regions. The selective search algorithm calculates the similarity of image regions (such as color, texture, size, etc.) and gradually merges similar regions to generate the final candidate regions. Each candidate region is cropped and scaled to make its size uniform to the size required for the CNN network input (e.g., 227x227). The normalized candidate regions are input into a pre-trained CNN network to extract the feature vector of each candidate region. Commonly used CNN network structures include AlexNet and VGGNet. Support Vector Machines (SVMs) are used to classify the feature vectors, determining whether the candidate region contains the target object and assigning the target object's category. An SVM classifier is trained for each category. A bounding box regressor is then used. The bounding box regressor fine-tunes the bounding box positions of candidate regions to improve localization accuracy. The bounding box regressor learns the mapping relationship between candidate regions and ground truth bounding boxes through a linear regression model. After classification, candidate regions are grouped according to categories. In each category, non-maximum suppression is performed based on the candidate region's score (given by the SVM classifier) ​​and overlap (such as IoU, Intersection over Union). The candidate region with the highest score is retained, and other candidate regions with an overlap exceeding a certain threshold are removed. The candidate regions processed by NMS are output as the final detection result, which includes the target object's category, bounding box position, and score.

[0084] In step S204 above, when the target detection algorithm finds a target similar to the tracking target information in the panoramic image, it marks and records this target as the tracking target. Then, it uses this target information to start monocular tracking mode for continuous tracking of the target. The object with the highest detection score found by the target detection algorithm can be considered as the similar target.

[0085] In one embodiment, the step of using the target monocular camera to track the target according to a preset method, wherein the preset method includes step S4 of cyclically switching between a monocular tracking mode of a first preset duration and a panoramic shooting mode of a second preset duration, includes:

[0086] S401. Set the current shooting mode of the target monocular camera to monocular tracking mode;

[0087] S402. The target is tracked using the monocular tracking mode, and the first tracking time starting point is recorded.

[0088] S403. When the tracking duration from the first tracking time start point reaches the first preset duration, the monocular tracking mode is switched to the panoramic shooting mode, and the second tracking time start point of switching to the panoramic shooting mode is recorded. The field of view of the panoramic shooting mode is greater than or equal to the field of view of the monocular tracking mode.

[0089] S404. When the tracking duration from the second tracking time start point reaches the second preset duration, switch the panoramic shooting mode to the monocular tracking mode.

[0090] In this embodiment, in step S401 above, after receiving the tracking command or initialization command, the shooting mode of the target monocular camera is configured to monocular tracking mode. In this mode, the camera will automatically adjust parameters such as focal length and exposure to optimize the tracking effect on the target.

[0091] In step S402 above, in monocular tracking mode, the camera activates the tracking algorithm to continuously track the preset target. Simultaneously, the time recording module inside the processor starts timing, recording the precise time point at which tracking begins, as the first tracking time starting point.

[0092] In step S403 above, the tracking duration is continuously monitored and compared with a preset first duration. Once the tracking duration reaches or exceeds the first preset duration, the processor triggers a mode switching command, switching the camera's shooting mode from monocular tracking mode to panoramic shooting mode. Simultaneously, the time point of the switch is recorded as the second tracking time starting point, which is used to subsequently calculate the duration of the panoramic shooting mode. In panoramic shooting mode, the camera's field of view is adjusted to a preset value, which is greater than or equal to the field of view in monocular tracking mode. This allows the camera to capture a wider field of view, providing more environmental context for the tracking task.

[0093] In step S404 above, in panoramic shooting mode, the tracking duration is continuously monitored and compared with a preset second preset duration. Once the duration of panoramic shooting mode reaches or exceeds the second preset duration, a mode switching command is triggered again. Through the corresponding control mechanism, the camera's shooting mode is switched from panoramic shooting mode back to monocular tracking mode, preparing for the next round of precise tracking.

[0094] By cyclically executing the above steps S401 to S404, automatic switching between monocular tracking mode and panoramic shooting mode is achieved. This ensures continuous and accurate tracking of the target while also allowing for broader field of view information to be obtained through panoramic shooting when needed, thereby improving the overall flexibility of tracking.

[0095] In one embodiment, the step S7 of re-identifying the tracking target from the updated panoramic image includes:

[0096] S8. Calculate the speed of the tracked target;

[0097] S9. Adjust the values ​​of the first preset duration and / or the second preset duration according to the preset speed-duration correspondence.

[0098] In this embodiment, in step S8 above, the motion velocity of the tracked target needs to be calculated in real time or periodically during the tracking process. This can be achieved by comparing the positional changes of the tracked target between the current frame and the previous frame or multiple frames. Specifically, the position coordinates (such as the center point coordinates) of the tracked target in each frame can be recorded, and the displacement of the tracked target between two frames can be calculated using these coordinate data. Then, combined with the inter-frame time interval, the motion velocity of the tracked target can be calculated. To calculate the motion velocity more accurately, various methods, such as linear regression and Kalman filtering, can be used to smooth the position data of the tracked target to reduce the influence of noise and errors.

[0099] In step S9 above, a speed-duration correspondence table or function is pre-defined based on the actual application scenario and requirements. This correspondence defines the optimal duration of the monocular tracking mode and panoramic shooting mode at different movement speeds. For example, when the target is moving quickly, the duration of the monocular tracking mode may need to be shortened to allow for more frequent switching to the panoramic shooting mode to capture changes in the target's position; conversely, when the target is moving slowly, the duration of the monocular tracking mode can be extended to achieve a more stable tracking effect. After calculating the target's movement speed, the optimal duration for the corresponding monocular tracking mode and panoramic shooting mode is automatically found based on the pre-defined speed-duration correspondence. Then, these duration values ​​are compared with the currently set first and second pre-defined durations, and these values ​​are adjusted accordingly.

[0100] By dynamically adjusting the duration of monocular tracking mode and panoramic shooting mode based on the target's movement speed, tracking tasks under different motion conditions can be handled more flexibly. This not only improves tracking accuracy and stability but also reduces unnecessary panoramic shooting, thereby increasing overall tracking efficiency.

[0101] In one embodiment, before step S8 of calculating the motion speed of the tracked target, the following is included:

[0102] S10. Obtain the current position of the tracking target in the updated panoramic image;

[0103] S11. Associate the current location with the time point at which the updated panoramic image was acquired and add it to the target's movement path.

[0104] In this embodiment, step S10 above requires identifying and locating the current position of the tracking target after each update of the panoramic image. This can be achieved through image processing techniques, such as target detection, feature extraction, and matching. First, the panoramic image is preprocessed, such as denoising and contrast enhancement, to improve the accuracy of subsequent processing. Then, using preset tracking target features (such as color, shape, and texture) or a trained target detection model, such as a convolutional neural network (CNN) in deep learning or a target detection framework (such as YOLO, SSD, etc.), the tracking target is searched for and located in the panoramic image. Once the target is found, its current position is recorded, for example, it can be represented by pixel coordinates or position offset relative to the image center.

[0105] In step S11 above, after obtaining the current position of the tracked target, the position information is associated with the time point when the position was obtained (i.e., the time point when the panoramic image was updated), and this pair of information is added to the movement path of the tracked target. The movement path is a dynamically updated data structure used to record the movement trajectory of the tracked target over a period of time. It contains a series of time point-position pairs, reflecting the positional changes of the tracked target at different time points.

[0106] By recording the target's current location and time and adding it to the movement path, a complete trajectory of the target's movement can be constructed. This not only helps users intuitively understand the target's movement but also provides support for subsequent adjustments to tracking strategies. Simultaneously, real-time updates and dynamic management of the movement path ensure real-time responses to changes in the target, improving the real-time nature and accuracy of tracking.

[0107] In one embodiment, step S7, which involves re-identifying the tracking target from the updated panoramic image, includes:

[0108] S701. Based on the movement path of the tracking target, a preset tracking algorithm is used to estimate the position of the tracking target to obtain the estimated position area;

[0109] S702, Re-identify and track the target starting from the estimated location area of ​​the updated panoramic image.

[0110] In this embodiment, in step S701 above, each time it is necessary to re-identify the tracking target from the updated panoramic image, the movement path of the tracking target is first referenced. The movement path records the position changes and timestamps of the tracking target over a period of time, providing an important basis for predicting its current or future position. Based on this historical position data, a preset tracking algorithm (such as Kalman filtering, particle filtering, etc.) is used to estimate the position of the tracking target, taking into account the motion model of the tracking target (such as uniform motion, accelerated motion, etc.), thereby obtaining a relatively accurate estimated position area.

[0111] The selection and implementation of the preset tracking algorithm depends on the specific application scenario and performance requirements. For example, Kalman filtering is suitable for linear systems and Gaussian noise environments, while particle filtering can handle more complex nonlinear systems and non-Gaussian noise. During implementation, the algorithm parameters need to be adjusted according to the actual situation to obtain the best prediction results.

[0112] In one specific embodiment, step S701, which involves estimating the location of the target based on its movement path using a preset tracking algorithm to obtain the estimated location area, includes:

[0113] S701a. Extract the N closest time points and their corresponding location information from the tracking target's movement path, where N is a preset positive integer;

[0114] S701b: Taking the N time points and their corresponding location information as input, the preset tracking algorithm is used to estimate the position of the tracking target at the current time point or a preset future time point, and the coordinate value of the estimated position is obtained.

[0115] S701c. Using the coordinates of the estimated position as the center, determine the estimated position area according to the preset search range radius.

[0116] In this embodiment, the estimated location area can be a circular region. The size of the preset search range radius is adaptively adjusted according to the movement speed of the tracked target, the image resolution, and the performance of the tracking algorithm.

[0117] In step S702 above, after obtaining the estimated location area, this area is used as the search starting point to re-identify the tracking target in the updated panoramic image. Specifically, a search window (whose size and shape can be adjusted according to the actual situation) is defined with the center of the estimated location area as the reference point, and a target detection or recognition algorithm is applied within this window to search for the tracking target. Since the search range is limited to the vicinity of the estimated location area, the search efficiency and accuracy can be significantly improved.

[0118] Reference Figure 2This invention also provides a panoramic target search and monocular linkage tracking device, comprising:

[0119] Module 10 is used to acquire tracking target information;

[0120] The panoramic search module 20 is used to identify the tracking target in the panoramic image through a target detection algorithm. The panoramic image is stitched together from images captured by multiple monocular cameras at the same time in panoramic shooting mode.

[0121] The camera determination module 30 is used to determine the target monocular camera corresponding to the tracking target from the panoramic image;

[0122] Monocular tracking module 40 is used to track the target using the target monocular camera according to a preset method, the preset method including cyclic switching between a monocular tracking mode of a first preset duration and a panoramic shooting mode of a second preset duration;

[0123] The panoramic image generation module 50 is used to re-synthesize an updated panoramic image when the target monocular camera switches to the panoramic shooting mode. The updated panoramic image is stitched together from the latest images captured by multiple monocular cameras at the same moment in the panoramic shooting mode.

[0124] The judgment module 60 is used to determine whether the tracking target has been lost from the field of view of the target monocular camera in the monocular tracking mode;

[0125] The re-identification module 70 is used to re-identify the tracking target from the updated panoramic image if it is lost.

[0126] For the specific implementation of each module in the above device embodiment, please refer to the above method embodiment, and it will not be repeated here.

[0127] In one specific embodiment, the monocular tracking module 40 includes:

[0128] The image acquisition unit is used to acquire images captured by each of the monocular cameras at the same time in a preset panoramic shooting mode;

[0129] The image stitching unit is used to stitch together the images captured by each of the monocular cameras at the same time using a preset image stitching algorithm to generate the panoramic image;

[0130] The target search unit is used to traverse the panoramic image using the target detection algorithm to search for whether there is a target in the panoramic image that is similar to the tracking target information;

[0131] The tracking target determination unit is used to identify a target similar to the tracking target information as the tracking target if such a target exists.

[0132] For the specific implementation of each module in the above device embodiment, please refer to the above method embodiment, and it will not be repeated here.

[0133] Reference Figure 3 This invention also provides a computer device, which can be a server, and its internal structure can be as follows: Figure 3 As shown, the computer device includes a processor, memory, display screen, input device, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and the database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database stores the data corresponding to this embodiment. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements the aforementioned panoramic target search and monocular linkage tracking methods.

[0134] Those skilled in the art will understand that Figure 3 The structures shown are merely block diagrams of some structures related to the present invention and do not constitute a limitation on the computer devices on which the present invention is applied.

[0135] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method. It is understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0136] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the present invention and embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM, etc.

[0137] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0138] The above description is only a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural or procedural transformations made based on the content of the present invention specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for panoramic target search and monocular linkage tracking, characterized in that, include: Obtain information about the target being tracked; The target is identified and tracked in the panoramic image by a target detection algorithm. The panoramic image is stitched together from images captured by multiple monocular cameras at the same time in panoramic shooting mode. Identify the target monocular camera corresponding to the tracking target from the panoramic image; The target is tracked using the target monocular camera according to a preset method, the preset method including cyclic switching between a monocular tracking mode of a first preset duration and a panoramic shooting mode of a second preset duration, the cyclic switching step including: Set the current shooting mode of the target monocular camera to monocular tracking mode; The target is tracked using the monocular tracking mode, and the first tracking time starting point is recorded. When the tracking duration from the first tracking time start point reaches the first preset duration, the monocular tracking mode is switched to the panoramic shooting mode, and the second tracking time start point of switching to the panoramic shooting mode is recorded. The field of view of the panoramic shooting mode is greater than or equal to the field of view of the monocular tracking mode. When the tracking duration from the second tracking time start point reaches the second preset duration, the panoramic shooting mode is switched to the monocular tracking mode; When the target monocular camera switches to the panoramic shooting mode, it re-synthesizes an updated panoramic image. The updated panoramic image is stitched together from the latest images captured by multiple monocular cameras at the same moment in the panoramic shooting mode. Determine whether the tracked target has been lost from the field of view of the target monocular camera in the monocular tracking mode; If so, the tracking target is re-identified from the updated panoramic image, including: Calculate the speed of the tracked target; Adjust the values ​​of the first preset duration and / or the second preset duration according to the preset speed-duration correspondence; Based on the movement path of the tracking target, a preset tracking algorithm is used to estimate the location of the tracking target to obtain the estimated location area; The target is re-identified and tracked starting from the estimated location region of the updated panoramic image.

2. The panoramic target search and monocular linkage tracking method according to claim 1, characterized in that, The step of identifying and tracking targets in panoramic images using a target detection algorithm, wherein the panoramic images are stitched together from images captured simultaneously by multiple monocular cameras in panoramic shooting mode, includes: Images captured simultaneously by each of the monocular cameras in a preset panoramic shooting mode; The images captured simultaneously by each of the monocular cameras are stitched together using a preset image stitching algorithm to generate the panoramic image; The target detection algorithm is used to traverse the panoramic image to search for whether there is a target in the panoramic image that is similar to the tracking target information; If so, then a target similar to the tracking target information will be used as the tracking target.

3. The panoramic target search and monocular linkage tracking method according to claim 1, characterized in that, Before the step of calculating the speed of the tracked target, the following steps are included: Obtain the current position of the tracked target in the updated panoramic image; The current location and the time point at which the updated panoramic image was acquired are associated and added to the tracking target's movement path.

4. The panoramic target search and monocular linkage tracking method according to claim 1, characterized in that, The step of estimating the location of the tracked target based on its movement path using a preset tracking algorithm to obtain the estimated location area includes: Extract the N closest time points and their corresponding location information from the tracking target's movement path, where N is a preset positive integer; Using the N time points and their corresponding location information as input, the preset tracking algorithm is used to estimate the location of the tracking target at the current time point or a preset future time point, and the coordinate value of the estimated location is obtained. Centered on the coordinates of the estimated location, the estimated location area is determined according to the preset search range radius.

5. A panoramic target search and monocular linkage tracking device, characterized in that, include: The acquisition module is used to acquire information about the tracking target. The panoramic search module is used to identify and track targets in panoramic images using a target detection algorithm. The panoramic images are stitched together from images captured by multiple monocular cameras at the same time in panoramic shooting mode. A camera determination module is used to determine, from the panoramic image, the target monocular camera corresponding to the tracking target; A monocular tracking module is used to track the target using the target monocular camera according to a preset method. The preset method includes a cyclic switching between a monocular tracking mode of a first preset duration and a panoramic shooting mode of a second preset duration. The monocular tracking module includes: The shooting setting submodule is used to set the current shooting mode of the target monocular camera to monocular tracking mode; The target tracking submodule is used to track the target using the monocular tracking mode and record the first tracking time starting point. The first mode switching submodule is used to switch the monocular tracking mode to the panoramic shooting mode when the tracking duration from the first tracking time start point reaches the first preset duration, and to record the second tracking time start point of switching to the panoramic shooting mode. The field of view of the panoramic shooting mode is greater than or equal to the field of view of the monocular tracking mode. The second mode switching submodule is used to switch the panoramic shooting mode to the monocular tracking mode when the tracking duration from the second tracking time start point reaches the second preset duration. A panoramic image generation module is used to re-synthesize an updated panoramic image when the target monocular camera switches to the panoramic shooting mode. The updated panoramic image is stitched together from the latest images captured by multiple monocular cameras at the same moment in the panoramic shooting mode. The judgment module is used to determine whether the tracked target has been lost from the field of view of the target monocular camera in the monocular tracking mode; A re-identification module, configured to, if so, re-identify the tracking target from the updated panoramic image, including: The speed calculation submodule is used to calculate the movement speed of the tracked target; The duration adjustment submodule is used to adjust the values ​​of the first preset duration and / or the second preset duration according to a preset speed-duration correspondence. The target location estimation submodule is used to estimate the location of the tracked target based on the target's movement path using a preset tracking algorithm, and obtain the estimated location area. The target re-tracking submodule is used to re-identify and track the target starting from the estimated location region of the updated panoramic image.

6. The panoramic target search and monocular linkage tracking device according to claim 5, characterized in that, The panoramic search module includes: The image acquisition unit is used to acquire images captured by each of the monocular cameras at the same time in a preset panoramic shooting mode; The image stitching unit is used to stitch together the images captured by each of the monocular cameras at the same time using a preset image stitching algorithm to generate the panoramic image; The target search unit is used to traverse the panoramic image using the target detection algorithm to search for whether there is a target in the panoramic image that is similar to the tracking target information; The tracking target determination unit is used to identify a target similar to the tracking target information as the tracking target if such a target exists.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the panoramic target search and monocular linkage tracking method as described in any one of claims 1 to 4.

Images