An autonomously switching dual-field-of-view camera and target detection and tracking method

Abstract

The present invention discloses an autonomously switching dual-field-of-view camera and a target detection and tracking method in the field of target detection technology. The autonomously switching dual-field-of-view camera includes a dual-field-of-view camera body and a turntable arranged at the bottom of the dual-field-of-view camera body to adjust the viewing angle of the dual-field-of-view camera body. A fill light board, a focus panel, a signal board and an interface board connected in sequence through copper columns are arranged inside the dual-field-of-view camera body. A target detection model is deployed inside the signal board, and the signal board integrates a target search module and a target detection and tracking module. The target search module is designed with different search strategies for wide and narrow fields of view based on the rotation angle limit of the turntable. It can intelligently adjust and switch to the most appropriate field of view according to the changes in the actual scene and the needs of target detection, so as to achieve a seamless transition from wide-angle monitoring to fine observation. In addition, it can efficiently identify and track targets, and maintain a high degree of accuracy and robustness even in complex and changing environments.

Description

Technical Field

[0001] The present invention relates to the technical field of target detection, and in particular to an autonomously switching dual-field-of-view camera and a target detection and tracking method. Background Art

[0002] With the widespread application of computer vision technology in target detection, security monitoring, autonomous driving, industrial automation and other fields, higher requirements are placed on the performance of camera systems. Traditional camera systems are limited by a fixed field of view and are difficult to adapt to complex and changing application scenarios, especially in situations where large-scale monitoring and specific detail observation are required at the same time. Single-field-of-view cameras lack an intelligent autonomous adjustment mechanism and cannot automatically switch the field of view according to environmental changes or target detection requirements, which limits the efficiency and accuracy of target detection and makes it difficult to meet the needs of modern intelligent systems.

[0003] Existing dual-field-of-view camera systems attempt to solve this problem through mechanical switching or electronic switching, but both have significant shortcomings. Although the mechanical switching method can achieve field-of-view switching, it is limited by the physical limitations of mechanical components, the switching speed is slow and the mechanical life is limited, and long-term use can easily lead to performance degradation. Although the electronic switching method has a faster switching speed, due to its essential image processing differences, it may cause significant changes in image quality between different fields of view and a lack of field-of-view continuity, affecting user experience and the stability of target detection. Summary of the invention

[0004] The purpose of this section is to summarize some aspects of the embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0005] Therefore, the purpose of the present invention is to provide an autonomously switched dual-field camera and target detection and tracking method, which can intelligently adjust and switch to the most appropriate field of view according to the changes in the actual scene and the needs of target detection, and achieve a seamless transition from wide-angle monitoring to fine observation. In addition, it can efficiently identify and track targets, and maintain a high degree of accuracy and robustness even in complex and changing environments.

[0006] To solve the above technical problems, according to one aspect of the present invention, the present invention provides the following technical solutions:

[0007] An autonomously switching dual-field-of-view camera comprises: a dual-field-of-view camera body and a turntable arranged at the bottom of the dual-field-of-view camera body to adjust the viewing angle of the dual-field-of-view camera body;

[0008] The dual-field camera body is provided with a fill light board, a focus board, a signal board and an interface board which are connected in sequence through copper pillars;

[0009] A target detection model is deployed inside the signal board, and the signal board integrates a target search module and a target detection and tracking module;

[0010] The target search module is designed with different search strategies for wide field of view and narrow field of view based on the rotation angle limit of the turntable;

[0011] The target detection and tracking module is used to capture video data from the VPSS of the signal board, perform color space conversion and preprocessing, use the target detection model for forward reasoning, output the detection result, and judge whether the target appears stably according to the number of target frames in the detection result, and then decide whether to enter the target detection and tracking stage from the target search;

[0012] The target detection and tracking module also includes a field of view switching evaluation mechanism and a turntable control mechanism. The field of view switching evaluation mechanism decides whether to switch between a wide and narrow field of view by comparing the target frame size with a preset resolution threshold. The turntable control mechanism automatically adjusts the turntable angle according to the latest target position information provided by the target detection and tracking module to ensure that the target object remains in the center area of ​​the field of view.

[0013] As a preferred solution of the autonomously switching dual-field-of-view camera described in the present invention, the different search strategies for the wide field of view and the narrow field of view are specifically:

[0014] Wide field search strategy: Use an S-shaped search path, set the horizontal step length to 15°, and the elevation step length to 10° for scanning;

[0015] Narrow field of view search strategy: Scanning is performed in a continuous spiral trajectory with a fine step size of 10° in both horizontal and elevation directions.

[0016] As a preferred solution of the autonomously switching dual-field-of-view camera described in the present invention, the target detection model is trained based on the YOLOV7 algorithm and is used to identify and locate the target object in the image.

[0017] As a preferred solution of the autonomously switching dual-field-of-view camera described in the present invention, the fill light board has four high-brightness LED fill light modules built in.

[0018] As a preferred solution of the autonomously switching dual-field-of-view camera described in the present invention, the focus panel integrates two Sony IMX415 CMOS sensors and a focus panel power supply.

[0019] As a preferred solution of the autonomously switching dual-field-of-view camera described in the present invention, the interface board includes a coprocessor, a WIFI module, a charging circuit and a dual-frequency antenna.

[0020] As a preferred solution of the autonomously switching dual-field camera described in the present invention, the target detection model is specifically trained based on the YOLOV7 algorithm as follows:

[0021] Dataset construction: Based on the requirements of target recognition, detection, and tracking, a high-quality sample dataset is constructed and annotated. The dataset covers images of the target in different scenes, lighting conditions, and viewing angles.

[0022] Model training: The YOLOV7 target detection algorithm is used to perform in-depth training on the sample data set constructed above. An efficient target detection model is formed by adjusting the network structure, optimizing hyperparameters and data enhancement.

[0023] A target detection and tracking method for an autonomously switched dual-field camera, the specific steps are as follows:

[0024] S1, the dual-field camera body searches for the target based on the wide-field search strategy of the target search module, obtains the searched video image in real time and transmits it to the signal board;

[0025] S2, the target detection and tracking module obtains a frame of data from the video processing subsystem VPSS of the signal board in real time, and converts the format of the image output by the video processing subsystem VPSS through the HiSilicon color space conversion algorithm;

[0026] S3, preprocessing the converted RGB image, then loading it into the target detection model for processing, outputting the detection result containing the high-confidence candidate box, and then releasing the current frame data;

[0027] S4, repeat the above steps to continuously capture and process N frames of images. If the number of target frames detected in the N frames of images exceeds a preset threshold, it is considered that the target has stably appeared, and the search phase is stopped immediately, and the target detection and tracking phase is entered;

[0028] S5. Compare the size of the target frame in each frame image with the preset resolution threshold through the field of view switching evaluation mechanism, count the number of frames that meet the conditions for switching to the narrow field of view in the continuous frames, and calculate the average value of the center coordinates of the target frame. When the statistical result meets the preset conditions, the switching operation of the dual-field-of-view camera body from the wide field of view to the narrow field of view is automatically triggered based on the target search module;

[0029] S6. If the current conditions do not meet the criteria for switching the field of view, the turntable control mechanism automatically adjusts the turntable angle according to the latest target position information provided by the target detection and tracking module to ensure that the target object remains in the center area of ​​the field of view.

[0030] Compared with the prior art, the present invention has the following beneficial effects: the present invention integrates two lenses with different viewing angles in the same camera structure, and through the independently developed fast and seamless switching mechanism, it can intelligently adjust and switch to the most appropriate field of view according to the changes in the actual scene and the needs of target detection, thereby achieving a seamless transition from wide-angle monitoring to fine observation. In addition, combined with advanced computer vision technology, especially target detection and tracking algorithms, it can efficiently identify and track targets, and maintain a high degree of accuracy and robustness even in complex and changing environments, which will greatly improve the adaptability and performance of the camera system in complex application environments, and has broad application prospects and important technical value. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be described in detail below in combination with the accompanying drawings and detailed embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0032] Figure 1 A schematic diagram of the camera hardware structure assembly provided by the present invention;

[0033] Figure 2 A schematic diagram of the camera hardware structure circuit design provided by the present invention;

[0034] Figure 3 A schematic diagram of a wide-field search curve in an embodiment of the present invention;

[0035] Figure 4 A schematic diagram of a narrow field of view search curve in an embodiment of the present invention;

[0036] Figure 5 This is an overall flow chart of the target detection and tracking module in an embodiment of the present invention;

[0037] Figure 6 Schematic diagram of a wide-field virtual nine-square grid partition in an embodiment of the present invention. DETAILED DESCRIPTION

[0038] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0039] like Figure 1 As shown, the present invention provides an autonomously switching dual-field-of-view camera, which includes: a dual-field-of-view camera body 100 and a turntable 200 arranged at the bottom of the dual-field-of-view camera body 100 to adjust the viewing angle of the dual-field-of-view camera body 100.

[0040] like Figure 2 As shown, the dual-field-of-view camera body 100 is internally provided with a fill light board 110, a focus panel 120, a signal board 130 and an interface board 140 which are connected in sequence through copper columns. Each module is fixed by precision-machined copper columns to ensure a stable structure while forming a compact cube structure. The outside is covered with an iron box to enhance the protection performance.

[0041] The fill light board 110 has four built-in high-brightness LED fill light modules. In this embodiment, the model is HL-798H256WC-MD, which is specially designed for low-light environments, effectively improving image brightness and contrast, and ensuring that clear images can still be obtained under low-light conditions.

[0042] The focus panel 120 is the core imaging module, integrating two Sony IMX415 CMOS sensors and a focus panel power supply. The sensor is known for its excellent image quality and maximum resolution of 3840×2160. Through windowing technology, the system can flexibly adjust the sensor resolution to 2560×1440 to meet the different needs of wide field of view and narrow field of view.

[0043] Signal board 130 is the core control and processing unit, equipped with Hi3516DV300 high-performance processor, and SPIFlash (W25Q128, 128Mbits capacity, SFC protocol interface) to store system code to ensure smooth system operation. The system crystal oscillator uses a 24MHz passive crystal oscillator to provide a stable clock signal for the system.

[0044] The interface board 140 integrates a wealth of communication and expansion functions, including a coprocessor (ATSAMR30E18A-MU, 3.3V power supply, internal analog voltage 1.8V), a WIFI module (RTL8188EUS chip, 3.3V power supply, supporting 900MHz and 2.4GHz dual-band communication), a charging circuit, and a dual-band antenna. Through GPIO control, the coprocessor can flexibly manage WIFI connections and antenna switching to meet communication needs in different scenarios.

[0045] The signal board 130 has a target detection model disposed therein, and the signal board 130 integrates a target search module and a target detection and tracking module.

[0046] The target detection model is trained based on the YOLOV7 algorithm and is used to identify and locate target objects in images. The specific training steps of the target detection model in this embodiment are as follows:

[0047] Dataset construction: Based on the requirements of target recognition, detection, and tracking, a high-quality sample dataset is constructed and annotated. The dataset covers images of the target in different scenes, lighting conditions, and viewing angles.

[0048] Model training: The YOLOV7 target detection algorithm is used to perform in-depth training on the sample data set constructed above. An efficient target detection model is formed by adjusting the network structure, optimizing hyperparameters and data enhancement.

[0049] The target search module is designed with different search strategies for wide and narrow fields of view based on the rotation angle limit of the turntable 200 (in this embodiment, the turntable angle limit is: horizontal angle ±170°, upward tilt +85°, downward tilt -25°). According to the different requirements of wide and narrow fields of view, an efficient search route map is customized to ensure comprehensive coverage of the search area.

[0050] Figure 3 The wide-field search curve diagram proposed for this embodiment aims to quickly locate the target. By adopting an optimized S-shaped search path, setting the horizontal step length to 15° and the pitch step length to 10°, this strategy effectively covers a wide search area and significantly improves the search efficiency.

[0051] Figure 4 This is a schematic diagram of the narrow field of view search curve in this embodiment. The search method uses a fine step of 10° in both the horizontal and pitch directions. Through the scanning of continuous spiral trajectories, it not only achieves a smooth transition of the search mode, but also ensures the accuracy and efficiency of target locking.

[0052] The target detection and tracking module is used to capture video data from the VPSS of the signal board 130, perform color space conversion and preprocessing, use the target detection model for forward reasoning, output the detection results, and determine whether the target appears stably based on the number of target boxes in the detection results, and then decide whether to enter the target detection and tracking stage from the target search.

[0053] The target detection and tracking module also includes a field of view switching evaluation mechanism and a turntable control mechanism. The field of view switching evaluation mechanism determines whether to switch between a wide and narrow field of view by comparing the target frame size with a preset resolution threshold. The turntable control mechanism automatically adjusts the turntable 200 degrees according to the latest target position information provided by the target detection and tracking module to ensure that the target object remains in the center of the field of view.

[0054] Specifically, Figure 5 As shown in the figure, the specific detection and tracking process of the target detection and tracking module is as follows:

[0055] The dual-field-of-view camera body 100 searches for the target based on the wide-field-of-view search strategy of the target search module, obtains the searched video image in real time and transmits it to the signal board 130 .

[0056] Image data acquisition: The target detection and tracking module acquires a frame of data from the video processing subsystem VPSS of the signal board 130 in real time.

[0057] Color space conversion: The YUV format image output by the video processing subsystem VPSS is converted into RGB888 format through the HiSilicon color space conversion algorithm.

[0058] Image preprocessing and loading: Perform preprocessing operations (such as resizing, confidence setting, etc.) on the converted RGB image, and then load it into the detection model for processing.

[0059] Forward reasoning: Use HiSilicon's high-performance neural network inference engine NNIE (Neural Network Inference Engine) to perform forward reasoning calculations. After the reasoning calculations are completed, the detection results are obtained. The detection results of each frame of the image will contain multiple candidate boxes.

[0060] Get the detection results and output the target box: The detection results output multiple bounding boxes and their confidence scores, and arrange them in descending order by score. Select the box with the highest confidence as the initial detection result. Calculate the intersection of union (IoU) between the remaining boxes and the selected box. If the IoU exceeds the set threshold (0.65 in this embodiment), compare the confidence and retain the one with the higher score. Repeat the above steps until all boxes are processed, and the processed box is the final target box. At the same time, calculate the width (W), height (H) and center point coordinates (Xn, Yn) of the target box.

[0061] Image resource release: Release the currently processed image frame to optimize system resource utilization and reduce memory usage.

[0062] Continuity detection and stability assessment: Repeat the above steps to continuously obtain N frames of images from VPSS (6 in this embodiment), count the total number of target frames in this sequence, and if the number exceeds the preset threshold (4 in this embodiment), it is confirmed that the target exists stably in the continuous frames, and the search process is terminated and target detection and tracking is started. If the threshold is not reached, repeat the above steps until the condition is met.

[0063] Target tracking and field of view switching evaluation: Given that the camera wide field of view resolution is 3840×2160 pixels, this embodiment sets a resolution threshold (1280x720 pixels) as the basis for wide and narrow field of view switching. This threshold is based on the proportional division of the field of view (in this embodiment, the theoretical 9 equal divisions are used, each approximately 1280×720 pixels), not the actual physical division. During the target tracking process, the target frame size is calculated frame by frame. If its width and height do not exceed the threshold, it is considered that the target occupies a smaller area of ​​the field of view, and it may be necessary to switch to a narrow field of view to improve image clarity and tracking accuracy. The variable Qn is introduced to record the number of frames that meet the narrow field of view switching conditions in the continuous frames (if satisfied, it is set to 1), and the average value of the target frame center coordinates in these N frames is calculated (X_average = (X1 + X2+… Xn) / N, Y_average = (Y1 + Y2+… Yn) / N) to support subsequent tracking operations.

[0064] Switching condition Q statistics: Calculate the variable Q, which is the sum of the Qn sequence (Q1 to Qn). This value reflects the total number of frames in which the target frame size meets the narrow field of view switching condition in consecutive frames. Figure 6 The figure shows a schematic diagram of a virtual wide-field nine-square grid partition provided by an embodiment of the present invention. In this embodiment, the wide-field is virtually divided into a 3×3 nine-square grid layout, with the center being the fifth area for subsequent analysis.

[0065] Wide and narrow field of view switching decision: If the Q value reaches or exceeds the preset threshold (4 in this embodiment), and the average value of the target frame center coordinates is located in the center area of ​​the nine-square grid (area 5), ​​the wide field of view to narrow field of view switching mechanism is triggered. This decision is based on the observation that the target size is continuously small and located in the center of the field of view in multiple consecutive frames, indicating that more detailed observation is required.

[0066] Field of view and turntable adjustment: If the Q value and the average value of the target frame center coordinates do not meet the switching conditions, the system automatically adjusts the turntable angle according to the specific area of ​​the target in the nine-square grid to reposition the target in the center of the field of view. This step is to optimize the observation and detection performance during target tracking and ensure that the target is always in the best observation position.

[0067] This embodiment achieves accurate detection and tracking of the target by comprehensively considering the target size, position information, and stability in continuous frames. The intelligent decision-making mechanism automatically determines whether the field of view needs to be switched based on the above parameters, and adjusts the turntable in time to ensure that the target is always in the best observation state. This strategy not only improves the accuracy of target tracking and recognition, but also optimizes resource utilization efficiency.

[0068] Although the present invention has been described above with reference to the embodiments, various modifications may be made thereto and parts thereof may be replaced by equivalents without departing from the scope of the present invention. In particular, as long as there is no structural conflict, the various features in the embodiments disclosed in the present invention may be used in combination with each other in any manner, and the fact that these combinations are not exhaustively described in this specification is only for the sake of omitting space and saving resources. Therefore, the present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. An autonomously switching dual-field camera, characterized in that: include: A dual-viewing field camera body (100) and a turntable (200) arranged at the bottom of the dual-viewing field camera body (100) for adjusting the viewing angle of the dual-viewing field camera body (100); The dual-viewing field camera body (100) is provided with a fill light board (110), a focus board (120), a signal board (130) and an interface board (140) which are sequentially connected via copper pillars. A target detection model is deployed inside the signal board (130), and the signal board (130) is integrated with a target search module and a target detection and tracking module; The target search module is designed with different search strategies for wide and narrow fields of view based on the rotation angle limit of the turntable (200); The target detection and tracking module is used to obtain a frame of data from the video processing subsystem VPSS of the signal board (130) in real time, convert the format of the image output by the video processing subsystem VPSS through the HiSilicon color space conversion algorithm, perform preprocessing operations on the converted RGB image, and then load it into the target detection model for processing, output the detection result containing the high-confidence candidate frame, and then release the current frame data, repeat the above steps to continuously capture and process N frames of images, if the number of target frames detected in the N frames of images exceeds a preset threshold, it is considered that the target has stably appeared, and the search phase is immediately stopped, and the target detection and tracking phase is entered; The target detection and tracking module also includes a field of view switching evaluation mechanism and a turntable control mechanism. The field of view switching evaluation mechanism determines whether to switch between a wide and narrow field of view by comparing the target frame size with a preset resolution threshold. The turntable control mechanism automatically adjusts the turntable (200) angle according to the latest target position information provided by the target detection and tracking module to ensure that the target object remains in the central area of ​​the field of view.

2. The autonomously switching dual-field camera according to claim 1, characterized in that: The different search strategies for the wide field of view and the narrow field of view are specifically: Wide field search strategy: Use an S-shaped search path, set the horizontal step length to 15°, and the elevation step length to 10° for scanning; Narrow field of view search strategy: Scanning is performed in a continuous spiral trajectory with a fine step size of 10° in both horizontal and elevation directions.

3. The autonomously switching dual-field camera according to claim 1, characterized in that: The target detection model is trained based on the YOLOV7 algorithm and is used to identify and locate target objects in images.

4. The autonomously switching dual-field camera according to claim 1, characterized in that: The fill light panel (110) has four high-brightness LED fill light modules built in.

5. The autonomously switching dual-field camera according to claim 1, characterized in that: The focal panel (120) integrates two Sony IMX415 CMOS sensors and a focal panel power supply.

6. The autonomously switching dual-field camera according to claim 1, characterized in that: The interface board (140) comprises a coprocessor, a WIFI module, a charging circuit and a dual-frequency antenna.

7. The autonomously switching dual-field camera according to claim 3, characterized in that: The target detection model is trained based on the YOLOV7 algorithm as follows: Dataset construction: Based on the requirements of target recognition, detection and tracking, a high-quality sample dataset is constructed and annotated. The dataset covers images of the target in different scenes, lighting conditions and viewing angles. Model training: The YOLOV7 target detection algorithm is used to perform in-depth training on the sample data set constructed above. An efficient target detection model is formed by adjusting the network structure, optimizing hyperparameters and data enhancement.

8. A target detection and tracking method for an autonomously switching dual-field camera as claimed in claim 1, characterized in that: The specific steps are as follows: S1, the dual-field-of-view camera body (100) searches for a target based on a wide-field-of-view search strategy of a target search module, acquires the searched video image in real time and transmits it to a signal board (130); S2, the target detection and tracking module obtains a frame of data from the video processing subsystem VPSS of the signal board (130) in real time, and converts the format of the image output by the video processing subsystem VPSS through the HiSilicon color space conversion algorithm; S3, preprocessing the converted RGB image, then loading it into the target detection model for processing, outputting the detection result containing the high-confidence candidate box, and then releasing the current frame data; S4, repeat the above steps to continuously capture and process N frames of images. If the number of target frames detected in the N frames of images exceeds a preset threshold, it is considered that the target has stably appeared, and the search phase is stopped immediately, and the target detection and tracking phase is entered; S5. By using a field of view switching evaluation mechanism, the size of the target frame in each frame image is compared with a preset resolution threshold, the number of frames in the continuous frames that meet the conditions for switching to the narrow field of view is counted, and the average value of the center coordinates of the target frame is calculated. When the statistical result meets the preset conditions, the target search module automatically triggers the switching operation of the dual-field of view camera body (100) from the wide field of view to the narrow field of view; S6. If the current conditions do not meet the judgment criteria for field of view switching, the turntable control mechanism automatically adjusts the turntable (200) angle according to the latest target position information provided by the target detection and tracking module to ensure that the target object remains in the central area of ​​the field of view.

Images