Holder searching and tracking method

Abstract

The invention discloses a pan-tilt searching and tracking method, and particularly belongs to pan-tilt searching and tracking methods. According to the method, images are collected in real time through a camera; the control module identifies the dial plate in the preprocessed image, responds to the dial plate contained in the preprocessed image, and determines the spatial position of the dial plate in the image coordinate system; in response to the situation that the preprocessed image does not contain the dial plate, identifying a fixed object near the dial plate for auxiliary positioning, and determining the spatial position of the dial plate in the image coordinate system; the control module determines a control signal based on the spatial position of the dial plate in the image coordinate system; and based on the control signal, the driving part is controlled to adjust the lifting height, the yaw angle and the pitching angle of the holder, the focal length of the camera and the tracking path of the inspection robot so as to track the dial plate. According to the method, the offset of the preset pose of the holder can be accurately adjusted, so that the camera is aligned with the dial plate to ensure accurate reading, and the overall inspection efficiency is improved.

Description

Technical Field

[0001] This application relates to the field of gimbal control technology, and in particular to a gimbal search and tracking method. Background Technology

[0002] Traditional industrial inspections rely heavily on manual labor, resulting in high labor intensity and low efficiency, particularly in delicate tasks such as automated dashboard reading. With technological advancements, explosion-proof wheeled inspection robots effectively address these pain points through the integration of multiple technologies. Equipped with gimbal monitoring equipment, these robots reduce the risks of high-risk operations, improve the efficiency of digital maintenance, and ultimately form a closed-loop inspection capability in specialized scenarios, encompassing "environmental perception - defect diagnosis - risk warning," providing a reliable supplement or alternative to manual inspections.

[0003] In related technologies, there are significant limitations in dial recognition. Current systems only support the recognition of conventional targets such as people, vehicles, and digital dials, which cannot meet the precise recognition requirements for instrument dials and special equipment in industrial scenarios. When the inspection robot's navigation and positioning accuracy decreases, the preset pose of the gimbal will shift, resulting in inaccurate alignment with the dial, leading to reading failures and directly affecting overall inspection efficiency.

[0004] Therefore, there is an urgent need to develop a gimbal search and tracking method to solve the above problems. Summary of the Invention

[0005] In view of this, this application provides a gimbal search and tracking method that can accurately adjust the offset of the preset pose of the gimbal when the navigation and positioning accuracy of the inspection robot decreases, so that the camera can be accurately aligned with the dial, thereby ensuring accurate readings and improving the overall inspection efficiency.

[0006] Specifically, the following technical solutions are included:

[0007] This application provides a gimbal search and tracking method applied to a dial tracking system. The dial tracking system includes an inspection robot, a drive unit mounted on the inspection robot, a gimbal mounted on the drive unit, a camera mounted on the gimbal, and a control module. The method includes:

[0008] The camera captures images in real time;

[0009] The control module identifies the dial in the preprocessed image. The dial is located using two modes: direct positioning and indirect positioning. In direct positioning, the dial is located in the image coordinate system if the preprocessed image contains the dial. In indirect positioning, the dial is located in the image coordinate system if the preprocessed image does not contain the dial. Fixed objects near the dial are identified to assist in the positioning and determine the dial's spatial position in the image coordinate system.

[0010] The control module determines the control signal based on the spatial position of the dial in the image coordinate system; based on the control signal, it controls the drive components to adjust the gimbal's lifting height, yaw angle, and pitch angle, the camera's focal length, and the inspection robot's tracking path to track the dial.

[0011] In some embodiments, preprocessing includes scaling, filtering, and pixel adjustment of the image.

[0012] In some embodiments, the control module carries a YOLOv8 model, which is used to identify and mark the dial in the preprocessed image, and determine the spatial position of the dial in the image coordinate system.

[0013] In some embodiments, the control module carries a YOLOv8 model, which is used to identify and mark fixed objects in the preprocessed image for auxiliary positioning, thereby determining the spatial position of the dial in the image coordinate system.

[0014] In some embodiments, the fixed object is an object that is fixedly located near the dial.

[0015] In some embodiments, identifying a fixed object near the dial for assisted localization to determine the spatial position of the dial in the image coordinate system includes:

[0016] Identify fixed objects near the dial and determine their positions.

[0017] Based on the relative position of the dial and a fixed object near the dial, the gimbal's lifting height, yaw angle, pitch angle, and camera focal length adjustment direction are determined until the dial is included in the image. The relative position of the dial and a fixed object near the dial has been input into the Yolov8 model.

[0018] Determine the spatial position of the dial in the image coordinate system to complete the auxiliary positioning.

[0019] In some embodiments, before the control module identifies the dial in the preprocessed image, the method further includes:

[0020] The control module controls the inspection robot to reach the preset position to identify the dial, and adjusts the gimbal's lifting height, yaw angle, pitch angle, and camera focal length to the initial pose.

[0021] In some embodiments, the method further includes a precise positioning process:

[0022] The control module responds to the dial not being centered on the image and determines the fine-tuning signal;

[0023] Based on the fine-tuning signal, the control drive adjusts the gimbal's lifting height, yaw angle, and pitch angle, the camera's focal length, and the inspection robot's tracking path.

[0024] The beneficial effects of the technical solutions provided in this application include at least the following:

[0025] This application provides a gimbal search and tracking method. The method involves real-time image acquisition via a camera; a control module identifies a dial in the pre-processed image, where the dial is located using two modes: direct and indirect positioning. Direct positioning determines the dial's spatial position in the image coordinate system if the pre-processed image contains the dial; indirect positioning determines the dial's spatial position in the image coordinate system if the pre-processed image does not contain the dial, by identifying a fixed object near the dial for auxiliary positioning. Based on the dial's spatial position in the image coordinate system, the control module determines a control signal. Based on the control signal, the drive mechanism adjusts the gimbal's height, yaw and pitch angles, the camera's focal length, and the inspection robot's tracking path to track the dial. This method can accurately adjust the gimbal's preset pose offset when the inspection robot's navigation and positioning accuracy decreases, ensuring the camera accurately aligns with the dial, guaranteeing accurate readings, and improving overall inspection efficiency. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a flowchart of a gimbal search and tracking method provided in an embodiment of this application.

[0028] Figure 2 This is a schematic diagram illustrating indirect positioning in a gimbal search and tracking method provided in an embodiment of this application.

[0029] Figure 3 This is another implementation of a gimbal search and tracking method provided in the embodiments of this application. Detailed Implementation

[0030] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0031] To make the technical solutions and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0032] This application provides a gimbal search and tracking method for a dial tracking system. The dial tracking system includes an inspection robot, a drive unit mounted on the inspection robot, a gimbal mounted on the drive unit, a camera mounted on the gimbal, and a control module. Figure 1 As shown, the method includes:

[0033] Step 101: The camera captures images in real time.

[0034] Step 102: The control module identifies the dial in the preprocessed image. The dial positioning method includes direct positioning mode and indirect positioning mode: the direct positioning mode determines the spatial position of the dial in the image coordinate system in response to the presence of the dial in the preprocessed image; the indirect positioning mode determines the spatial position of the dial in the image coordinate system in response to the absence of the dial in the preprocessed image by identifying fixed objects near the dial for auxiliary positioning.

[0035] Indirect positioning mode can reduce repetitive dial search processes. When direct positioning is not possible, indirect positioning can use a fixed object to spatially locate the target dial, saving time spent on repeated searches.

[0036] When the watch face is within the camera's field of view (i.e., the captured image includes the watch face), the direct positioning mode can directly locate the watch face's position, and the watch face can be adjusted to the optimal reading position by fine-tuning the gimbal angle. When the watch face is not within the camera's field of view (i.e., the captured image does not include the watch face), the indirect positioning mode can first identify fixed objects near the watch face, thereby assisting in locating the watch face and indirectly locating its position.

[0037] In some embodiments, preprocessing includes scaling, filtering, and pixel adjustment of the image.

[0038] Step 103: The control module determines the control signal based on the spatial position of the dial in the image coordinate system; based on the control signal, it controls the drive components to adjust the gimbal's lifting height, yaw angle, and pitch angle, the camera's focal length, and the inspection robot's tracking path to track the dial.

[0039] In some embodiments, the control module carries a YOLOv8 model, which is used to identify and label the dial in the preprocessed image, determining the spatial position of the dial in the image coordinate system. The large model used for dial recognition needs to be trained first to achieve the required recognition accuracy.

[0040] In some embodiments, the control module carries a YOLOv8 model. The YOLOv8 model identifies and marks fixed objects in the preprocessed image for auxiliary localization, determining the spatial position of the dial in the image coordinate system. The large model used for identifying fixed objects needs to be trained first to achieve the required recognition accuracy.

[0041] In some embodiments, the fixed object is an object that is fixedly located near the dial.

[0042] In some embodiments, identifying a fixed object near the dial for assisted localization to determine the spatial position of the dial in the image coordinate system includes:

[0043] Identify fixed objects near the dial and determine their positions.

[0044] Based on the relative position of the dial and a fixed object near the dial, the gimbal's lifting height, yaw angle, pitch angle, and camera focal length adjustment direction are determined until the dial is included in the image. The relative position of the dial and a fixed object near the dial has been input into the Yolov8 model.

[0045] The spatial position of the dial in the image coordinate system is determined to complete the assisted positioning. In some embodiments, the relative position of the dial to a fixed object near the dial is included in the dataset generated after measurement. The dataset is stored on the system hard drive and can be accessed at any time after each program startup.

[0046] In some embodiments, before the control module identifies the dial in the preprocessed image, the method further includes:

[0047] The control module controls the inspection robot to reach the preset position to identify the dial, and adjusts the gimbal's lifting height, yaw angle, pitch angle, and camera focal length to the initial pose, thereby achieving automatic adjustment of its pose to bring the dial into the inspection robot's camera field of view.

[0048] In some embodiments, the method further includes a precise positioning process:

[0049] The control module responds to the dial not being centered on the image and determines the fine-tuning signal;

[0050] Based on fine-tuning signals, the control actuators adjust the gimbal's height, yaw and pitch angles, the camera's focal length, and the inspection robot's tracking path. These fine-tuning signals regulate the gimbal's height, yaw and pitch angles, the camera's focal length, and the inspection robot's tracking path, ensuring the display is centered on the image.

[0051] Such as a 2D top view Figure 2As shown, the solid line represents the ideal pose, the short dashed line represents the actual pose, and the long dashed line represents the corrected pose. Figure 2 As can be seen, in the actual pose, the camera is not aligned with the center of the watch face, indicating that the watch face is not in the center of the image. This can be addressed by identifying fixed objects near the watch face. Figure 2 The camera obtains the position of a fixed object near the dial. At this time, the camera, in its actual pose, further determines the gimbal's lifting height, yaw angle, and pitch angle, as well as the camera's focal length adjustment direction, based on the relative position of the dial and the fixed object near the dial, until the dial is included in the image. This results in a corrected pose that is close to the ideal pose, enabling the dial to be close to the center of the dial.

[0052] like Figure 3 As shown, the following implementation method for gimbal search and tracking is given:

[0053] Step ① Initialize positioning: After the inspection robot reaches the preset dial reading position, adjust the gimbal's lifting height, yaw angle, pitch angle, and camera focal length to the initial pose.

[0054] Step ② Direct Detection Stage: Detects whether the dial is within the camera's field of view; if Step ② fails, it enters indirect positioning mode:

[0055] Step 8: Environmental Feature Recognition: Simultaneously identify predefined fixed objects around the dial;

[0056] Step 9: Focus adjustment: Shorten the focal length to expand the search range;

[0057] Step 10: Compensation and Adjustment: Based on the identified environmental features (i.e., fixed objects), automatically calculate the gimbal's elevation, yaw and pitch angles, the camera's focal length, and the inspection robot's tracking path.

[0058] Once the watch face is successfully captured, the precise positioning process is executed:

[0059] Step 3: Centralized Adjustment: The control module responds to the fact that the dial is not in the center of the image and determines the fine-tuning signal; based on the fine-tuning signal, the control drive adjusts the gimbal's lifting height, yaw angle and pitch angle, the camera's focal length and the inspection robot's tracking path to make the dial dynamically centered.

[0060] Step 4: Focus adjustment: Extend the focal length and check whether the pixel size of the dial in the image meets the resolution requirements;

[0061] Step ⑤ Size verification: Automatically adjust optical zoom if size requirements are not met;

[0062] Final execution:

[0063] Step 6: Data Reading: After completing the dual calibration of position and size, take accurate readings of the dial.

[0064] In summary, the gimbal search and tracking method provided in this application embodiment can accurately adjust the offset of the preset pose of the gimbal when the navigation and positioning accuracy of the inspection robot decreases, so that the camera can accurately align with the dial, thereby ensuring accurate readings and improving overall inspection efficiency.

[0065] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only.

[0066] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A gimbal search and tracking method, characterized in that, An application is made to a dial tracking system, the dial tracking system comprising an inspection robot, a drive unit mounted on the inspection robot, a gimbal mounted on the drive unit, a camera mounted on the gimbal, and a control module, the method comprising: The camera captures images in real time; The control module identifies the dial in the preprocessed image. The dial is located using two modes: direct positioning and indirect positioning. In direct positioning, the dial is located in the image coordinate system if the preprocessed image contains the dial. In indirect positioning, the dial is located in the image coordinate system if the preprocessed image does not contain the dial. Fixed objects near the dial are identified to assist in the positioning and determine the dial's spatial position in the image coordinate system. The control module determines the control signal based on the spatial position of the dial in the image coordinate system; based on the control signal, it controls the drive components to adjust the gimbal's lifting height, yaw angle, and pitch angle, the camera's focal length, and the inspection robot's tracking path to track the dial.

2. The gimbal search and tracking method according to claim 1, characterized in that, Preprocessing includes scaling, filtering, and pixel adjustment of the image.

3. The gimbal search and tracking method according to claim 1, characterized in that, The control module contains a YOLOv8 model, which is used to identify and mark the dial in the preprocessed image, and determine the spatial position of the dial in the image coordinate system.

4. The gimbal search and tracking method according to claim 1, characterized in that, The control module contains a YOLOv8 model. The YOLOv8 model is used to identify and mark fixed objects in the preprocessed image for auxiliary positioning, thereby determining the spatial position of the dial in the image coordinate system.

5. The gimbal search and tracking method according to claim 1, characterized in that, Identify fixed objects near the dial for auxiliary localization, and determine the spatial position of the dial in the image coordinate system, including: Identify fixed objects near the dial and determine their positions. Based on the relative position of the dial and a fixed object near the dial, the gimbal's lift height, yaw angle, pitch angle, and camera focal length adjustment direction are determined until the dial is included in the image. The fixed object is an object that is always present near the dial. The relative position of the dial and the fixed object near the dial has been input into the Yolov8 model. Determine the spatial position of the dial in the image coordinate system to complete the auxiliary positioning.

6. The gimbal search and tracking method according to claim 1, characterized in that, Before the control module identifies the dial in the preprocessed image, the method further includes: The control module controls the inspection robot to reach the preset position to identify the dial, and adjusts the gimbal's lifting height, yaw angle, pitch angle, and camera focal length to the initial pose.

7. The gimbal search and tracking method according to claim 1, characterized in that, The method also includes a precise positioning process: The control module responds to the fact that the dial is not in the center of the image and determines the fine-tuning signal; based on the fine-tuning signal, it controls the drive components to adjust the gimbal's lifting height, yaw angle and pitch angle, the camera's focal length, and the inspection robot's tracking path.

8. The gimbal search and tracking method according to claim 1, characterized in that, The method further includes: The control module responds to the dial being centered on the image, reads the dial's indicated parameters, and obtains the reading.

Images