A two-stage visual tracking pose publisher and a publishing method thereof

Abstract

The application discloses a double-stage visual tracking pose publisher and a publishing method thereof. The pose publisher obtains a first pose result of a servo observation part or a second observation unit in a reference coordinate system through a first-stage visual observation link, observes a mobile tag with an identity provided on a tracked component through a second-stage visual observation link, obtains a second pose result of the mobile tag or the tracked component relative to the second observation unit and a target identity, and resolves a six-degree-of-freedom pose of the tracked component in the reference coordinate system in combination with a pre-calibrated mounting relationship. The target identity, a unified timestamp, a six-degree-of-freedom pose and a state quantity corresponding to the same tracked component and corresponding to the same resolving time under a unified time reference are bound into a standardized pose message and externally published through message packaging and a message publishing unit. The application improves real-time performance and consistency in a multi-device collaborative scene.

Description

Technical Field

[0001] This invention relates to the field of machine vision technology, specifically to a two-level visual tracking pose publisher and its publishing method. Background Technology

[0002] With the development of applications such as industrial measurement, robot guidance, spatial positioning, intelligent sensing, and digital modeling, visual tagging-based target tracking and pose estimation technologies have been widely applied in scenarios such as object localization, tool tracking, spatial measurement, and control guidance. Existing optical tracking systems typically acquire tag information on the target through one or more cameras and recover the position and pose of the tracked object based on the imaging results of the tags.

[0003] US Patent 8224024B2 has demonstrated that the complete pose of a rigid body object is typically represented by six degrees of freedom. Common markers include LEDs, reflective dots, and coded patterns. Camera systems can track objects by detecting and measuring these markers. Furthermore, existing technologies also include schemes for target identification and marker encoding. For example, US 12025735B2 discloses a coordinate measuring device for automatic target identification, which can automatically identify target objects and bind specification information using a target database, ID pattern, or ID color; CN 109716061B discloses a marker and a pose estimation method using the marker; and CN 118369062A discloses a novel optical tracking marker structure. These technologies illustrate that establishing a stable one-to-one correspondence between target identity and pose has become an important direction for improving the usability and automation level of tracking systems.

[0004] On the other hand, in scenarios involving multiple cameras or modules working collaboratively, time synchronization issues directly affect the consistency and reusability of pose results. US10313103B1 discloses a precise time synchronization scheme based on IEEE 1588 / PTP and discusses the time error problem caused by time delay asymmetry; CN112449172A illustrates the limitations of existing multi-camera exposure synchronization methods in terms of robustness, efficiency, and timestamp consistency. It is evident that in multi-level vision links, simply completing target recognition and pose calculation is insufficient; how to generate reusable, standardized pose results under a unified time reference is also a key issue in systems engineering.

[0005] However, existing technologies mostly focus on obtaining measurement point coordinates, target recognition, marker design, or multi-camera synchronization itself. There is still a lack of public implementation of identifying luminous markers with identification features, completing six-DOF pose calculations, and uniformly encapsulating the identification, pose, timestamp, and state variables into a standardized pose message for external devices. Subsequent devices often still need to perform some recognition, timing alignment, coordinate transformation, or repetitive calculations themselves, leading to high system integration complexity and insufficient consistency across multiple devices. Summary of the Invention

[0006] To address the problems of easily confused target identities, inconsistent pose calculation time references, redundant calculations by downstream devices, and inconsistent coordinate references when multiple devices call the system in existing visual tracking systems, this invention provides a two-level visual tracking pose publisher, characterized by comprising a first-level visual observation link, a second-level visual observation link, a follow-up observation part, a pose calculation unit, and a message encapsulation and message publishing unit; the follow-up observation part specifically includes a reference marker component and a second observation unit. The first-level visual observation link includes a fixed observation unit and a reference marker assembly. The fixed observation unit is mounted on a support mechanism, and the reference marker assembly is disposed on a follow-up observation part. The fixed observation unit determines the position and orientation of the follow-up observation part or the second observation unit relative to the reference coordinate system by observing the reference marker assembly. The first-level visual observation link is used to obtain the first orientation result of the follow-up observation part or the second observation unit relative to the reference coordinate system. The second-level visual observation link includes a second observation unit installed on the follow-up observation part. The second observation unit tracks and locates the tracked component by observing a moving tag set on the tracked component. The moving tag is used to provide a basis for target identification and pose calculation. The second-level visual observation link is used to observe the moving tag with identification set on the tracked component and obtain the second pose result of the moving tag relative to the second observation unit and the target identification. The pose calculation unit is used to calculate the six-degree-of-freedom pose of the tracked component in the reference coordinate system based on the first pose result, the second pose result, and the pre-calibrated installation relationship. The message encapsulation and message publishing unit is used to bind the target identity identifier, unified timestamp, six-degree-of-freedom pose and state variables corresponding to the same tracked component and the same solution time under the same time base into a standardized pose message, and publish the standardized pose message to the outside world.

[0007] Furthermore, it also includes a support mechanism for supporting the first-level visual observation link and a follow-up observation platform for supporting the second-level visual observation link; the support mechanism includes a base, and the follow-up observation platform includes a heading rotation mechanism and a pitch rotation platform. The heading rotation mechanism is used to drive the follow-up observation platform to rotate around the heading axis, and the pitch rotation platform is used to drive the second observation unit to rotate around the pitch axis.

[0008] Furthermore, the mobile tag and reference mark components are both luminescent marks or reflective marks with identification, or target structures containing coded patterns, dot matrix arrangements, and time-sequential luminescence features.

[0009] Furthermore, it also includes a solution and release mechanism, which specifically includes: a data acquisition unit, a timing control unit, a tracked object identity solution unit, a pose solution unit, and a message encapsulation and message release unit; The data acquisition unit is used to receive image data output by the fixed observation unit, the second observation unit and / or the wide-angle auxiliary observation unit, and to preprocess the received image data; The timing control unit is used to provide a unified time reference for the observation results of the first-level visual observation link and the observation results of the second-level visual observation link, and to generate timestamp information. The tracked object identity calculation unit is used to identify and calculate the identity of the mobile tag based on the image data collected by the second observation unit and / or the wide-angle auxiliary observation unit; The pose calculation unit is used to calculate the six-degree-of-freedom pose of the tracked component in the reference coordinate system based on the observation results of the first-level visual observation link and the second-level visual observation link. The message encapsulation and message publishing unit is used to receive timestamp information, identity information, six-degree-of-freedom pose and state information, and encapsulate them into standardized pose messages for external publication.

[0010] Furthermore, the second-level visual observation link also includes a wide-angle auxiliary observation unit. The wide-angle auxiliary observation unit is set on the follow-up observation platform and is used to perform wide-field observation on the moving tag and output coarse positioning results or field-of-view guidance results when the moving tag initially enters, moves rapidly, deviates from the field of view of the second observation unit, or reappears after being lost for a short time. This is to assist the second observation unit in recaptured the moving tag or to assist the pose calculation unit in performing auxiliary correction or state judgment.

[0011] Furthermore, the standardized pose message specifically includes a target identity identifier, a unified timestamp, a six-degree-of-freedom pose and state variables, and further includes one or more of the following: message header, protocol version, message type, message length, device identifier, coordinate system identifier, confidence level, serial number, calibration version and verification field; When encapsulating the standardized pose message, the message encapsulation and message publishing unit binds the target identity identifier, unified timestamp, six-degree-of-freedom pose, and state variables as associated fields in the same message instance, so that each standardized pose message has a corresponding target, time, and pose.

[0012] Furthermore, when the message encapsulation and message publishing unit encapsulates the standardized pose message, it generates the standardized pose message only when the target identity is valid, the six-degree-of-freedom pose is valid, the unified timestamp is valid, and the current solution result meets the preset confidence condition or state condition. If any of the above conditions are not met, output a message with abnormal state variables, or output only a state message instead of the standardized pose message at that moment.

[0013] A method for publishing a two-level visual tracking pose publisher, characterized by comprising the following steps: S1. The reference marker component is observed through the first-level visual observation link, and the mobile tag with identification set on the tracked component is observed through the second-level visual observation link. The observed image data is transmitted to the data acquisition unit for preprocessing. S2. Identify the mobile tag based on the image data collected by the second-level visual observation link to obtain the target identity identifier corresponding to the currently tracked component; S3. Based on the observation results of the first-level visual observation link, obtain the first pose result of the second observation unit or the follow-up observation part relative to the reference coordinate system; S4. Based on the observation results of the moving tag by the second-level visual observation link, obtain the second pose result of the moving tag or the tracked component relative to the second observation unit; S5. Based on the first pose result obtained in step S3, the second pose result obtained in step S4, and the pre-calibrated installation relationship, perform a fusion calculation of the six-degree-of-freedom pose of the tracked component in the reference coordinate system; S6. The target identity, unified timestamp, six-degree-of-freedom pose and state variables corresponding to the same tracked component and the same solution time under the same time base are encapsulated into a standardized pose message through the message encapsulation and message publishing unit, and the standardized pose message is sent to the external device according to the preset publishing method.

[0014] Furthermore, the pose of the second observation unit in the reference coordinate system in step S3 satisfies: ; In the formula, This represents the pose of the reference marker component relative to the reference coordinate system at time k. This indicates the pre-calibrated installation relationship between the reference marker assembly and the second observation unit. This represents the pose of the second observation unit relative to the reference coordinate system at time k.

[0015] Furthermore, in step S5, the pose of the tracked component in the reference coordinate system satisfies: ; In the formula, This represents the pose of the moving label relative to the second observation unit at time k. This indicates the pre-defined installation relationship between the mobile tag and the tracked component. This represents the six-DOF pose of the tracked component in the reference coordinate system at time k. This represents the pose of the second observation unit relative to the reference coordinate system at time k. When the observation results of the second observation unit are valid, the solution results of the second observation unit are used as the main pose results. When the confidence of the observation results of the second observation unit is insufficient or the target deviates from the field of view, the observation results of the wide-angle auxiliary observation unit are used for coarse positioning, auxiliary correction or recovery tracking.

[0016] Furthermore, the six-degree-of-freedom pose of the tracked component in the reference coordinate system is represented as follows: ; In the formula, This indicates the position parameters of the tracked component in the reference coordinate system. The attitude parameters of the tracked component in the reference coordinate system are represented by a rotation matrix, Euler angles, quaternions, or direction cosine matrix.

[0017] Furthermore, the standardized pose message in step S6 is represented as follows: ; ; In the formula, This indicates at least one of the following header fields: message header, protocol version, message type, and message length. This represents the set of message body fields or payload fields for the k-th standardized pose message. This represents the validation field. Indicates equipment identification. Indicates the target's identity identifier. Indicates a unified timestamp. This represents the target's six-degree-of-freedom pose in the reference coordinate system. Represents state variables. Indicates the coordinate system identifier. Indicates the confidence level. Indicates the serial number. Indicates the calibration version.

[0018] Furthermore, the preset publishing method includes at least one of broadcast publishing, multicast publishing, point-to-point publishing, polling-response publishing, and event-triggered publishing.

[0019] This invention provides a two-level visual tracking pose publisher and its publishing method, which has the following beneficial effects: This invention establishes spatial relationships between a reference coordinate system, a second observation unit, a moving tag, and the tracked component through a first-level visual observation link and a second-level visual observation link, thereby improving the uniformity of pose calculation. By binding and publishing the target identity identifier, unified timestamp, six-degree-of-freedom pose, and state variables corresponding to the same tracked component and the same calculation time under a unified time reference, it avoids identity mismatch and timing mismatch in multi-target scenarios. By publishing standardized pose messages externally, external devices can directly access the target's pose information in the reference coordinate system, thereby reducing system integration complexity and improving real-time performance and consistency in multi-device collaborative scenarios. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0021] Figure 1 A schematic diagram of the pose publisher structure provided by the present invention; Figure 2 This is a schematic diagram of the installation scenario of the pose publisher provided in an embodiment of the present invention. Detailed Implementation

[0022] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0023] The following detailed description of the implementation method of the present invention is in conjunction with the accompanying drawings. The description is only a partial embodiment and not all embodiments. For clarity, representations and descriptions unrelated to the present invention are omitted in the drawings and description.

[0024] To provide a clearer understanding of the technical features, objectives, and beneficial effects of this invention, the following detailed description of the technical solution is provided. Obviously, the described embodiments are only a portion of the embodiments of this invention, not all of them, and should not be construed as limiting the scope of implementation of this invention. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without inventive effort are within the protection scope of this invention.

[0025] like Figure 1 As shown, the two-level visual tracking pose publisher can form an application relationship with the tracked components, where the number of tracked components can be determined according to the number of targets that need to be tracked simultaneously in the actual application scenario. The pose publisher may include a carrier motion mechanism, a two-level observation mechanism, and a calculation and publishing mechanism. The carrier motion mechanism may include a base, a yaw rotation mechanism, and a pitch rotation table; the two-level observation mechanism may include a fixed observation unit, a reference marker component, a second observation unit, and a wide-angle auxiliary observation unit; the calculation and publishing mechanism may include a data acquisition unit, a timing control unit, a tracked object identity calculation unit, a pose calculation unit, and a message encapsulation and message publishing unit. A motion tag is provided on the tracked component, and the motion tag may be an luminous marker with an identification identifier.

[0026] The base serves as the overall fixed structure supporting the pose generator and as the mounting foundation for components such as the fixed observation unit and the heading rotation mechanism. During operation, the base can be fixed to a bracket, the ground, a platform, or other mounting interface to ensure the stability of the entire generator's reference coordinate system. The base itself may have mounting holes, reinforcing structures, wiring channels, and protective structures to ensure equipment assembly accuracy, structural rigidity, and long-term stability. The base also serves as the reference mounting benchmark for the fixed observation unit, providing a unified mechanical reference for subsequent pose calculations.

[0027] The heading rotation mechanism, mounted on the base, supports the pitch rotary table and drives it to rotate around the heading axis. The heading rotation mechanism can be driven by a motor, reducer, encoder feedback, or other rotary drive methods to adjust the observation direction of the second observation unit and the wide-angle auxiliary observation unit in the horizontal plane. The heading rotation mechanism can receive control signals from the control unit to perform actions such as angle adjustment, preset scanning, target following, or position reset, and sends its own angle status, operating status, or feedback information to relevant processing units for subsequent pose calculation or system control.

[0028] The pitch rotary table, mounted on the yaw rotation mechanism, supports the second observation unit, the wide-angle auxiliary observation unit, and the reference marker assembly, and drives these components to rotate around the pitch axis. The pitch rotary table enables vertical field-of-view adjustment, allowing the dual-stage observation mechanism to cover a larger spatial area. The pitch rotary table can employ a motor drive, a reduction gear mechanism, an angle detection structure, or other motion execution mechanisms to adjust the pitch angle of the optical observation unit and transmit the current angle or operating status to the relevant processing unit.

[0029] The fixed observation unit, mounted on the base, is used to observe the reference marker assembly positioned on the pitch rotary table. Its function is to acquire image information of the reference marker assembly in the reference coordinate system, thereby determining the position and attitude of the pitch rotary table and its supporting components within the reference coordinate system. The fixed observation unit is the core component of the system's first-level visual observation link; its output image data is sent to the data acquisition unit for processing by the subsequent pose calculation unit. The fixed observation unit can employ a monocular camera, binocular camera, infrared camera, or other image sensors, and can be equipped with lenses, filters, illumination structures, etc., as needed.

[0030] The reference marker assembly is mounted on the pitch rotation stage and is used for identification and tracking by the fixed observation unit. The reference marker assembly can be an luminous mark, a reflective mark, a coded pattern, a dot matrix mark, or other image-recognizable identification structure. Its function is to provide a stable visual reference for the fixed observation unit, thereby establishing the position and attitude relationship of the pitch rotation stage and its supporting components relative to the reference coordinate system. The reference marker assembly, along with the second observation unit and the wide-angle auxiliary observation unit, belongs to the servo component. Its attitude changes reflect the attitude changes of the platform where the second observation unit is located, making it an important component of the system's first-level visual observation link.

[0031] The second observation unit, mounted on a pitch and rotation stage, is used to observe the moving tag on the tracked component. This unit acquires image information from the moving tag and provides key observation data for identifying and assuming the tracked object. As the core component of the system's second-level visual observation link, its image data is sent to the data acquisition unit for subsequent processing. The second observation unit can employ industrial cameras, high-definition cameras, infrared cameras, or other imaging devices, and the appropriate focal length, field of view, and imaging method can be selected based on actual needs to balance recognition accuracy and tracking stability.

[0032] The wide-angle auxiliary observation unit is mounted on a pitch rotation stage and is used to observe the moving tag on the tracked component with a large field of view. When the moving tag initially enters, moves rapidly, deviates from the field of view of the second observation unit, or reappears after a short period of loss, the wide-angle auxiliary observation unit can output coarse positioning results or field-of-view guidance results to assist the second observation unit in re-capturing the moving tag, or to assist the pose calculation unit in making auxiliary corrections or status judgments. The image data acquired by the wide-angle auxiliary observation unit is also sent to the data acquisition unit and can be used together with the image information from the second observation unit to participate in identity recognition and pose calculation, thereby improving the robustness and tracking range of the system.

[0033] The data acquisition unit receives image data output from the fixed observation unit, the second observation unit, and the wide-angle auxiliary observation unit, and performs buffering, organization, temporal alignment, forwarding, or preprocessing on the data. The data acquisition unit serves as the data interface between each visual observation unit and the subsequent identity and pose calculation modules. Depending on the system hardware architecture, it can utilize an acquisition card, image interface board, embedded processing module, bus interface module, or other data access methods. The data acquisition unit sends the acquired image information to the tracked object identity and pose calculation units, providing raw input for subsequent processing.

[0034] The timing control unit provides a unified time reference for data acquisition, identity recognition, pose calculation, and message output processes, and generates, allocates, corrects, or maintains timestamp information within the system. The timing control unit can connect to the data acquisition unit, pose calculation unit, and message encapsulation and publishing unit to ensure time consistency across different observation links, processing stages, and output messages. The timing control unit can employ a local clock, synchronous trigger circuit, unified time base module, or other time control structures, its function being to ensure the time traceability of pose messages and consistency across multiple devices.

[0035] In one implementation, the timing control unit assigns timestamps to the observation results of the first-level visual observation link and the observation results of the second-level visual observation link, respectively. The pose calculation unit uses the first-level observation results and the second-level visual observation results that are in the same calculation cycle or within a preset time window for the same fusion calculation, and writes the unified timestamp corresponding to this fusion calculation into a standardized pose message. When the timestamp difference between the first-level observation results and the second-level observation results exceeds a preset time threshold, the message encapsulation and message publishing unit can output a low-confidence state, a prediction state, or an abnormal state.

[0036] The tracked object identification unit is used to identify and solve the identity of the mobile tag based on the image data acquired by the second observation unit and / or the wide-angle auxiliary observation unit. This unit can identify the unique identity information of the currently tracked object based on the coding features, emission features, spatial distribution features, temporal features, or other identification features on the mobile tag, and bind the identity information to the subsequent pose results. The identity information output by the tracked object identification unit is sent to the message encapsulation and message publishing unit, and can also be provided to the pose calculation unit as auxiliary information to support multi-target differentiation and stable tracking.

[0037] The pose calculation unit is used to calculate the six-degree-of-freedom pose information of the tracked component in the reference coordinate system based on the first-level visual observation results and the second-level visual observation results. Specifically, the pose calculation unit uses the observation results of the fixed observation unit on the reference marker component to determine the position and orientation of the follower part where the second observation unit is located in the reference coordinate system; on the other hand, it uses the observation results of the second observation unit and / or the wide-angle auxiliary observation unit on the moving tag to determine the pose of the moving tag or the tracked component relative to the second observation unit; finally, by combining the calibration relationship, the installation relationship and the necessary geometric transformation relationship, the position and orientation information of the tracked component in the reference coordinate system is obtained.

[0038] For ease of explanation, the reference coordinate system is defined as follows: Fixed to the carrier mechanism, base, or installation environment of the pose publisher; the reference marker component coordinate system is... Fixed to the reference marker component; the second observation unit coordinate system is Fixed in the second observation unit; the moving label coordinate system is Fixed to the moving label; the coordinate system of the tracked component is It is fixed to the tracked component. The origin and axis of each coordinate system are determined by marking structures, equipment installation relationships, or pre-calibration. In the first... At any given moment, the first-level visual observation link can obtain the pose of the reference marker component relative to the reference coordinate system. Combined with the pre-calibrated installation relationship between the reference marker assembly and the second observation unit The pose of the second observation unit in the reference coordinate system can be obtained: The second-level visual observation link can obtain the pose of the moving tag relative to the second observation unit. Combined with the pre-defined installation relationship between the mobile tag and the tracked component This allows us to obtain the pose of the tracked component in the reference coordinate system: If a certain observation link outputs a coordinate transformation relationship in the opposite direction, the pose calculation unit can unify it through the corresponding inverse transformation. For example, when the second-level visual observation link outputs an inverse transformation relationship from the moving label coordinate system to the second observation unit coordinate system, the following equivalent transformation can be used: The six-DOF pose of the tracked component in the reference coordinate system can be represented by a homogeneous transformation matrix as follows: in, This represents the attitude rotation matrix of the tracked component in the reference coordinate system. This represents the position vector of the tracked component in the reference coordinate system. The position vector can be represented as: Attitude parameters can also be expressed as: Therefore, the six-degree-of-freedom pose can also be denoted as: in, Indicates position parameters, This represents attitude parameters. Attitude parameters can be expressed using Euler angles, quaternions, rotation matrices, or direction cosine matrices, and can be mapped to a uniform field format for output during message encapsulation.

[0039] During the fusion calculation process, when the second observation unit can stably observe the moving tag and the observation results meet the confidence requirements, the pose calculation unit can use the calculation results of the second observation unit as the main pose result. When the confidence of the observation results of the second observation unit is insufficient, the moving tag deviates from the field of view of the second observation unit, or the target is temporarily lost, the pose calculation unit can use the observation results of the wide-angle auxiliary observation unit to perform coarse target localization, auxiliary correction, or recovery tracking. The coarse localization results output by the wide-angle auxiliary observation unit can be used as a state judgment or auxiliary fusion basis before the second observation unit resumes stable observation.

[0040] The message encapsulation and message publishing unit receives timestamp information from the timing control unit, identity information from the tracked object identity calculation unit, six-DOF pose information and related state information from the pose calculation unit, and encapsulates them into a standardized pose message for external publication. The standardized pose message includes at least the target identity identifier, unified timestamp, six-DOF pose, and state variables, and may further include one or more of the following: message header, protocol version, message type, message length, device identifier, coordinate system identifier, confidence level, serial number, calibration version, and verification field.

[0041] For the The output result, and the core input received by the message encapsulation and message publishing unit, can be represented as follows: in, Indicates the target's identity identifier. Indicates a unified timestamp. This represents the target's six-degree-of-freedom pose in the reference coordinate system. Represents state variables. It represents a set of metadata such as coordinate system identifier, confidence level, message sequence number, and calibration version.

[0042] Correspondingly, the encapsulated standardized pose message can be represented as: in, This indicates at least one of the following header fields: message header, protocol version, message type, and message length. This represents the set of message body fields or payload fields for the k-th standardized pose message. This represents the validation field. This indicates the device identifier for the pose publisher. Indicates the target's identity identifier. Indicates the coordinate system identifier. Indicates the confidence level. Indicates the serial number. Indicates calibration version

[0043] To ensure the traceability and sequentiality of each message, the message encapsulation and message publishing unit assigns a sequence number to each output message. If the sequence number corresponding to the current message is... Then the sequence number of subsequent messages can satisfy: Alternatively, it can wrap around to the initial value and start counting again after reaching the set maximum value. At the same time, when encapsulating pose messages, the message encapsulation and message publishing units bind the unified timestamp, target identity, six-degree-of-freedom pose, and state variables corresponding to the same tracked component and the same solution time under the same time base as associated fields in the same message instance. This ensures that each pose message has a one-to-one correspondence between "target-time-pose", thereby guaranteeing that the subscriber can correctly recover the system time and target identity corresponding to the current result.

[0044] The message encapsulation and message publishing unit can generate a standardized pose message only when the following conditions are met simultaneously: the identity calculation unit successfully outputs a valid identity identifier, the pose calculation unit successfully outputs a valid six-DOF pose result, the timing control unit provides a valid unified timestamp, and the current result meets preset confidence conditions or state conditions. When a certain condition is not met, the message encapsulation and message publishing unit can output a message with an abnormal state quantity, or it can not output the standardized pose message at that moment, but only output a state message; the abnormal state quantity can be used to characterize at least one of the following states: invalid identity, invalid pose, invalid timestamp, low confidence, target occlusion, target leaving the main field of view, prediction output, or recovery tracking.

[0045] After message encapsulation, the message encapsulation and message publishing unit sends the standardized pose message to external devices via wired interfaces, wireless interfaces, network buses, or other data links. The publishing method can be broadcast publishing, multicast publishing, point-to-point publishing, polling-response publishing, or event-triggered publishing. In broadcast publishing mode, the pose publisher periodically outputs the pose messages of all targets for simultaneous reception by multiple external devices; in point-to-point or subscription publishing mode, the pose publisher can send the pose messages of the corresponding target only to specified external devices according to pre-established subscription relationships.

[0046] After receiving the standardized pose message, the external device can directly parse the target identity, unified timestamp, six-DOF pose, and state variables from the standardized pose message to obtain the target's valid pose information in the reference coordinate system corresponding to the unified timestamp. Based on the parsing results, it can perform control, display, recording, navigation, or other business processing. Since target recognition, two-level visual pose calculation, coordinate unification, and timestamp binding are all completed on the pose publisher side, the external device can reduce repetitive visual calculations and timing alignment processing.

[0047] The tracked component is the object whose pose information is observed and published by the system. It can be a tool head, actuator, mobile terminal, tooling, measuring component, robot end effector, handheld device, or other physical part requiring pose recognition and publication. The tracked component itself is not part of the pose publisher's internal structure, but as an external object of the pose publisher, it forms a complete application relationship with the pose publisher. During operation, the tracked component carries a motion tag for identification and tracking by the second observation unit and the wide-angle auxiliary observation unit. The motion tag is attached to the tracked component to provide the pose publisher with an identifiable visual target. The motion tag can be an luminous mark with identification, or a target structure containing coded patterns, dot matrix arrangements, temporal luminescence features, or other visual marking features. The functions of the motion tag include: providing a basis for identifying the tracked object, providing a basis for calculating the pose of the tracked object, and supporting multi-target differentiation and status judgment. After the motion tag is observed by the second observation unit and / or the wide-angle auxiliary observation unit, its identity information and spatial pose information are processed to form a standardized pose message, which is then published externally by the pose publisher.

[0048] Example: Figure 2 As shown, the pose transmitter of the present invention can be installed on the roof frame of a factory or other high-level fixed structure, so that the pose transmitter remains fixed relative to the factory environment, thereby establishing a stable reference coordinate system. The pose transmitter is equipped with a two-stage visual tracking mechanism, wherein the follow-up observation part can perform yaw rotation and / or pitch adjustment within a certain range to continuously observe and track moving tags in the working area.

[0049] The motion tag is set on an external subscription device or a tracked component associated with an external subscription device. The pose publisher observes the motion tag, identifies its identity, and calculates its six-DOF pose information in the reference coordinate system. Subsequently, the pose publisher encapsulates the identity information, timestamp, pose information, and status information corresponding to the motion tag into a standardized pose message and publishes it to the external subscription device.

[0050] Therefore, the external subscribing device is observed by the pose publisher as a tracked object on the one hand, and receives pose messages corresponding to itself as a message receiver on the other, thereby obtaining the real-time position and attitude information of the device in the factory's reference coordinate system. This application method is suitable for mobile devices, inspection equipment, handling equipment, robot end effectors, handheld terminals, or other devices in the factory that need to obtain their own spatial pose in real time.

[0051] In this application scenario, because the pose publisher is installed at a high position and has a large observation range, it can continuously track targets in a certain area on the ground or in the air. At the same time, through a unified time reference and a standardized pose message publishing mechanism, external devices do not need to complete complex visual positioning and coordinate transformation on their own, and can directly obtain effective pose information corresponding to their own identity, thereby reducing the computational burden on the terminal side and improving the overall consistency and real-time performance of the system.

[0052] In addition to installation on factory rooftops, the pose publisher can also be applied to robot workstations, mobile platform positioning, warehouse handling equipment tracking, handheld tool positioning, or spatial pose publishing for detection equipment.

[0053] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A two-stage visual tracking pose publisher, characterized in that, It includes a first-level visual observation link, a second-level visual observation link, a follow-up observation part, a pose calculation unit, and a message encapsulation and message publishing unit; the follow-up observation part specifically includes a reference marker component and a second observation unit. The first-level visual observation link includes a fixed observation unit and a reference marker assembly. The fixed observation unit is mounted on a support mechanism, and the reference marker assembly is disposed on a follow-up observation part. The fixed observation unit determines the position and orientation of the follow-up observation part or the second observation unit relative to the reference coordinate system by observing the reference marker assembly. The first-level visual observation link is used to obtain the first orientation result of the follow-up observation part or the second observation unit relative to the reference coordinate system. The second-level visual observation link includes a second observation unit installed on the follow-up observation part. The second observation unit tracks and locates the tracked component by observing a moving tag set on the tracked component. The moving tag is used to provide a basis for target identification and pose calculation. The second-level visual observation link is used to observe the moving tag with identification set on the tracked component and obtain the second pose result of the moving tag relative to the second observation unit and the target identification. The pose calculation unit is used to calculate the six-degree-of-freedom pose of the tracked component in the reference coordinate system based on the first pose result, the second pose result, and the pre-calibrated installation relationship. The message encapsulation and message publishing unit is used to bind the target identity identifier, unified timestamp, six-degree-of-freedom pose and state variables corresponding to the same tracked component and the same solution time under the same time base into a standardized pose message, and publish the standardized pose message to the outside world.

2. The dual-level visual tracking pose publisher according to claim 1, characterized in that, It also includes a support mechanism for supporting the first-level visual observation link and a follow-up observation platform for supporting the second-level visual observation link; the support mechanism includes a base, and the follow-up observation platform includes a heading rotation mechanism and a pitch rotation platform. The heading rotation mechanism is used to drive the follow-up observation platform to rotate around the heading axis, and the pitch rotation platform is used to drive the second observation unit to rotate around the pitch axis.

3. The dual-level visual tracking pose publisher according to claim 1, characterized in that, The mobile tag and reference mark components are both luminescent marks or reflective marks with identification, or target structures containing coded patterns, dot matrix arrangements, and time-sequential luminescence features.

4. The dual-level visual tracking pose publisher according to claim 2, characterized in that, It also includes a solution and release mechanism, which specifically includes: a data acquisition unit, a timing control unit, a tracked object identity solution unit, a pose solution unit, and a message encapsulation and message release unit; The data acquisition unit is used to receive image data output by the fixed observation unit, the second observation unit and / or the wide-angle auxiliary observation unit, and to preprocess the received image data; The timing control unit is used to provide a unified time reference for the observation results of the first-level visual observation link and the observation results of the second-level visual observation link, and to generate timestamp information. The tracked object identity calculation unit is used to identify and calculate the identity of the mobile tag based on the image data collected by the second observation unit and / or the wide-angle auxiliary observation unit; The pose calculation unit is used to calculate the six-degree-of-freedom pose of the tracked component in the reference coordinate system based on the observation results of the first-level visual observation link and the second-level visual observation link. The message encapsulation and message publishing unit is used to receive timestamp information, identity information, six-degree-of-freedom pose and state information, and encapsulate them into standardized pose messages for external publication.

5. The dual-level visual tracking pose publisher according to claim 4, characterized in that, The second-level visual observation link also includes a wide-angle auxiliary observation unit. The wide-angle auxiliary observation unit is set on the follow-up observation platform and is used to perform wide-field observation of the moving tag and output coarse positioning results or field-of-view guidance results when the moving tag initially enters, moves rapidly, deviates from the field of view of the second observation unit, or reappears after being lost for a short time. This is to assist the second observation unit in recaptured the moving tag, or to assist the pose calculation unit in performing auxiliary correction or state judgment.

6. The dual-level visual tracking pose publisher according to claim 1, characterized in that, The standardized pose message specifically includes a target identity identifier, a unified timestamp, a six-degree-of-freedom pose and state variables, and further includes one or more of the following: message header, protocol version, message type, message length, device identifier, coordinate system identifier, confidence level, serial number, calibration version and verification field; When encapsulating the standardized pose message, the message encapsulation and message publishing unit binds the target identity identifier, unified timestamp, six-degree-of-freedom pose, and state variables as associated fields in the same message instance, so that each standardized pose message has a corresponding target, time, and pose.

7. The dual-level visual tracking pose publisher according to claim 6, characterized in that, When the message encapsulation and message publishing unit encapsulates the standardized pose message, it generates the standardized pose message only when the target identity is valid, the six-degree-of-freedom pose is valid, the unified timestamp is valid, and the current solution result meets the preset confidence conditions or state conditions. If any of the above conditions are not met, output a message with abnormal state variables, or output only a state message instead of the standardized pose message at that moment.

8. A method for publishing a pose generator based on any one of claims 1 to 7, characterized in that, Includes the following steps: S1. The reference marker component is observed through the first-level visual observation link, and the mobile tag with identification set on the tracked component is observed through the second-level visual observation link. The observed image data is transmitted to the data acquisition unit for preprocessing. S2. Identify the mobile tag based on the image data collected by the second-level visual observation link to obtain the target identity identifier corresponding to the currently tracked component; S3. Based on the observation results of the first-level visual observation link, obtain the first pose result of the second observation unit or the follow-up observation part relative to the reference coordinate system; S4. Based on the observation results of the moving tag by the second-level visual observation link, obtain the second pose result of the moving tag or the tracked component relative to the second observation unit; S5. Based on the first pose result obtained in step S3, the second pose result obtained in step S4, and the pre-calibrated installation relationship, perform a fusion calculation of the six-degree-of-freedom pose of the tracked component in the reference coordinate system; S6. The target identity, unified timestamp, six-degree-of-freedom pose and state variables corresponding to the same tracked component and the same solution time under the same time base are encapsulated into a standardized pose message through the message encapsulation and message publishing unit, and the standardized pose message is sent to the external device according to the preset publishing method.

9. The publishing method of the two-level visual tracking pose publisher according to claim 8, characterized in that, In step S3, the pose of the second observation unit in the reference coordinate system satisfies: ; In the formula, This represents the pose of the reference marker component relative to the reference coordinate system at time k. This indicates the pre-calibrated installation relationship between the reference marker assembly and the second observation unit. This represents the pose of the second observation unit relative to the reference coordinate system at time k.

10. The publishing method of the two-level visual tracking pose publisher according to claim 8, characterized in that, In step S5, the pose of the tracked component in the reference coordinate system satisfies: ; In the formula, This represents the pose of the moving label relative to the second observation unit at time k. This indicates the pre-defined installation relationship between the mobile tag and the tracked component. This represents the six-DOF pose of the tracked component in the reference coordinate system at time k. This represents the pose of the second observation unit relative to the reference coordinate system at time k. When the observation results of the second observation unit are valid, the solution results of the second observation unit are used as the main pose results. When the confidence of the observation results of the second observation unit is insufficient or the target deviates from the field of view, the observation results of the wide-angle auxiliary observation unit are used for coarse positioning, auxiliary correction or recovery tracking.

11. The publishing method of the two-level visual tracking pose publisher according to claim 10, characterized in that, The six-degree-of-freedom pose of the tracked component in the reference coordinate system is represented as follows: ; In the formula, This indicates the position parameters of the tracked component in the reference coordinate system. The attitude parameters of the tracked component in the reference coordinate system are represented by a rotation matrix, Euler angles, quaternions, or direction cosine matrix.

12. The publishing method of the dual-level visual tracking pose publisher according to claim 8, characterized in that, The standardized pose message in step S6 is represented as follows: ; ; In the formula, This indicates at least one of the following header fields: message header, protocol version, message type, and message length. This represents the set of message body fields or payload fields for the k-th standardized pose message. This represents the validation field. Indicates equipment identification. Indicates the target's identity identifier. Indicates a unified timestamp. This represents the target's six-degree-of-freedom pose in the reference coordinate system. Represents state variables. Indicates the coordinate system identifier. Indicates the confidence level. Indicates the serial number. Indicates the calibration version.

13. The publishing method of the two-level visual tracking pose publisher according to claim 8, characterized in that, The preset publishing methods include at least one of broadcast publishing, multicast publishing, point-to-point publishing, polling response publishing, and event-triggered publishing.

Images