Pose calculation method, electronic device, and storage medium
Patent Information
- Application Number
- CN202610948145.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-06-29
- Publication Date
- 2026-07-24
AI Technical Summary
Traditional pose calculation methods involve numerous iterations and have low computational efficiency during monocular vision tracking with a light pen, making it difficult to meet real-time requirements, especially when there is environmental interference or the light pen moves at high speeds.
The predicted position in the current frame image is determined based on the position of the reference marker on the light pen in the previous frame image. Initial matching is performed, and the initial pose parameters are optimized by utilizing the coordinate system relationship between the light pen and the camera. This reduces the calculation range to a small area near the marker and reduces the iteration process.
It improves the efficiency of pose calculation, meets the real-time and high-precision requirements of monocular vision light pen tracking, and reduces computational complexity.
Smart Images
Figure CN122453932A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of visual inspection, and in particular to pose calculation methods, electronic devices, and storage media. Background Technology
[0002] In monocular vision tracking with a light pen, calculating the pen's pose is a prerequisite for accurate localization. Traditional pose calculation methods employ the perspective n-point method. n This method combines the Point-in-Place (PnP) algorithm and the Random Sample Consensus (RANSAC) algorithm to find the correspondence between two-dimensional (2D) points in an image and three-dimensional (3D) points in space through random sampling, and then determines the pose of the light pen based on the correspondence. However, determining the correspondence between the 2D points in the image and the 3D points in space requires multiple iterations, resulting in long computation time. The computational efficiency drops significantly, especially when the number of 2D points to be determined is large, environmental interference exists, or the light pen moves at high speeds, making it difficult to meet real-time requirements. Summary of the Invention
[0003] In view of this, embodiments of this application provide a pose calculation method, an electronic device, and a storage medium to solve the problems of high number of pose calculation iterations and low calculation efficiency in the prior art.
[0004] The first aspect of this application provides a pose calculation method, including: Based on the positions of multiple reference markers on the light pen in the previous frame image, determine the predicted positions of the multiple reference markers in the current frame image; Based on the positions of multiple first markers in the current frame image, a first marker matching each predicted position is determined, wherein the distance between the predicted position and the matched first marker is less than a preset value; The initial pose parameters of the light pen are calculated based on the position of the first marker matched with each predicted position and the three-dimensional coordinates of the reference marker corresponding to each predicted position in the coordinate system of the light pen. Based on the three-dimensional coordinates of the second marker on the light pen in the coordinate system of the light pen and the actual position of the second marker in the current frame image, the initial pose parameters are optimized to obtain the target pose parameters of the light pen.
[0005] In one embodiment, the initial pose parameters are optimized based on the three-dimensional coordinates of the second marker on the light pen in the coordinate system of the light pen and the actual position of the second marker in the current frame image to obtain the target pose parameters of the light pen, including: The projection position of each second marker in the current frame image is determined based on the initial pose parameters; Based on the distance between the projected position of each second marker and the actual position of each second marker in the current frame image, valid markers are filtered from a plurality of second markers; Based on the three-dimensional coordinates of the effective marker in the coordinate system of the light pen and the actual position of the effective marker in the current frame image, the initial pose parameters are optimized to obtain the target pose parameters of the light pen.
[0006] In one embodiment, the initial pose parameters are optimized based on the three-dimensional coordinates of the effective marker in the coordinate system of the light pen and the actual position of the effective marker in the current frame image to obtain the target pose parameters of the light pen, including: If the number of effective markers is greater than or equal to a preset number, the initial pose parameters are optimized based on the three-dimensional coordinates of the effective markers in the coordinate system of the light pen and the actual position of the effective markers in the current frame image, so as to obtain the target pose parameters of the light pen.
[0007] In one embodiment, the method further includes: If the number of valid markers is less than the preset number, the first marker that matches each predicted location is re-determined.
[0008] In one embodiment, the initial pose parameters are optimized based on the three-dimensional coordinates of the second marker on the light pen in the coordinate system of the light pen and the actual position of the second marker in the current frame image to obtain the target pose parameters of the light pen, including: Based on the three-dimensional coordinates of the second marker on the light pen in the coordinate system of the light pen and the actual position of the second marker in the current frame image, the initial pose parameters are optimized using the LM algorithm to obtain the target pose parameters of the light pen.
[0009] In one embodiment, determining the first marker matching each predicted location based on the positions of multiple first markers in the current frame image includes: Based on the distance between the positions of the multiple first markers and each predicted position, a marker sequence corresponding to each predicted position is generated; For each predicted location, the first marker matching the predicted location is determined from the marker sequence in order of distance priority.
[0010] In one embodiment, for each predicted location, determining the first marker matching the predicted location from the marker sequence according to distance priority includes: For each predicted location, the first marker closest to the predicted location is determined from the marker sequence according to distance priority; If the first markers corresponding to each of the predicted positions are different, then the first marker closest to the predicted position is taken as the first marker that matches the predicted position.
[0011] In one embodiment, determining the predicted positions of multiple reference markers in the current frame image based on their positions in the previous frame image includes: The positions of the multiple reference markers in the previous frame image are respectively used as the predicted positions of the multiple reference markers in the current frame image; or, Based on the positions of the multiple reference markers in the previous frame image and the motion information of the light pen, the predicted positions of the multiple reference markers in the current frame image are determined.
[0012] A second aspect of this application provides a pose calculation device, including: The prediction module is used to determine the predicted positions of multiple reference markers in the current frame image based on the positions of multiple reference markers on the light pen in the previous frame image; The matching module is used to determine the first marker that matches each predicted position based on the position of multiple first markers in the current frame image, wherein the distance between the predicted position and the matched first marker is less than a preset value; The calculation module is used to calculate the initial pose parameters of the light pen based on the position of the first marker matched with each predicted position and the three-dimensional coordinates of the reference marker corresponding to each predicted position in the coordinate system of the light pen. The output module is used to optimize the initial pose parameters based on the three-dimensional coordinates of the second marker on the light pen in the coordinate system of the light pen and the actual position of the second marker in the current frame image, so as to obtain the target pose parameters of the light pen.
[0013] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the pose calculation method as described in the first aspect above.
[0014] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the pose calculation method as described in the first aspect above.
[0015] The fifth aspect of this application provides a computer program product that, when run on an electronic device, causes the electronic device to execute the pose calculation method described in any one of the first aspects.
[0016] The beneficial effects of this application embodiment compared with the prior art are as follows: the predicted position in the current frame image is determined based on the position of the reference marker in the previous frame image; then, the predicted position and the first marker whose distance from the predicted position is less than a preset value are initially matched; then, the initial pose of the light pen is calculated based on the position of the first marker obtained from the initial matching and the three-dimensional coordinates of the corresponding reference marker; then, the initial pose is optimized based on the three-dimensional coordinates of the second marker and the actual position of the second marker in the current frame image, so that the calculation range is reduced to a small range near each marker in the image, thereby enabling the optimization process to converge quickly, thus avoiding a complex iterative process and improving the pose calculation efficiency. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.
[0018] Figure 1 This is a schematic diagram illustrating the implementation process of a pose calculation method provided in an embodiment of this application; Figure 2 This is a schematic diagram of the pose calculation device provided in an embodiment of this application; Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation
[0019] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.
[0020] It should be understood that, when used in this specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.
[0021] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.
[0022] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.
[0023] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."
[0024] Furthermore, in the description of this application, the terms "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.
[0025] The pose calculation method provided in this application is illustrated below.
[0026] Please see the appendix Figure 1 The pose calculation method provided in one embodiment of this application includes steps S101 to S104.
[0027] S101: Determine the predicted positions of the multiple reference markers in the current frame image based on their positions in the previous frame image.
[0028] The number of reference markers is at least three. For example, three markers fixed on the light pen (such as a circular light source) can be pre-selected as reference markers. The camera captures images of the light pen to obtain the previous frame image and the current frame image. Both the previous frame image and the current frame image include images of multiple reference markers.
[0029] In one embodiment, the matching of the two-dimensional coordinates of the marker on the light pen in the image and the three-dimensional coordinates in space has been completed in the previous frame image. Due to the continuity of the light pen movement, the position of the marker changes little between adjacent frames. Therefore, the position (two-dimensional coordinates) of the reference marker in the previous frame image can be directly used as the predicted position of the reference marker in the current frame image to improve the calculation speed.
[0030] In another embodiment, the position of the reference marker in the current frame image can be predicted based on the position of the reference marker in the previous frame image and combined with the motion information of the light pen, so as to obtain the predicted position of the reference marker in the current frame image.
[0031] For example, the motion speed and direction of the reference marker can be determined based on its position in the previous two frames and the acquisition time of those frames. Then, based on the reference marker's position in the previous frame, its motion speed and direction, the predicted position of the reference marker in the current frame can be obtained. Alternatively, the pose change of the light pen can be determined based on the pose calculated from the previous two frames. Then, based on the light pen's pose and pose change calculated from the previous frame, the predicted pose of the light pen can be obtained. Finally, based on the reference marker's 3D coordinates in the light pen's coordinate system, camera intrinsics, and the predicted pose, the predicted position of the reference marker in the current frame can be determined.
[0032] It is understandable that if the current frame image is not the first frame image, the above method is used to obtain the predicted position of the reference marker in the current frame image. If the current frame image is the first frame image, the pose of the light pen can be determined by combining the PnP algorithm and the RANSAC algorithm using an iterative calculation method.
[0033] S102: Based on the positions of multiple first markers in the current frame image, determine the first marker that matches each predicted position, wherein the distance between the predicted position and the matched first marker is less than a preset value.
[0034] Specifically, the first marker is a marker located at a specified position on the light pen (e.g., an illuminated button or reflective marker at that position). The current frame image is detected to obtain the position (two-dimensional coordinates) of the first marker within the current frame image. For example, a pre-defined marker extraction algorithm (e.g., thresholding, connected component extraction, sub-pixel localization algorithm) can be used to extract the first marker from the current frame image, thus obtaining its position. Alternatively, the current frame image can be input into a pre-trained detection model to obtain the position of the first marker in the current frame image, output by the detection model.
[0035] For each predicted location, select one of the multiple first markers as the first marker that matches the predicted location.
[0036] In one embodiment, for each predicted location, the distance between the predicted location and the location of each first marker is calculated to generate a marker sequence corresponding to the predicted location. The first markers in the marker sequence are sorted in ascending order of distance. For each predicted location, the first marker closest to the predicted location is selected from the marker sequence according to distance priority (ascending order of distance) as the first marker matching the predicted location, thereby improving the accuracy of the subsequently obtained initial pose.
[0037] In one embodiment, for each predicted location, a first marker closest to the predicted location is selected from the marker sequence according to distance priority, and it is determined whether the first markers corresponding to each predicted location are the same. If the first markers corresponding to each predicted location are all different, the first marker closest to the predicted location is taken as the first marker matching the predicted location, thereby avoiding the situation where the same reference marker corresponds to multiple markers in the image, ensuring the uniqueness and rationality of the initial matching, and improving the reliability and stability of the subsequently obtained initial pose parameters. If there are identical first markers among the first markers corresponding to each predicted location, for the predicted locations with identical first markers, the next first marker is selected from the marker sequence according to distance priority. If the next first marker is different from the first markers corresponding to other predicted locations, the next first marker is taken as the first marker matching the predicted location.
[0038] In another embodiment, the current frame image is detected to obtain the position of each first marker in the current frame image, and the image features of each first marker (e.g., area size, contour shape, etc.) are obtained simultaneously. For each predicted position, a marker sequence corresponding to the predicted position is generated according to the position of each first marker in ascending order of distance. The first marker closest to the predicted position is selected from the marker sequence according to distance priority. The image features of the reference marker corresponding to the predicted position are obtained from the pre-stored reference marker. If the similarity between the image features of the first marker and the image features of the reference marker is greater than or equal to a preset value, then the first marker is used as the first marker matching the predicted position, thereby further improving the matching accuracy. If the similarity between the image features of the first marker and the image features of the reference marker is less than the preset value, then the next first marker is selected from the marker sequence according to distance priority, and the similarity between the image features of the next first marker and the image features of the reference marker is calculated.
[0039] S103: Calculate the initial pose parameters of the light pen based on the position of the first marker matching each predicted position and the three-dimensional coordinates of the reference marker corresponding to each predicted position in the coordinate system of the light pen.
[0040] The initial pose parameters of the light pen also represent the transformation relationship between the coordinate system of the light pen and the coordinate system of the camera, including the transformation matrix and translation vector of the coordinate system of the light pen relative to the coordinate system of the camera. For each predicted position, the three-dimensional coordinates of the reference marker corresponding to the predicted position in the coordinate system of the light pen and the two-dimensional coordinates of the first marker matching the predicted position are used as a set of calculation data. Multiple predicted positions correspond to multiple sets of calculation data. The PNP algorithm is used to solve multiple sets of calculation data to obtain the initial pose parameters of the light pen.
[0041] S104: Based on the three-dimensional coordinates of the second marker on the light pen in the coordinate system of the light pen and the actual position of the second marker in the current frame image, optimize the initial pose parameters to obtain the target pose parameters of the light pen.
[0042] The second marker includes the reference marker and other markers on the light pen. The second marker may correspond to all of the first markers or to some of the first markers.
[0043] In one embodiment, based on the three-dimensional coordinates of the second marker in the coordinate system of the light pen and its initial pose parameters, combined with the camera calibration parameters, the second marker is projected onto the current frame image to obtain the projected position of the second marker in the current frame image. Valid markers are then selected from multiple second markers based on the distance between the projected position of each second marker and its actual position in the current frame image. For example, the current frame image is detected to obtain the positions of multiple second markers in the current frame image. For each projected position, the second marker closest to the projected position is determined based on the positions of the multiple second markers in the current frame image. If the distance between the second marker closest to the projected position and the projected position is less than a set threshold (e.g., 0.001), it indicates that the projected position is relatively close to the actual position, and the second marker is determined to be a valid marker. By selecting valid markers, abnormal marker data caused by noise, occlusion, and mismatches can be eliminated, improving the accuracy of subsequent target pose parameter calculations.
[0044] After determining the valid markers, for each projection position, the three-dimensional coordinates of the second marker (which is a valid marker) corresponding to the projection position are determined in the coordinate system of the light pen. At the same time, the actual position of the second marker corresponding to the projection position (e.g., the second marker closest to the projection position) in the current frame image is determined. The three-dimensional coordinates and the actual position are used as a set of calculation data. Multiple projection positions correspond to multiple sets of calculation data. The initial pose parameters are optimized based on multiple sets of calculation data to obtain the target pose parameters of the light pen, thereby reducing errors and improving the accuracy of the obtained target pose parameters.
[0045] After determining the effective markers, the number of effective markers in the second marker can also be determined first. If the number of effective markers is greater than or equal to the preset number, it indicates that the initial pose is reliable. Then, based on the three-dimensional coordinates of the effective markers in the coordinate system of the light pen and the actual position of the effective markers in the current frame image, the initial pose parameters are optimized to obtain the target pose parameters of the light pen, thereby reducing the number of iterations and improving computational efficiency.
[0046] If the number of valid markers is less than the preset number, it indicates that the initial pose parameters are inaccurate. A new first marker is then determined for each predicted position, allowing for error correction and retries, thus improving the reliability of the calculation process. For example, the next first marker is selected from the marker sequence according to distance priority as the first marker to match the predicted position. If it is determined that the first markers corresponding to each predicted position are different, the next first marker is used as the first marker to match the predicted position. Based on the position of the newly determined first marker and the three-dimensional coordinates of the reference marker corresponding to the predicted position in the coordinate system of the light pen, the initial pose parameters are re-determined to determine better initial pose parameters.
[0047] In one embodiment, if the number of valid markers is less than a preset number, the next first marker is selected from the marker sequence according to the distance priority order. If the next first marker is the last one in the marker sequence and does not meet the condition that it is different from the first markers corresponding to other predicted positions, or if the initial pose parameters are calculated based on the next first marker and the number of valid markers determined based on the initial pose parameters is still less than the preset number, then the RANSAC algorithm is used to calculate the target pose parameters of the light pen, thereby ensuring that the effective target pose parameters of the light pen can be obtained based on the current frame image.
[0048] In another embodiment, the initial pose parameters can be optimized by using the three-dimensional coordinates of all second markers in the coordinate system of the light pen and their actual positions in the current frame image to obtain the target pose parameters.
[0049] In one embodiment, after determining the initial pose, based on the three-dimensional coordinates of the second marker on the light pen in the coordinate system of the light pen and the actual position of the second marker in the current frame image, combined with the camera calibration parameters (i.e., camera intrinsic parameters), a Levonburg method can be used. The Levenberg-Marquardt (LM) algorithm finely adjusts the rotation and translation parameters with the goal of minimizing the global reprojection error, resulting in optimized rotation and translation parameters, which are the target pose parameters of the light pen. This approach balances the convergence speed and stability of the computation process, thereby improving the accuracy of the obtained target pose parameters.
[0050] In other embodiments, algorithms such as Gauss-Newton's method, gradient descent, Dogleg's algorithm, or conjugate gradient method can be used to optimize the initial pose parameters to obtain the target pose parameters of the light pen.
[0051] In the above embodiments, the three-dimensional coordinates and two-dimensional coordinates of the reference marker are matched using the previous frame image as a reference to obtain the initial pose parameters. Then, the initial pose parameters are optimized based on the three-dimensional coordinates and corresponding two-dimensional coordinates of multiple markers to determine the target pose parameters. This reduces the search range of the marker from the global image to a local area near the projection position, avoiding global traversal and random search, reducing the amount of computation, and improving the pose calculation speed and stability. It can meet the high precision and low latency requirements of monocular vision light pen tracking, dynamic measurement, real-time positioning and other scenarios, and effectively solves the problems of multiple iterations, long time consumption and poor robustness of the traditional PnP algorithm combined with RANSAC algorithm.
[0052] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.
[0053] Corresponding to the pose calculation method described in the above embodiments, Figure 2 A structural block diagram of the pose calculation device provided in the embodiments of this application is shown. For ease of explanation, only the parts related to the embodiments of this application are shown.
[0054] like Figure 2 As shown, the pose calculation device includes: Prediction module 21 is used to determine the predicted position of multiple reference markers in the current frame image based on the position of multiple reference markers on the light pen in the previous frame image; Matching module 22 is used to determine the first marker that matches each predicted position based on the position of multiple first markers in the current frame image, wherein the distance between the predicted position and the matched first marker is less than a preset value; The calculation module 23 is used to calculate the initial pose parameters of the light pen based on the position of the first marker matched with each predicted position and the three-dimensional coordinates of the reference marker corresponding to each predicted position in the coordinate system of the light pen. The output module 24 is used to optimize the initial pose parameters based on the three-dimensional coordinates of the second marker on the light pen in the coordinate system of the light pen and the actual position of the second marker in the current frame image, so as to obtain the target pose parameters of the light pen.
[0055] In one embodiment, the output module 24 is specifically used for: The projection position of each second marker in the current frame image is determined based on the initial pose parameters; Based on the distance between the projected position of each second marker and the actual position of each second marker in the current frame image, valid markers are filtered from a plurality of second markers; Based on the three-dimensional coordinates of the effective marker in the coordinate system of the light pen and the actual position of the effective marker in the current frame image, the initial pose parameters are optimized to obtain the target pose parameters of the light pen.
[0056] In one embodiment, the output module 24 is specifically used for: If the number of effective markers is greater than or equal to a preset number, the initial pose parameters are optimized based on the three-dimensional coordinates of the effective markers in the coordinate system of the light pen and the actual position of the effective markers in the current frame image, so as to obtain the target pose parameters of the light pen.
[0057] In one embodiment, the output module 24 is further configured to: If the number of valid markers is less than the preset number, the first marker that matches each predicted location is re-determined.
[0058] In one embodiment, the output module 24 is specifically used for: Based on the three-dimensional coordinates of the second marker on the light pen in the coordinate system of the light pen and the actual position of the second marker in the current frame image, the initial pose parameters are optimized using the LM algorithm to obtain the target pose parameters of the light pen.
[0059] In one embodiment, the matching module 22 is specifically used for: Based on the distance between the positions of the multiple first markers and each predicted position, a marker sequence corresponding to each predicted position is generated; For each predicted location, the first marker matching the predicted location is determined from the marker sequence in order of distance priority.
[0060] In one embodiment, the matching module 22 is specifically used for: For each predicted location, the first marker closest to the predicted location is determined from the marker sequence according to distance priority; If the first markers corresponding to each of the predicted positions are different, then the first marker closest to the predicted position is taken as the first marker that matches the predicted position.
[0061] In one embodiment, the prediction module 21 is specifically used for: The positions of the multiple reference markers in the previous frame image are respectively used as the predicted positions of the multiple reference markers in the current frame image; or, Based on the positions of the multiple reference markers in the previous frame image and the motion information of the light pen, the predicted positions of the multiple reference markers in the current frame image are determined.
[0062] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.
[0063] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may be a desktop computer, a smart camera, a vision sensor, a mobile terminal, a robot, or other computing device.
[0064] like Figure 3 As shown, the electronic device in this embodiment includes: a processor 11, a memory 12, and a computer program 13 stored in the memory 12 and executable on the processor 11. When the processor 11 executes the computer program 13, it implements the steps described in the above-described pose calculation method embodiment, for example... Figure 1 Steps S101 to S104 are shown. Alternatively, when the processor 11 executes the computer program 13, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 2 The functions of prediction module 21 to output module 24 are shown.
[0065] For example, the computer program 13 may be divided into one or more modules / units, which are stored in the memory 12 and executed by the processor 11 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program 13 in the electronic device.
[0066] Those skilled in the art will understand that Figure 3 This is merely an example of an electronic device and does not constitute a limitation on the electronic device. It may include more or fewer components than illustrated, or combine certain components, or different components. For example, the electronic device may also include input / output devices, network access devices, buses, etc.
[0067] The processor 11 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.
[0068] The memory 12 can be an internal storage unit of the electronic device, such as a hard drive or RAM. The memory 12 can also be an external storage device of the electronic device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 12 can include both internal and external storage units. The memory 12 is used to store the computer program and other programs and data required by the electronic device. The memory 12 can also be used to temporarily store data that has been output or will be output.
[0069] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.
[0070] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.
[0071] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings or direct couplings or communication connections may be through some interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.
[0072] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0073] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0074] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.
[0075] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0076] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.
Claims
1. A pose calculation method, characterized in that, include: Based on the positions of multiple reference markers on the light pen in the previous frame image, determine the predicted positions of the multiple reference markers in the current frame image; Based on the positions of multiple first markers in the current frame image, a first marker matching each predicted position is determined, wherein the distance between the predicted position and the matched first marker is less than a preset value; The initial pose parameters of the light pen are calculated based on the position of the first marker matched with each predicted position and the three-dimensional coordinates of the reference marker corresponding to each predicted position in the coordinate system of the light pen. Based on the three-dimensional coordinates of the second marker on the light pen in the coordinate system of the light pen and the actual position of the second marker in the current frame image, the initial pose parameters are optimized to obtain the target pose parameters of the light pen.
2. The pose calculation method according to claim 1, characterized in that, Based on the three-dimensional coordinates of the second marker on the light pen in the coordinate system of the light pen and the actual position of the second marker in the current frame image, the initial pose parameters are optimized to obtain the target pose parameters of the light pen, including: The projection position of each second marker in the current frame image is determined based on the initial pose parameters; Based on the distance between the projected position of each second marker and the actual position of each second marker in the current frame image, valid markers are filtered from a plurality of second markers; Based on the three-dimensional coordinates of the effective marker in the coordinate system of the light pen and the actual position of the effective marker in the current frame image, the initial pose parameters are optimized to obtain the target pose parameters of the light pen.
3. The pose calculation method according to claim 2, characterized in that, Based on the three-dimensional coordinates of the effective marker in the coordinate system of the light pen and the actual position of the effective marker in the current frame image, the initial pose parameters are optimized to obtain the target pose parameters of the light pen, including: If the number of effective markers is greater than or equal to a preset number, the initial pose parameters are optimized based on the three-dimensional coordinates of the effective markers in the coordinate system of the light pen and the actual position of the effective markers in the current frame image, so as to obtain the target pose parameters of the light pen.
4. The pose calculation method according to claim 3, characterized in that, The method further includes: If the number of valid markers is less than the preset number, the first marker that matches each predicted location is re-determined.
5. The pose calculation method according to any one of claims 1 to 4, characterized in that, Based on the three-dimensional coordinates of the second marker on the light pen in the coordinate system of the light pen and the actual position of the second marker in the current frame image, the initial pose parameters are optimized to obtain the target pose parameters of the light pen, including: Based on the three-dimensional coordinates of the second marker on the light pen in the coordinate system of the light pen and the actual position of the second marker in the current frame image, the initial pose parameters are optimized using the LM algorithm to obtain the target pose parameters of the light pen.
6. The pose calculation method according to claim 1, characterized in that, Based on the positions of multiple first markers in the current frame image, determine the first marker matching each predicted position, including: Based on the distance between the positions of the multiple first markers and each predicted position, a marker sequence corresponding to each predicted position is generated; For each predicted location, the first marker matching the predicted location is determined from the marker sequence in order of distance priority.
7. The pose calculation method according to claim 6, characterized in that, For each predicted location, the first marker matching the predicted location is determined from the marker sequence according to distance priority, including: For each predicted location, the first marker closest to the predicted location is determined from the marker sequence according to distance priority; If the first markers corresponding to each of the predicted positions are different, then the first marker closest to the predicted position is taken as the first marker that matches the predicted position.
8. The pose calculation method according to claim 1, characterized in that, Based on the positions of multiple reference markers on the light pen in the previous frame image, the predicted positions of the multiple reference markers in the current frame image are determined, including: The positions of the multiple reference markers in the previous frame image are respectively used as the predicted positions of the multiple reference markers in the current frame image; or, Based on the positions of the multiple reference markers in the previous frame image and the motion information of the light pen, the predicted positions of the multiple reference markers in the current frame image are determined.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the pose calculation method as described in any one of claims 1 to 8.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the pose calculation method as described in any one of claims 1 to 8.