Pose recognition method, device, readable storage medium and transcranial magnetic therapy instrument

Abstract

The application discloses a pose recognition method for tracking a target, an electronic device, a navigation device, a readable storage medium and a transcranial magnetic therapy instrument, and belongs to the technical field of medical treatment. The pose recognition method comprises the following steps: identifying and determining a mark module matched with a preset parameter in multiple mark modules located in the visual range of a camera based on the preset parameter; and obtaining the pose of the tracking target under the camera based on a first space mapping relationship between the mark module matched with the preset parameter and the camera, and a second space mapping relationship between the mark module matched with the preset parameter and the tracking target provided with the multiple mark modules. The pose recognition method for tracking a target, the electronic device, the navigation device, the readable storage medium and the transcranial magnetic therapy instrument provided by the application can make the tracking target move at any time and at will within the visual range of the camera through the automatic graphic recognition method, and the pose of the tracking target can be obtained without re-registration.

Description

Technical Field

[0001] This invention relates to the field of medical technology, and in particular to a method for recognizing the position of a tracked target, electronic devices, navigation devices, readable storage media, and transcranial magnetic stimulation devices. Background Technology

[0002] With the rapid development of society and the accelerated pace of life and work, modern people face increasing life pressure, leading to a rise in the incidence of mental and neurological diseases. These include: depression, anxiety disorders, binocular emotional disorders, schizophrenia, somatic symptom disorder, sleep disorders, sequelae of cerebral infarction, post-stroke depression, as well as Alzheimer's disease, vascular cognitive impairment, Parkinson's disease, Meige syndrome, essential tremor, hereditary spinocerebellar ataxia, neuropathic pain and migraine, epilepsy, etc.

[0003] Mental illnesses and neurological disorders are often treated and managed by stimulating brain nerves. For example, transcranial magnetic stimulation (TMS) involves placing an energized stimulation coil above the subject's head to stimulate the cerebral cortex (e.g., the motor cortex).

[0004] In 1987, Amassian et al. demonstrated through experiments that the placement direction of the stimulation coil affects the effect of TMS on the cerebral cortex. In 1993, Hoflide et al. applied TMS to the treatment of mental and neurological disorders such as depression, demonstrating that TMS has a certain therapeutic effect on these disorders. In 2005, Edwards et al. demonstrated through experiments that low-intensity repetitive transcranial magnetic stimulation can induce excitability of neurons in the cerebral cortex. In 2007, Joo EY et al. found that long-term, low-frequency repetitive transcranial magnetic stimulation can alleviate the symptoms of epilepsy. In 2015, Ku Y et al. used single-pulse transcranial magnetic stimulation to stimulate the sensory cortex and lateral posterior parietal cortex, finding that this form of TMS has an intervention effect on the brain's cognitive function.

[0005] However, numerous difficulties exist in the clinical application of transcranial magnetic stimulation (TMS) therapy, significantly limiting its application and promotion in the treatment of mental and neurological diseases. The main difficulties include: First, the positioning of the TMS stimulation coil relies heavily on the physician's experience and skill, making it highly subjective; inaccurate coil placement will affect the treatment effect. Second, the structural features of the subject's brain are not visible during coil placement, and individual differences in head structure make the positioning cap insufficiently universal and lacking in precision. Third, each TMS treatment session typically lasts 15 to 30 minutes; even slight head movement during this time will alter the coil's placement. If the subject's head is fixed, muscle contraction and tension as stimulation time increases will cause severe discomfort.

[0006] Therefore, it is indeed necessary to provide a pose recognition method, electronic device, navigation device, readable storage medium, and transcranial magnetic stimulation device that can at least satisfy the requirement that the subject can move a large range of distances during transcranial magnetic stimulation without re-registration of the tracking target. Summary of the Invention

[0007] To address at least one of the aforementioned problems and deficiencies in the prior art, the present invention provides a pose recognition method for tracking a target, an electronic device, a navigation device, a readable storage medium, and a transcranial magnetic stimulation therapy device. The technical solution is as follows:

[0008] One object of the present invention is to provide a pose recognition method for tracking targets.

[0009] Another object of the present invention is to provide an electronic device.

[0010] Another object of the present invention is to provide a navigation device.

[0011] Another object of the present invention is to provide a readable storage medium.

[0012] Another object of the present invention is to provide a transcranial magnetic stimulation therapy device.

[0013] According to one aspect of the present invention, a pose recognition method for tracking a target is provided, the pose recognition method comprising the following steps:

[0014] Based on preset parameters, identify and determine the marker module that matches the preset parameters among multiple marker modules located within the camera's field of view;

[0015] Based on the first spatial mapping relationship between the marker module matched with preset parameters and the camera, and the second spatial mapping relationship between the marker module matched with preset parameters and the tracking target equipped with the multiple marker modules, the pose of the tracking target under the camera is obtained.

[0016] Specifically, identifying and determining at least one marker module that matches the preset parameters among multiple marker modules located within the camera's field of view based on preset parameters includes:

[0017] The identification elements on the marker module detected by the camera are compared with preset parameters to obtain marker modules that match the preset parameters;

[0018] Based on the determined marker module that matches the preset parameters, obtain the first spatial mapping relationship between the marker module and the camera and the second spatial mapping relationship between the marker module and the tracking target.

[0019] Preferably, the preset parameters include at least the identification elements of all the multiple marking modules.

[0020] Furthermore, the identification element includes the color and / or graphic unit of the marking module, and the preset parameters include color parameters and / or shape parameters corresponding to the identification element of the marking module.

[0021] Specifically, the identification elements on the marker module detected by the camera are compared with preset parameters to obtain marker modules that match the preset parameters, including:

[0022] When the recognition element is color, the color on the marker module recognized by the camera is compared with the color parameters to obtain a marker module that matches the color parameters; and / or

[0023] When the identified element is a graphic unit, the graphic unit on the marker module identified by the camera is compared with the shape parameters to obtain a marker module that matches the shape parameters.

[0024] Furthermore, the shape parameters include unit styles, which include 3D styles and 2D styles.

[0025] The process of comparing the graphic units on the marker module detected by the camera with shape parameters to obtain a marker module that matches the shape parameters includes:

[0026] The graphic units on the marker module identified by the camera are compared with a preset unit style to obtain a marker module that matches the unit style.

[0027] Specifically, each of the multiple marking modules is provided with a graphic, and the graphic is composed of at least one graphic element.

[0028] The graphic unit is at least one graphic element in the graphic, and the planar style corresponds to all the graphic units on the plurality of marker modules.

[0029] Specifically, the type of the graph includes at least one of the following: checkerboard pattern, circular array pattern, grid pattern, ChArUco pattern, and deep learning graph pattern.

[0030] The circular array patterns include symmetrical and asymmetrical spot patterns.

[0031] The graphic units between adjacent marker modules can be arranged as any one or any combination of different graphic types, different background colors, and different foreground colors.

[0032] Furthermore, when at least two of the multiple marker modules have the same type of graphics, and all graphic elements in the marker modules with the same type of graphics are arranged differently, the unit style includes graphic layout styles with all different arrangements.

[0033] Specifically, when at least two of the multiple marker modules have the same type of graphics, and all graphic elements in the marker modules with the same type of graphics are arranged differently, the graphic units on the marker modules recognized by the camera are compared with a preset unit style to obtain a marker module that matches the unit style, including:

[0034] The shape of at least one graphic element on the same marker module recognized by the camera is compared with the shape of all graphic elements in at least one preset graphic element, and the layout style of all graphic elements on the same marker module recognized by the camera is compared with the preset layout style to obtain a marker module that matches the preset shape and the preset layout style.

[0035] Furthermore, the color parameters include background color parameters and foreground color parameters.

[0036] The background and foreground colors of the marking module are different colors.

[0037] When the recognition element is color, the color on the marker module recognized by the camera is compared with the color parameters to obtain a marker module that matches the color parameters, including:

[0038] The background color of the marker module detected by the camera is compared with the background color parameter, and the foreground color of the marker module detected by the camera is compared with the foreground color parameter, so as to obtain a marker module that matches both the background color parameter and the foreground color parameter.

[0039] Preferably, when at least two of the multiple marker modules within the camera's field of view match the preset parameters,

[0040] The marking modules are evaluated based on the pose of the recognized elements on the same marking module in at least two marking modules, the distance between the plane with the recognized elements on the marking module and the camera, and the size of the recognized elements as recognized by the camera, in order to select the optimal marking module.

[0041] Based on the first spatial mapping relationship between the optimal marking module and the camera, and the second spatial mapping relationship between the optimal marking module and the tracking target, the pose of the tracking target under the camera is obtained.

[0042] More preferably, the orientation of the identified element is the angle between the plane on the marking module containing the identified element and the optical axis of the camera.

[0043] The size of the element recognized by the camera is the ratio between the actual area of ​​the element recognized by the camera and the area of ​​the image.

[0044] Specifically, the marking modules are evaluated based on the pose of the recognition element on the same marking module in the at least two marking modules, the distance between the plane with the recognition element on the marking module and the camera, and the size of the recognition element recognized by the camera, in order to select the optimal marking module, including:

[0045] Based on their respective weights, the angle, distance, and scale of the identified elements on all the at least two marking modules are used to obtain the angle evaluation value, distance evaluation value, and size evaluation value of the identified elements on each marking module.

[0046] The comprehensive evaluation value of the same identification element is obtained by adding the angle evaluation value, distance evaluation value and size evaluation value together;

[0047] Sort the comprehensive evaluation values ​​of the recognition elements of all the at least two labeling modules in descending order, and determine the labeling module corresponding to the largest comprehensive evaluation value as the optimal labeling module.

[0048] Furthermore, based on the first spatial mapping relationship, the second spatial mapping relationship, and the real-time position of the marker module identified by the camera, the pose of the tracked target under the camera is obtained.

[0049] Specifically, when the position of the tracked target changes, a marker module that is recognized by the camera in real time and matches the preset parameters is determined based on preset parameters;

[0050] Based on the first spatial mapping relationship between all the marker modules identified by the camera in real time and the camera, the second spatial mapping relationship between the marker modules identified by the camera in real time and the tracking target, and the real-time position of the marker module, the real-time pose of the tracking target under the camera is obtained.

[0051] According to another aspect of the present invention, an electronic device is provided, wherein,

[0052] The electronic device includes a memory and at least one processor. The memory is communicatively connected to the at least one processor. The memory stores programs or instructions. When the programs or instructions are executed by the at least one processor, the electronic device is used to implement the pose recognition method for tracking targets as described above.

[0053] According to another aspect of the present invention, a navigation device is provided, wherein,

[0054] The navigation device includes a camera and an electronic device connected to the camera signal, wherein the electronic device is the one described above.

[0055] According to another aspect of the present invention, a readable storage medium is provided, wherein,

[0056] The readable storage medium stores a program or instructions that, when executed by a processor, perform the pose recognition method for tracking targets as described above.

[0057] According to another aspect of the present invention, a transcranial magnetic stimulation (TMS) device is provided, wherein,

[0058] The transcranial magnetic stimulation device includes a coil and a main body connected to the coil. The main body is provided with the electronic device or the readable storage medium. The electronic device is the aforementioned electronic device, and the readable storage medium is the readable storage medium according to the above claims.

[0059] The target pose recognition method, electronic device, navigation device, readable storage medium, and transcranial magnetic stimulation device according to the present invention have at least one of the following advantages:

[0060] (1) The pose recognition method, electronic device, navigation device, readable storage medium and transcranial magnetic therapy device for tracking targets provided by the present invention use a structured light camera to automatically recognize the image, so that the tracking target can move at any time and at will within the field of view of the camera, and the pose of the tracking target can be obtained without re-registration;

[0061] (2) The pose recognition method, electronic device, navigation device, readable storage medium and transcranial magnetic therapy device provided by the present invention increase the range of camera recognition of the tracked target through the design of multiple marking modules, so that the tracked target can be moved and rotated in a larger range during navigation, control and other processes;

[0062] (3) The pose recognition method, electronic device, navigation device, readable storage medium and transcranial magnetic therapy device provided by the present invention can automatically recognize the image in real time, so that the camera can automatically select and switch the marker module to be tracked and recognized during the movement and rotation of the target, so as to track the target in real time. Attached Figure Description

[0063] These and / or other aspects and advantages of the present invention will become apparent and readily understood from the following description of preferred embodiments taken in conjunction with the accompanying drawings, in which:

[0064] Figure 1 This is a flowchart of a target pose recognition method according to an embodiment of the present invention;

[0065] Figure 2A yes Figure 1 The graphic on the marker module shown is a view with a symmetrical dot pattern;

[0066] Figure 2B yes Figure 1 The graphic on the marker module shown is a view with an asymmetrical spot pattern;

[0067] Figure 3 This is a schematic diagram of the structure of a transcranial magnetic stimulation therapy device according to another embodiment of the present invention. Detailed Implementation

[0068] The technical solution of the present invention will be further described in detail below through embodiments and in conjunction with the accompanying drawings. In this specification, the same or similar reference numerals indicate the same or similar components. The following description of the embodiments of the present invention with reference to the accompanying drawings is intended to explain the overall inventive concept of the present invention and should not be construed as a limitation thereof.

[0069] In today's society, people pay far less attention to mental illnesses than to organ diseases.

[0070] With the rapid development of medical imaging and medical image processing technologies, image-guided surgical systems have emerged. These systems utilize three-dimensional reconstruction models of medical images to enable doctors to intuitively and accurately analyze the structure of organs or tissues and their surrounding tissues. This can assist doctors in treating mental illnesses and improve their efficiency in diagnosing and treating mental illnesses.

[0071] Image-guided surgery systems can use intraoperative images of the patient and three-dimensional models of the lesion and surrounding tissues to guide clinical procedures in real time. During the surgical procedure, image-guided surgery systems can accurately display the patient's tissue anatomy and the details of the three-dimensional space surrounding the lesion.

[0072] Image-guided surgery systems can also use medical imaging technology to scan head images for segmentation and three-dimensional reconstruction to create a three-dimensional head model containing brain tissue. Stimulation targets are planned on the reconstructed three-dimensional brain model, and then image registration technology is used to map the targets on the three-dimensional brain model onto the patient's head during surgery, thereby assisting doctors in locating the targets.

[0073] Image-guided surgery systems utilize optically assisted navigation systems, which are currently the most representative TMS navigation systems. In 2008, Lars Matthaüs et al. combined the Polaris Spectra optical tracking device with the Adept Viper s850 six-axis robot to create a TMS robotic therapy system. The TMS robotic therapy system uses a robot to hold the stimulation coils, and the optical tracking device is fixed by a support. A marker is fixed to the subject's head to facilitate the optical tracking device's positioning of the head coordinates. Currently, there are no publicly available research results on robotic systems used for transcranial magnetic stimulation (TMS) therapy in China. Research on medical robotics technology started relatively late, but is developing rapidly.

[0074] In recent years, with the development of artificial intelligence and the maturity of face matching technology, point cloud matching algorithms have gradually been introduced into navigation. However, due to problems such as low accuracy, there are currently no very mature high-precision commercial medical navigation systems (such as TMS navigation systems, transcranial magnetic stimulation devices, etc.).

[0075] While optical navigation systems can visualize the transcranial magnetic stimulation (TMS) process and improve the accuracy of coil positioning to some extent, current navigation methods are relatively cumbersome. For example, subject registration is required before treatment begins, and re-registration is necessary if the camera is accidentally touched, which is unsuitable for subjects with autism or other conditions.

[0076] In view of this, the present invention provides a novel method for pose recognition of a tracking target.

[0077] See Figure 1 This paper illustrates a pose recognition method for a tracking target according to an embodiment of the present invention. The pose recognition method includes the following steps:

[0078] Based on preset parameters, identify and determine the marker module that matches the preset parameters among multiple marker modules located within the camera's field of view;

[0079] Based on the first spatial mapping relationship between the marker module and the camera, and the second spatial mapping relationship between at least one marker module and the tracking target with multiple marker modules, the pose of the tracking target under the camera is obtained.

[0080] Those skilled in the art will understand that the target pose recognition method provided by this invention can obtain the target pose in other coordinate systems. For example, it can obtain the target pose in a marker coordinate system or in a visualization model. In other words, those skilled in the art can obtain the target pose in the desired coordinate system or space as needed using the method or principle of this invention. This example is merely illustrative and should not be construed as a limitation of the invention.

[0081] Those skilled in the art will also understand that the tracking target can be a specified object, such as a target point of the subject, facial feature information of the subject, a magnetic stimulation coil, a treatment chair, a treatment bed, or a part of the subject's body, etc., and can also be any or all objects / human bodies within the camera's field of view. This example is merely illustrative and should not be construed as a limitation of the invention.

[0082] In one example, the camera's field of view is the camera's angle of view.

[0083] The camera can be a monochrome-depth camera or an RGB-D (color-depth) camera. Specifically, it can be one or more of the following: an RGB structured light camera, a triangulation camera, an RGB-TOF camera, an infrared depth camera, a stereo vision camera, and a 3D camera module.

[0084] In one example, the tracking target can be a stimulation module that transmits magnetic, electrical, or optical stimulation, such as a stimulation coil or positioning cap. Of course, those skilled in the art will understand that the tracking target can also be designed as the subject's head, arm, heart, or any part of the human body to be tracked.

[0085] In one example, multiple marker modules are deployed on the tracked target. These marker modules can be two, three, five, or more. The marker modules are arranged with spacing between them. Of course, those skilled in the art will understand that when multiple marker modules are arranged with spacing between them, they do not need to form overlapping areas. Alternatively, at least two marker modules can partially overlap, as long as it does not affect camera recognition.

[0086] In one example, the cross-sectional shape of the marker module can be designed as L-shaped, wedge-shaped, triangular, rectangular, etc.

[0087] In one example, there can be one, two, or more marker modules that match the preset parameters, but the maximum number should be less than or equal to the number of marker modules set on the tracking target.

[0088] In one example, the pose of the tracked target under the camera is obtained based on the first spatial mapping relationship, the second spatial mapping relationship, and the real-time position of the marker module identified by the camera. Those skilled in the art will understand that the target pose recognition method provided by this invention can obtain the real-time pose of the tracked target in other coordinate systems. For example, it can obtain the real-time pose of the tracked target in the marker coordinate system, or it can obtain the real-time pose of the tracked target in the visualization model. In other words, those skilled in the art can obtain the real-time pose of the tracked target in the desired coordinate system or desired space according to the method or principle of this invention. This example is merely illustrative and should not be construed as a limitation of the invention.

[0089] In one example, when the position of the tracked target changes, a marker module that is recognized by the camera in real time and matches the preset parameters is determined based on preset parameters.

[0090] Based on the first spatial mapping relationship between all the marker modules identified by the camera in real time and the camera, the second spatial mapping relationship between the marker modules identified by the camera in real time and the tracking target, and the real-time position of the marker module, the real-time pose of the tracking target under the camera is obtained.

[0091] The following example, using a magnetic stimulation coil as the tracking target and obtaining the target's pose in the camera coordinate system, illustrates the basic principles and steps of this invention. Those skilled in the art should understand that when the tracking target is an object other than the magnetic stimulation coil, the basic principles and steps are largely the same as the pose recognition method of this invention, and will not be elaborated upon here. Furthermore, those skilled in the art should also understand that when obtaining a pose other than the camera coordinate system, it is sufficient to obtain the spatial transformation relationship (i.e., spatial mapping relationship) between the pre-obtained space and the space of the tracking target. The basic principle of this is largely the same as the pose recognition method of this invention, and will not be elaborated upon here.

[0092] During use, the magnetic stimulation coils first need to be registered to establish a primary spatial mapping relationship between each of the multiple marker modules and the camera. Since each marker module and the magnetic stimulation coil have a rigid connection, this spatial mapping relationship is fixed. In one example, the primary spatial mapping relationship between each marker module and the camera is set in matrix form; similarly, the secondary spatial mapping relationship between each marker module and the magnetic stimulation coil can also be set in matrix form.

[0093] In one example, the first spatial mapping relationship and the second spatial mapping relationship can be spatial transformation matrices (i.e., spatial transformation matrices). Spatial transformations (i.e., spatial transformations) include translation, rotation, shearing, scaling, etc.

[0094] In one example, each marker module is equipped with a recognition element for camera identification. The marker module has a background color and a foreground color, and the background and foreground colors are different colors. For example, the foreground color can be designed as black, and the background color as white. Of course, the foreground color can be designed as white, the background color as black, or either the background or foreground color can be set to transparent. Those skilled in the art will understand that the difference in grayscale values ​​between the background and foreground colors can be designed to be between 1 and 255. This example is merely illustrative and should not be construed as a limitation of the invention.

[0095] In one example, adjacent marker modules can be distinguished by any one or any combination of different cell styles, different background colors, and different foreground colors. For example, two adjacent modules can be distinguished by different cell styles, different background colors, different foreground colors, different combinations of background and foreground colors, different graphic types and different background colors, or different combinations of graphic types, foreground colors, and background colors. This example is merely illustrative and should not be construed as a limitation of the invention by those skilled in the art.

[0096] In one example, the type of graphic set on the marking module includes at least one of the following: checkerboard pattern, circular array pattern, grid pattern, ChArUco pattern, and deep learning graphic pattern. The deep learning graphic pattern is the pattern that the camera can recognize based on deep learning (e.g., supervised learning, unsupervised learning) training. Preferably, the present invention uses a circular array pattern because it has high accuracy and robustness, achieving high-precision recognition at distances >50cm without requiring a large pattern. Figure 2A and Figure 2B As shown, the circular array patterns include symmetrical spot patterns (i.e., symmetrical dot matrix) and asymmetrical spot patterns (i.e., asymmetrical dot matrix).

[0097] For example, multiple marking modules include a first marking module and a second marking module, which are arranged at an angle to each other on the magnetic stimulation coil. The first marking module has a checkerboard pattern, and the second marking module has a circular array pattern. Of course, those skilled in the art can design the first and second marking modules as symmetrical dot matrices, where the background color of the first marking module is black and the foreground color (i.e., the color of the solid dots) is white. The background color of the second marking module is white, and the foreground color (i.e., the color of the solid dots) is black.

[0098] Of course, those skilled in the art can also design multiple marking modules as a first marking module, a second marking module, and a third marking module. The first marking module and the second marking module are adjacent to each other, and the second marking module and the third marking module are adjacent to each other, with the first marking module and the third marking module arranged opposite each other (i.e., one of the adjacent arrangement methods). The graphics of the first marking module, the second marking module, and the third marking module are all designed as asymmetrical dot matrices. Specifically, the first marking module and the third marking module are designed as asymmetrical dot matrices that are mirror images of each other, and the background color of the first marking module and the third marking module are both designed as black, and the foreground color (i.e., the color of the solid dots) is designed as white. The background color of the second marking module is designed as white, and the foreground color (i.e., the color of the solid dots) is designed as black.

[0099] Of course, those skilled in the art can also design multiple marking modules as a first marking module, a second marking module, a third marking module, and a fourth marking module, with the first to fourth marking modules arranged around the periphery of the magnetic stimulation coil. The pattern type on the first marking module is designed as a ChArUco pattern, the pattern type on the second marking module is designed as a symmetrical dot pattern, the pattern type on the third marking module is designed as a grid pattern, and the pattern type on the fourth marking module is designed as a checkerboard pattern. This example is merely an illustrative example, and those skilled in the art can design and adjust the number of marking modules, the type of pattern, the background color, and the foreground color, etc., as needed. This should not be construed as a limitation of the present invention.

[0100] In one example, identifying and determining at least one marker module that matches the preset parameters from among multiple marker modules located within the camera's field of view includes:

[0101] The identification elements on the marker module detected by the camera are compared with preset parameters to obtain marker modules that match the preset parameters;

[0102] Based on the determined marker module that matches the preset parameters, obtain the first spatial mapping relationship between the marker module and the camera and the second spatial mapping relationship between the marker module and the tracking target.

[0103] In one example, an RGB-D camera can capture the identification elements on the tagging module and determine the pose (i.e., spatial position and orientation) of the magnetic stimulation coil based on the position of the captured identification elements (which can also be considered the position of the tagging module). Those skilled in the art will understand that registering the tagging module with the RGB camera achieves parameter calibration between the tagging module and the RGB-D camera. In other words, once the identification elements on the tagging module are successfully registered with the RGB-D camera, the spatial mapping relationship (i.e., the first spatial mapping relationship) between the identification elements on the tagging module and the RGB camera can be obtained. Simultaneously, during use, the RGB camera can also acquire the pose of the identification elements on the tagging module in real time.

[0104] Therefore, once a marker module matching the preset parameters is determined, a first spatial mapping relationship and a second spatial mapping relationship corresponding to that marker module are also determined. In use, the first and second spatial mapping relationships corresponding to the determined marker module (or the identification element on the marker module) matching the preset parameters are searched from a database, for example. Alternatively, the spatial mapping relationship (i.e., the first spatial mapping relationship) between the identification element and the camera can be calculated using existing algorithms by detecting the identification element. This example is merely illustrative; those skilled in the art can use other methods to replace it, as long as the mapping relationship between the identification element and the camera can be constructed. Those skilled in the art should not interpret this example as a limitation of the present invention.

[0105] In one instance, based on the design of multiple marker modules, preset parameters for identifying and judging marker modules are preset in, for example, a database, so that the camera can determine whether the marker module it identifies matches the preset parameters, thereby enabling it to filter out marker modules that meet the preset criteria (i.e., preset parameters) from multiple marker modules.

[0106] In one example, the preset parameters include at least the identification elements of each of the multiple marking modules. Of course, those skilled in the art will understand that the preset parameters may also include other elements that can be used for camera recognition besides the identification elements on all marking modules, so that they can be replaced when the identification elements on the marking module cannot be recognized or when recognition results in recognition accuracy not meeting user requirements.

[0107] In one example, the identification element includes the color and / or graphic unit of the marking module; that is, the identification element can be color, graphic unit, or a combination of color and graphic unit. Of course, those skilled in the art can also set the identification element as other components, which are connected to the camera and tracking module during use. For example, the component can be a transmitter, a sticker with an identification code or pattern, etc. The size of the sticker can be approximately the size of the plane of the marking module facing the camera, or it can be other sizes, as long as it can be recognized and / or tracked by the camera.

[0108] In one example, the preset parameters may include color parameters corresponding to the color of the identified element, shape parameters corresponding to the graphic unit of the identified element, or both color and shape parameters, or other parameters besides those mentioned above, as long as they can be used to distinguish multiple identification modules.

[0109] In one example, color parameters are represented by at least one of the following: grayscale value, RGB value, CMY value, HSV value, and HIS value. The number of color parameters can be the same as, more than, or less than the total number of colors used in the multiple marking modules. For example, if the first, second, and third marking modules use a total of two colors (black and white), the number of color parameters can be preset to two: grayscale values ​​of 0 and 255, or two grayscale value ranges of 0–2 and 252–255.

[0110] In one example, when the recognition element is color, the color on the marker module recognized by the camera is compared with the color parameter to obtain a marker module that matches the color parameter.

[0111] For example, the identification elements on the first marking module are black, and the identification elements on the second marking module are white, and all colors on the first marking module are different from all colors on the second marking module. For example, the plane of the first marking module facing the camera is set to black, and the plane of the second marking module facing the camera is set to white. Of course, those skilled in the art can, as needed, design a portion of the plane of the first marking module facing the camera to be black, and the remaining portion to be a color different from all colors in the second marking module. In this case, the plane of the second marking module facing the camera can be designed to be entirely white, or it can be set to partially white, with the remaining portion set to a color different from all colors in the first marking module.

[0112] For example, when the camera detects an element, it compares the detected element with preset color parameters. If the detected element matches the preset grayscale value of black, then the detected marker module is identified as the first marker module. Similarly, when the camera detects an element, it compares the detected element with preset color parameters. If the detected element matches the preset grayscale value of white, then the detected marker module is identified as the second marker module. When the camera detects two elements, it compares one of the detected elements with preset color parameters. If the detected element matches the preset grayscale value of white, then the detected marker module is identified as the second marker module. Simultaneously, it compares the other detected element with preset color parameters. If the detected element matches the grayscale value of black, then the camera has detected the first marker module. In other words, the camera simultaneously detects both the first and second marker modules. This process continues, using different colors to determine the detected marker modules. Based on the determined detected marker modules, corresponding first and second spatial mapping relationships are obtained.

[0113] In one example, the marker module has a background color and a foreground color, and the background color and foreground color are different colors from each other. The color parameters include a background color parameter and a foreground color parameter.

[0114] For example, when the recognition element is color, the color on the marker module recognized by the camera is compared with color parameters to obtain a marker module that matches the color parameters, including:

[0115] The background color of the marker module detected by the camera is compared with the background color parameter, and the foreground color of the marker module detected by the camera is compared with the foreground color parameter, so as to obtain a marker module that matches both the background color parameter and the foreground color parameter.

[0116] For example, if the background color of the first marking module is set to black and the foreground color to white, and the background color of the second marking module is set to white and the foreground color to black, the camera can compare the identified element with the background color parameter and the foreground color parameter. If the foreground color of the element matches the preset foreground color parameter white, and the background color of the element matches the preset background color parameter black, then the marking module identified by the camera is determined to be the second marking module. This example is merely illustrative and should not be construed as a limitation of the invention by those skilled in the art.

[0117] In one instance, when the identified element is a graphic unit, the graphic unit on the marker module identified by the camera is compared with the shape parameters to obtain a marker module that matches the shape parameters.

[0118] In one example, the shape parameters include unit style. The graphic units on the marker module recognized by the camera are compared with a preset unit style to obtain marker modules that match the unit style. The unit style can be a 3D style or a 2D style.

[0119] For example, when the graphic unit on the marker module is designed as a 3D style, the shape parameter is the set of all 3D styles of the marker modules, and may also include other 3D styles besides those on all marker modules. The 3D style can be at least one of a cone, cylinder, hemisphere, etc. That is, the side of the marker module facing the camera can be designed as at least one of a cone, cylinder, hemisphere, etc., and correspondingly, the shape parameter includes cones, cylinders, hemispheres, etc., corresponding to the 3D style of the graphic unit.

[0120] In one example, a graphic is set on the plane facing the camera on all of the multiple marker modules. This graphic consists of at least one graphic element. A graphic unit is at least one graphic element in the graphic. That is, the identification element on the marker module can be one graphic element, multiple graphic elements, or all of the graphic elements in the graphic.

[0121] In one example, the graph type includes at least one of the following: checkerboard pattern, circular array pattern (including symmetrical and asymmetrical speckle patterns), grid pattern, ChArUco pattern, and deep learning graph pattern. Accordingly, the cell pattern includes the shapes of all graph elements in at least one preset graph element.

[0122] In one example, the planar style in the unit style corresponds to all the graphic units on the plurality of marking modules. For example, when the graphic type on the first marking module is set to a checkerboard style, the graphic elements in the checkerboard style are squares, that is, the graphic on the first marking module is composed of multiple squares, and the graphic units on the first marking module are set to black squares; the graphic type on the second marking module is set to an asymmetrical spot style, the graphic elements in the asymmetrical spot style are black solid circles; the graphic type on the third marking module is set to a ChArUco style, the graphic elements in the ChArUco style are any identification codes. Accordingly, the preset planar styles include black squares, black solid circles, and all identification codes (e.g., QR codes, barcodes) in the ChArUco style. Of course, the graphic elements can be a certain type of graphic, such as a checkerboard style, an asymmetrical spot style, etc., then the preset planar styles can include checkerboard styles and asymmetrical spot styles, and can also include grid styles, etc. This example is only an illustrative example and should not be construed as a limitation of the present invention by those skilled in the art.

[0123] When the camera recognizes an element, compares it with a preset planar pattern, and determines that the element matches a solid black circle, it is confirmed that the camera has recognized the second marker module.

[0124] When the camera recognizes an element, compares it with a preset planar pattern, and determines that the element matches a preset identification code, it is confirmed that the camera has recognized the third marker module.

[0125] When the camera detects two elements and compares them with a preset planar pattern, if one element matches a preset identification code, the camera has detected the third marker module. Simultaneously, if the other element matches a preset square, the camera has detected the first marker module. In other words, the camera simultaneously detects both the first and third marker modules.

[0126] Similarly, the camera can determine which or which marker modules it has identified by comparing the identified graphic unit with the preset unit style. Thus, it can determine which or which first spatial mapping relationship or second spatial mapping relationship needs to be acquired based on the identified marker modules.

[0127] In one example, the preset planar style should be identical to the unit style of all the recognition elements in the multiple marking modules. For example, when the multiple marking modules are equipped with three styles—checkerboard, symmetrical spot, and asymmetrical spot—then the preset unit style should at least include the checkerboard, symmetrical spot, and asymmetrical spot styles. It may also include grid and ChArUco styles, and of course, deep learning unit styles. When the graphic units of the multiple marking modules are all checkerboard styles, the preset unit style can include the same checkerboard style as the graphic units. It may also include grid styles and circular array styles (including symmetrical and asymmetrical spot styles). This example is merely illustrative and should not be construed as a limitation of the invention by those skilled in the art.

[0128] In one example, multiple marker modules are designed to include a first marker module and a second marker module, with the graphic units on both modules designed as asymmetric dot matrices (i.e., asymmetric speckle arrays). Misidentification can occur when the solid dots in an asymmetric dot matrice (i.e., asymmetric speckle pattern) are all the same size, or when one asymmetric dot matrice contains two asymmetric dot sub-dot matrices. Therefore, inverse colors (i.e., a grayscale difference of 255) can be used between the background and foreground colors of the first and second marker modules, allowing the camera to distinguish the two marker modules without causing misidentification.

[0129] In one example, the method for obtaining the first spatial mapping relationship (e.g., pose matrix, spatial transformation matrix) of the asymmetric point matrix in the camera coordinate system includes the following steps:

[0130] The RGB image of the asymmetric dot matrix on the first marking module, captured by the camera, is converted to grayscale. Then, white circles are detected in this grayscale image, and a circle center detection algorithm is used to detect the center of all solid circles in the asymmetric dot matrix. The eccentricity error is subtracted from the coordinates of each circle center in the image coordinate system; the pixel corresponding to this difference is the center of the corresponding solid circle in the asymmetric dot matrix. Since the distance to each circle center in the asymmetric dot matrix is ​​known, the PNP (Perspective-n-Point) algorithm can be used to calculate the pose of the asymmetric dot matrix on the first marking module in the camera coordinate system, i.e., to calculate the spatial transformation matrix or pose matrix M11 of the asymmetric dot matrix on the first marking module between the image coordinate system and the camera coordinate system.

[0131] Similarly, the RGB image of the asymmetric dot matrix on the second marking module captured by the camera is converted to grayscale. Then, black circles are detected in the grayscale image, and the center of all solid circles in the asymmetric dot matrix is ​​detected using a circle center detection algorithm. The eccentricity error is subtracted from the coordinates of each circle center in the image coordinate system, and the pixel corresponding to the difference is the center of the corresponding solid circle in the asymmetric dot matrix. Then, the PNP (Perspective-n-Point) algorithm is used to calculate the pose of the asymmetric dot matrix on the second marking module in the camera coordinate system, that is, to calculate the spatial transformation matrix or pose matrix M12 of the asymmetric dot matrix on the second marking module between the image coordinate system and the camera coordinate system.

[0132] The aforementioned spatial transformation matrix or attitude matrix M11 represents the first spatial mapping relationship between the first marking module and the camera, and the spatial transformation matrix or attitude matrix M12 represents the first spatial mapping relationship between the second marking module and the camera.

[0133] This example is merely illustrative. Those skilled in the art can establish the first spatial mapping relationship between the marker module and the camera using existing methods according to actual needs. For example, when multiple marker modules include a third marker module, and the third marker module also uses a circular array pattern for its graphic design, the first spatial transformation relationship between the third marker module and the image coordinate system and the camera coordinate system can be obtained using the methods described above for M11 and M12. Of course, other existing methods can also be used as alternatives. When the graphics on the marker module are checkerboard, grid, ChArUco, or deep learning graphic styles, existing methods can be used to obtain their first spatial mapping relationship between the image coordinate system and the camera coordinate system. This example is merely illustrative and should not be construed as a limitation of the present invention.

[0134] In one example, multiple marker modules are designed to include a first marker module, a second marker module, and a third marker module. The graphics on the first, second, and third marker modules are all designed as asymmetrical dot matrices. The first and third marker modules are arranged opposite each other (e.g., left-right, front-back, etc.). The second marker module is located between the first and third marker modules. The first to third marker modules are arranged along the circumference of the magnetic stimulation coil.

[0135] In one example, the background color of the first and third marker modules is black, and the foreground color is white, while the background color of the second marker module is white, and the foreground color is black. To avoid misidentification by the camera, the layout styles of the first and third marker modules are designed to be mirror images of each other.

[0136] In one example, the first spatial transformation relationships (e.g., pose matrix, spatial transformation matrix) between the image coordinate system and the camera coordinate system for the first marking module, the second marking module, and the third marking module are M21, M22, and M23, respectively. The basic principle and steps of obtaining these relationships are the same as those of obtaining the first spatial transformation relationships M11 and M12, and will not be repeated here.

[0137] In one example, when the graphic unit of the identified element is the entire graphic (i.e., all graphic elements in the graphic), the graphic design on the first and third marker modules is a layout style that is rotationally symmetrical to each other. That is, the graphic on the third marker module can be a graphic formed by rotating the graphic in the first marker module by an angle α (0°<α<360°) along the center of the graphic or the center of the plane of the marker module where the graphic is located. The arrangement of all graphic elements in the resulting graphic is different from the arrangement of all graphic elements in the first marker module, thus forming two different graphic layout styles of the same type of graphic.

[0138] In one example, the graphic on the third marker module can be formed by rotating the graphic on the first marker module 180° around the center of the graphic or the center of the plane of the marker module containing the graphic. That is, the unit styles on the first and third marker modules are centrally symmetrical (i.e., a form of rotational symmetry). The arrangement of all graphic elements in the resulting graphic is different from the arrangement of all graphic elements in the first marker module, thus forming two different graphic layout styles of the same type of graphic.

[0139] For example, a solid circle located in the first row of the graphic in the first marker module is located in the last row of the graphic in the third marker module, and so on from top to bottom, until a solid circle located in the last row of the graphic in the first marker module is located in the first row of the graphic in the third marker module. The asymmetrical spot pattern of the first marker module coincides with the asymmetrical spot pattern on the third marker module after rotating 180° clockwise (or counterclockwise) around the center of the asymmetrical spot pattern. That is, the asymmetrical spot pattern of the first marker module and the asymmetrical spot pattern of the graphic on the third marker module are centrally symmetrical (i.e., forming the same type of graphic but with different graphic layout styles).

[0140] In one example, the graphic in the first marking module can be rotated 90° clockwise or counterclockwise around the center of the graphic or the center of the plane of the marking module where the graphic is located to form the graphic on the third marking module. That is, the rotation angle between the graphics on the first marking module and the graphics on the third marking module is 90°, and they are symmetrical about each other by 90° rotation.

[0141] For example, a solid circle in the first row of the graphic in the first marking module is located in the first column to the left of the graphic in the third marking module (i.e., the solid circle in the first row is rotated counterclockwise to the first column to the left). Similarly, a solid circle in the last row of the graphic in the first marking module is located in the first column to the right of the graphic in the third marking module. That is, the asymmetrical spot pattern of the first marking module coincides with the asymmetrical spot pattern on the third marking module after rotating 90° counterclockwise (or clockwise) along the center of the asymmetrical spot pattern, so that the asymmetrical spot pattern of the first marking module and the asymmetrical spot pattern of the graphic on the third marking module are symmetrical after rotating 90°.

[0142] By changing the layout as described above, the real-time positions of the graphic units of the first and third marker modules can be distinguished, enabling the differentiation between the first marker module located to the left of the magnetic stimulation coil and the marker module located to the right of the magnetic stimulation coil. In subsequent tracking, the first spatial mapping relationship between the graphic matching the marker module and the camera, and the second spatial mapping relationship between the matching marker module and the tracking target can be determined and selected based on this difference.

[0143] In one instance, when at least two of a plurality of marker modules have the same type of graphics, and all graphic elements in the marker modules with the same type of graphics are arranged differently, the unit style includes graphic layout styles with all different arrangements.

[0144] In one example, when at least two of a plurality of marker modules have the same type of graphics, and all graphic elements in the marker modules with the same type of graphics are arranged differently, the graphic units on the marker modules identified by the camera are compared with a preset unit style to obtain marker modules that match the unit style, including:

[0145] The shape of at least one graphic element on the same marker module recognized by the camera is compared with the shape of all graphic elements in at least one preset graphic element. The layout style of all graphic elements on the same marker module recognized by the camera is compared with the preset layout style to obtain a marker module that matches the preset shape and the preset layout style.

[0146] In other words, all the marker modules in at least two marker modules can be distinguished by the different layout styles of the graphics of all the marker modules in at least two marker modules, so as to obtain the corresponding first spatial mapping relationship and second spatial mapping relationship based on the determined marker modules.

[0147] For example, when there are at least two marker modules with the same type of graphics but different layout styles, they can be distinguished by identifying the pixel coordinates of the same graphic element in different marker modules.

[0148] For example, when the graphics of the first and third marker modules are both asymmetrical spot patterns, the background color is black and the foreground color is white, and the layout of the graphics on the first and second marker modules is symmetrical with each other rotated 90° counterclockwise, when the layout of the graphics on the first marker module is set to the preset layout of the asymmetrical spot pattern, when distinguishing between the two marker modules, the layout of the graphics on the first and second marker modules in the captured image is compared with the preset layout, and it is determined that the layout of the graphics on the first and second marker modules is the asymmetrical spot pattern. Next, in the image coordinate system (i.e., the pixel coordinate system), with the top left corner of the image as the origin, the horizontal direction (from the origin to the right) as the positive direction of the horizontal axis Xi, and the vertical direction (from the origin downwards) as the positive direction of the vertical axis Yi, a straight line is drawn along the center of the nth (e.g., the 3rd) solid circle in the first row of the asymmetric spot pattern on the first marking module and the center of the first solid circle in that row. The angle between this line and the horizontal axis Xi of the image containing the first marking module is determined to be either acute or obtuse, thus distinguishing the two. Similarly, a straight line is drawn along the center of the nth solid circle in the first row and the center of the first solid circle in that row on the image containing the second marking module, and the angle between this line and the horizontal axis Xi of the image containing the second marking module is determined to be either acute or obtuse. For example, when the straight line on the first marking module forms an acute angle with the horizontal axis Xi, it is the graphic on the first marking module, and the marking module is identified as the first marking module; when it forms an obtuse angle, it is the graphic on the second marking module, and the marking module is identified as the second marking module.

[0149] For example, when the graphics between the first and third marker modules are all asymmetrical speckle patterns, the background color is all black, and the foreground color is all white, and the layout of the graphics on the first and second marker modules is symmetrical with a 90° counterclockwise rotation, when the preset layout style is set as the layout style of the graphics on the first marker module, when it is determined that the layout style and arrangement style of the solid circles on the marker module are the same as the preset layout style, a straight line is drawn connecting the center of the third solid circle in the first row of the marker module with the center of the first solid circle in the first row. The straight line forms an acute angle with the horizontal axis Xi of the image on which it is located. Then it is determined that the captured graphic is the graphic on the first marker module, and the marker module is the first marker module. When it is determined that the layout style of the solid circles on the marking module is the same as the preset layout style, but the arrangement is different, and it is determined that the graphic on the marking module is the arrangement formed by rotating the preset layout style clockwise by 90°, then the first column on the right of the asymmetrical spot pattern on the marking module is recorded as the first row, the first one in the first column is recorded as the first one in the first row, and the third one in the first column is recorded as the third one in the first row. Then, the center of the solid circle recorded as the first one in the first row and the center of the solid circle recorded as the third one in the first row are connected by a straight line. The straight line forms an obtuse angle with the horizontal axis Xi of the image. Then, it is determined that the captured graphic is the graphic on the second marking module, and the marking module is the second marking module.

[0150] Of course, those skilled in the art can also distinguish between the first and second marker modules by comparing the number of solid circles in two adjacent rows. For example, if the number of solid circles (i.e., elements) in the first row of a marker module is greater than the number of solid circles in the second row, then the layout style of that marker module is determined to be the layout style of the first marker module, and the marker module is determined to be the first marker module, while the other one is determined to be the second marker module; if the number of solid circles (i.e., elements) in the first row of a marker module is less than the number of solid circles in the second row, then the layout style of that marker module is determined to be the layout style of the second marker module, and the marker module is determined to be the second marker module, while the other marker module is determined to be the first marker module.

[0151] The two examples above are merely illustrative of the present invention and should not be construed as limiting the invention. Those skilled in the art can use existing technologies to make substitutions as needed, as long as it can distinguish between those with the same graphic type.

[0152] In one example, the camera can simultaneously determine the identification elements within its field of view that match each preset parameter based on color and graphic units, as well as their corresponding preset parameters. Those skilled in the art will understand that identification elements matching color parameters can be determined first based on color and color parameters, thereby determining the identification module corresponding to that identification element. When at least two identification modules matching color parameters are determined, identification elements matching shape parameters are then determined based on graphic units and shape parameters, thereby determining the identification module corresponding to that identification element. When at least two identification modules matching shape parameters are determined, the optimal identification module among the at least two identification modules matching shape parameters is selected based on a comprehensive evaluation value. For example, when multiple marking modules only have black and white colors, identification modules matching color parameters can be selected first based on color and color parameters, and then identification modules matching shape parameters can be selected based on graphic units and shape parameters.

[0153] Of course, those skilled in the art can first determine the identification elements matching the shape parameters based on graphic units and shape parameters, thereby determining the identification module corresponding to the identification element. When at least two identification modules matching the shape parameters are determined, then the identification elements matching the color parameters are determined based on color and color parameters, thereby determining the identification module corresponding to the identification element. When at least two identification modules matching the color parameters are determined, the optimal identification module among the at least two identification modules matching the shape parameters is selected based on a comprehensive evaluation value. For example, when multiple marking modules are colored, graphic units and shape parameters can be used first to select identification modules matching the shape parameters, and then color and color parameters can be used to select identification modules matching the color parameters.

[0154] In other words, the identification module can be determined first based on color, and then based on the graphic unit; alternatively, the identification module can be determined first based on the graphic unit, and then based on color. The order of determining color and graphic unit is not limited. Preferably, the identification is based on color first, and then on graphic unit. This example is merely illustrative and should not be construed as a limitation of the invention by those skilled in the art.

[0155] In one instance, when at least two of the multiple marker modules within the camera's field of view match the preset parameters (e.g., color parameters and / or shape parameters),

[0156] Each marking module is evaluated based on the pose of the recognized elements on all marking modules in at least two marking modules, the distance between the plane of the marking module containing the recognized element and the camera, and the size of the recognized element as recognized by the camera, in order to select the optimal marking module.

[0157] Based on the first spatial mapping relationship between the optimal marking module and the camera, and the second spatial mapping relationship between the optimal marking module and the tracking target, the pose of the tracking target under the camera is obtained.

[0158] In one example, the orientation of the graphic is the angle between the plane containing the graphic on the marking module and the optical axis of the camera, and the size of the identified element as determined by the camera is the ratio between the actual area of ​​the identified element in the image coordinate system and the area of ​​the image. This example is merely illustrative and should not be construed as a limitation of the invention.

[0159] In one example, the tagging modules are evaluated based on the pose of the recognized element on the same tagging module in the at least two tagging modules, the distance between the plane on the tagging module with the recognized element and the camera, and the size of the recognized element as recognized by the camera, in order to select the optimal tagging module, including:

[0160] Based on their respective weights, the angle, distance, and scale of the identified elements on all the at least two marking modules are used to obtain the angle evaluation value, distance evaluation value, and size evaluation value of the identified elements on each marking module.

[0161] The comprehensive evaluation value of the same identification element is obtained by adding the angle evaluation value, distance evaluation value and size evaluation value together;

[0162] Sort the comprehensive evaluation values ​​of the recognition elements of all the at least two labeling modules in descending order, and determine the labeling module corresponding to the largest comprehensive evaluation value as the optimal labeling module.

[0163] In one example, the formula for the comprehensive evaluation score is expressed as:

[0164] score = W1A + W2D + W3S;

[0165] Where A represents the angle evaluation value, W1 represents the weight of the angle evaluation value, D represents the distance evaluation value, W2 represents the weight of the distance evaluation value, S represents the size evaluation value, W2 represents the weight of the size evaluation value, and W1+W2+W3=1.

[0166] In one example, the range of W1 is set to 0.1 to 0.35, the range of W2 is set to 0.1 to 0.35, and the range of W3 is set to 0.1 to 0.35.

[0167] Those skilled in the art should understand that the selection of each weight should be set according to the needs of the application scenario. This example is only an illustrative example and should not be construed as a limitation of the present invention.

[0168] In one example, the actual area of ​​an element recognized by the camera is the area occupied by the portion of the element that is recognized in the image. The original area of ​​an element is the area occupied by the element in the image when it is fully captured by the camera.

[0169] In one example, when calculating the comprehensive evaluation value of the identified element, the angle evaluation value, distance evaluation value, and size evaluation value can be normalized first. The angle evaluation value can use 360° as the normalization standard, for example, by dividing the angle by 360°. The distance evaluation value can use the farthest distance captured by the camera as the normalization standard. The size evaluation value can use the area occupied by the identified element in the image when it is fully captured by the camera as the normalization standard. Those skilled in the art will understand that the purpose of normalization is to normalize the angle evaluation value, distance evaluation value, and size evaluation value to a similar order of magnitude, so that the optimal labeling module can be obtained based on the comprehensive evaluation value, thereby making the pose of the tracked target more accurate.

[0170] According to another embodiment of the present invention, an electronic device is provided. The electronic device (not shown) includes a memory (not shown) and at least one processor (not shown). The memory is communicatively connected to at least one processor; for example, the memory may be communicatively connected to one, two, or more processors. In one example, the communication connection can be made via physical devices, lines, or wireless signals, i.e., as long as communication between the memory and the processor is possible. In one example, the memory stores a program or instructions, which, when executed by the at least one processor, enable the electronic device to implement the aforementioned pose recognition method for tracking a target.

[0171] In one example, the processor can be a microprocessor, such as a general-purpose processor like a graphics processing unit (GPU), a central processing unit (CPU), or a digital signal processor (DSP). In another example, the processor can also be a microprocessor core implemented through hardware circuitry, such as a microprocessor core implemented in hardware logic components using reconfigurable logic, including field-programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), and systems-on-a-chip (SoCs).

[0172] In one example, the processor can also be a virtual processor, which can be a virtual processor with Intel x86 processor features or a virtual processor with PowerPC processor features. Preferably, the processor is a graphics processor. In one example, the processor can be a single-core processor or a multi-core processor.

[0173] In one example, the memory includes volatile memory (i.e., random access memory) and non-volatile memory. Volatile memory includes main memory, cache, etc., while non-volatile memory includes auxiliary memory, etc. In one example, the memory can be configured as remote memory, which can be connected to the processor via a network (wired or wireless network). The network includes, but is not limited to, wide area networks (WANs), local area networks (LANs), metropolitan area networks (MANs), personal area networks (PANs), the Internet, satellite communication networks, and any combination thereof.

[0174] In one example, the processor creates a corresponding task thread based on a program retrieved from memory and executes the thread. In another example, the processor retrieves a program from secondary storage based on a read instruction from memory to create a corresponding task thread and executes the thread. These threads are used to implement a pose recognition method for tracking a target.

[0175] According to another embodiment of the present invention, a navigation device is provided. The navigation device includes a camera 20 (e.g., ...). Figure 3 The navigation device includes an electronic device (as shown) and an electronic device connected to the camera signal (e.g., wired or wireless connection), wherein the electronic device is the aforementioned electronic device. The navigation device also includes a navigation interface, which navigates based on the real-time pose of the tracking target provided by the electronic device. During navigation, the navigation interface provides doctors with more accurate and convenient navigation guidance, improving treatment efficiency and effectiveness, and reducing the operational difficulty of navigation. It also reduces complex operating procedures.

[0176] According to another embodiment of the present invention, a readable storage medium is provided. In embodiments of the present invention, a "readable storage medium" refers to any medium that participates in providing a program or instructions to a processor for execution. The medium can take many forms, including but not limited to non-volatile media, volatile media, and transmission media. Non-volatile media include, for example, optical discs or magnetic disks, such as storage devices. Volatile media include dynamic memory, such as main memory. Transmission media include coaxial cables, copper wires, and optical fibers, including conductors containing buses. Transmission media can also take the form of acoustic or optical waves, such as acoustic or optical waves generated during radio frequency (RF) and infrared (IR) data communications. Common forms of readable storage media include, for example, floppy disks, flexible disks, hard disks, magnetic tapes, any other magnetic media, CD-ROMs, DVDs, any other optical media, punched cards, paper tape, any other physical media with a perforated pattern, RAM, PROMs and EPROMs, FLASH-EPROMs, any other memory chips or cartridges, carrier waves as described below, or any other medium from which a computer can read.

[0177] The readable storage medium stores a program or instructions that, when executed by a processor, perform the aforementioned pose recognition method for tracking targets.

[0178] In one example, the readable storage medium is deployed on a server in the form of a memory, such as a cloud server. The cloud server also has a processor that executes programs or instructions stored in the memory. The processor may be a central processing unit (CPU). In one example, the cloud server may be a virtual server mapped from a physical server using virtualization technology. There may be one or more physical servers; when there are multiple physical servers, the cloud server may be a virtual server mapped from a server cluster using virtualization technology. There may also be one or more virtual servers. In one example, the cloud server may be provided to users through a cloud platform.

[0179] like Figure 3As shown, according to another embodiment of the present invention, a transcranial magnetic stimulation (TMS) device 100 is provided. The TMS device includes a coil (not shown, such as a magnetic stimulation coil) and a main body 10 connected to the coil. The main body is equipped with an electronic device or a readable storage medium, wherein the electronic device is the aforementioned electronic device, and the readable storage medium is the aforementioned readable storage medium. The main body also includes at least one of a display module 11, a support frame 12, a robotic arm (not shown), and a treatment chair (or treatment bed, not shown). The TMS device provided by the present invention can be used for repetitive transcranial magnetic stimulation (rTMS). Repetitive transcranial magnetic stimulation (rTMS) is a painless and non-invasive neuromodulation technique that directly acts on the cerebral cortex through electromagnetic conversion, producing a series of physiological and biochemical changes. From Professor Baker's invention of the first TMS device in the UK in 1985, to Professor Liao Jiahua's team successfully developing China's first TMS device in 1988, and then to its re-emergence in the 21st century as one of the four new brain science technologies, TMS technology has undergone more than 30 years of in-depth research and experimentation, and its clinical value has been confirmed by experts from various disciplines. This technology was first used as an important adjunctive treatment for depression. With a deeper understanding of rTMS, its indications have been continuously expanded. It is now widely used in various clinical departments, including neurology, neurosurgery, psychiatry, rehabilitation medicine, and pain management, for a variety of diseases (different departments apply), including stroke, traumatic brain injury, spinal cord injury, neuropathic pain, tinnitus, Parkinson's disease, etc. (disease categories).

[0180] The target pose recognition method, electronic device, navigation device, readable storage medium, and transcranial magnetic stimulation device according to the present invention have at least one of the following advantages:

[0181] (1) The pose recognition method, electronic device, navigation device, readable storage medium and transcranial magnetic therapy device for tracking targets provided by the present invention adopt the method of automatic image recognition, so that the tracking target can move at any time and at will within the field of view of the camera, and the pose of the tracking target can be obtained without re-registration;

[0182] (2) The pose recognition method, electronic device, navigation device, readable storage medium and transcranial magnetic therapy device provided by the present invention increase the range of camera recognition of the tracked target through the design of multiple marking modules, so that the tracked target can be moved and rotated in a larger range during navigation, control and other processes;

[0183] (3) The pose recognition method, electronic device, navigation device, readable storage medium and transcranial magnetic therapy device provided by the present invention can automatically recognize the image in real time, so that the camera can automatically select and switch the marker module to be tracked and recognized during the movement and rotation of the target, so as to track the target in real time.

[0184] While some embodiments of the present general inventive concept have been shown and described, those skilled in the art will understand that changes may be made to these embodiments without departing from the principles and spirit of the present general inventive concept, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for recognizing a pose of a target, the method comprising the steps of: identifying and determining, based on a preset parameter, a marker module matching the preset parameter from a plurality of marker modules within a field of view of a camera; obtaining a pose of the target under the camera based on a first spatial mapping relationship between the marker module matching the preset parameter and the camera, and a second spatial mapping relationship between the marker module matching the preset parameter and the target provided with the plurality of marker modules. wherein when there are at least two marker modules matching the preset parameter from the plurality of marker modules within the field of view of the camera, evaluating the marker modules based on poses of recognition elements on all the marker modules, distances between planes having the recognition elements on the marker modules and the camera, and sizes of the recognition elements recognized by the camera, to obtain an optimal marker module by screening, obtaining the pose of the target under the camera based on the first spatial mapping relationship between the optimal marker module and the camera, and the second spatial mapping relationship between the optimal marker module and the target. 2.The method of claim 1, wherein the identifying and determining, based on a preset parameter, at least one marker module matching the preset parameter from a plurality of marker modules within a field of view of a camera comprises: comparing recognition elements on the marker modules recognized by the camera with a preset parameter to obtain the marker module matching the preset parameter; and obtaining the first spatial mapping relationship between the marker module matching the preset parameter and the camera, and the second spatial mapping relationship between the marker module matching the preset parameter and the target based on the marker module matching the preset parameter. 3.The method of claim 2, wherein the preset parameter comprises at least recognition elements of all the marker modules. 4.The method of claim 3, wherein the recognition elements comprise colors and / or graphic units of the marker modules, and the preset parameter comprises color parameters and / or shape parameters corresponding to the recognition elements of the marker modules. 5.The method of claim 4, wherein the comparing the recognition elements on the marker modules recognized by the camera with the preset parameter to obtain the marker module matching the preset parameter comprises: when the recognition elements are colors, comparing the colors on the marker modules recognized by the camera with color parameters to obtain the marker module matching the color parameters; and / or when the recognition elements are graphic units, comparing the graphic units on the marker modules recognized by the camera with shape parameters to obtain the marker module matching the shape parameters. 6.The method of claim 5, wherein the shape parameters comprise unit styles, and the unit styles comprise a three-dimensional style and a planar style, the comparing the graphic units on the marker modules recognized by the camera with the shape parameters to obtain the marker module matching the shape parameters comprises: The pattern unit recognized by the camera on the marker module is compared with a preset unit pattern to obtain a marker module matching the unit pattern.

7. The pose recognition method of claim 6, wherein, The plurality of marker modules are each provided with a pattern composed of at least one pattern element, The pattern unit is at least one pattern element in the pattern, and the plane pattern corresponds to all pattern units on the plurality of marker modules.

8. The pose recognition method of claim 7, wherein, The type of the pattern includes at least one of a checkerboard pattern, a circular array pattern, a grid pattern, a ChArUco pattern, and a deep learning pattern, The circular array pattern includes a symmetric spot pattern and an asymmetric spot pattern, The pattern units between adjacent marker modules are arranged in any one or any combination of different pattern types, different background colors, and different foreground colors.

9. The pose recognition method of claim 8, wherein, When at least two of the plurality of marker modules have the same type of pattern, and all pattern elements in the marker modules with the same type of pattern have different arrangement manners, the unit pattern includes pattern layout patterns of all different arrangement manners.

10. The pose recognition method of claim 9, wherein, When at least two of the plurality of marker modules have the same type of pattern, and all pattern elements in the marker modules with the same type of pattern have different arrangement manners, comparing the pattern unit recognized by the camera on the marker module with a preset unit pattern to obtain a marker module matching the unit pattern includes: Comparing the shape of at least one pattern element on the same marker module recognized by the camera with the shape of all pattern elements in the preset at least one pattern element, and comparing the layout pattern of all pattern elements on the same marker module recognized by the camera with a preset layout pattern to obtain a marker module matching the preset shape and matching the preset layout pattern.

11. The pose recognition method of claim 5, wherein, The color parameter includes a background color parameter and a foreground color parameter, The background color and the foreground color of the marker module are different colors, When the recognition element is color, comparing the color on the marker module recognized by the camera with the color parameter to obtain a marker module matching the color parameter includes: Comparing the background color on the marker module recognized by the camera with the background color parameter, and comparing the foreground color on the marker module recognized by the camera with the foreground color parameter to obtain a marker module matching the background color parameter and matching the foreground color parameter.

12. The pose recognition method of claim 2, wherein, The pose of the recognition element is an angle of a plane on the marker module having the recognition element and an included angle between the camera optical axis. The recognition element size is a ratio between an actual area of the recognition element recognized by the camera and an image area.

13. The pose recognition method of claim 12, wherein, The marker module is evaluated based on the pose of the recognition element on all of the at least two marker modules, the distance between the plane with the recognition element on the marker module and the camera, and the size of the recognition element recognized by the camera, to screen an optimal marker module, comprising: The angle, the distance, and the size of the recognition element on all of the at least two marker modules are obtained based on respective weights, to obtain an angle evaluation value, a distance evaluation value, and a size evaluation value of the recognition element on each marker module; The angle evaluation value, the distance evaluation value, and the size evaluation value of the same recognition element are added to obtain a comprehensive evaluation value of the recognition element; The comprehensive evaluation values of the recognition elements of all of the at least two marker modules are sorted in descending order, and a marker module corresponding to the maximum comprehensive evaluation value is determined as the optimal marker module.

14. The pose recognition method of claim 1, wherein, The pose of the tracking target under the camera is obtained based on the first spatial mapping relationship, the second spatial mapping relationship, and a real-time position of the marker module recognized by the camera.

15. The pose recognition method of claim 14, wherein, When the position of the tracking target changes, a marker module recognized by the camera in real time and matching a preset parameter is determined based on the preset parameter; A real-time pose of the tracking target under the camera is obtained based on the first spatial mapping relationship between all of the marker modules recognized by the camera in real time and the camera, the second spatial mapping relationship between the marker module recognized by the camera in real time and the tracking target, and a real-time position of the marker module.

16. An electronic device, characterized in that, The electronic device comprises a memory and at least one processor, the memory is in communication connection with the at least one processor, the memory stores programs or instructions, and the programs or instructions are executed by the at least one processor to implement the pose recognition method of the tracking target according to any one of claims 1-15.

17. A navigation device, characterized in that, The navigation device comprises a camera and an electronic device connected with the camera, and the electronic device is the electronic device of claim 16.

18. A readable storage medium, characterized in that, The readable storage medium stores programs or instructions, and the programs or instructions are executed by a processor to implement the pose recognition method of the tracking target according to any one of claims 1-15.

19. A transcranial magnetic therapy instrument, characterized in that, The transcranial magnetic therapy instrument comprises a coil and a main body connected with the coil, the main body is provided with an electronic device or a readable storage medium, the electronic device is the electronic device of claim 16, and the readable storage medium is the readable storage medium of claim 18.

Images