Pose recognition method and device, readable storage medium and transcranial magnetic therapeutic apparatus

Abstract

The invention discloses a pose recognition method of a tracking target, electronic equipment, navigation equipment, a readable storage medium and a transcranial magnetic therapeutic apparatus, and belongs to the technical field of medical treatment. The pose identification method comprises the following steps: identifying and determining a marking module matched with a preset parameter in a plurality of marking modules in a visual field range of a camera based on the preset parameter; and obtaining the pose of the tracking target under the camera based on a first space mapping relation between the marking module matched with the preset parameter and the camera and a second space mapping relation between the marking module matched with the preset parameter and the tracking target provided with the plurality of marking modules. According to the pose recognition method of the tracking target, the electronic equipment, the navigation equipment, the readable storage medium and the transcranial magnetic therapeutic apparatus, the tracking target can move randomly at any time in the visual range of the camera through an automatic graph recognition method, and the pose of the tracking target can be obtained without re-registration.

Description

Technical Field

[0001] The present invention relates to the field of medical technology, and in particular to a posture recognition method for tracking a target, an electronic device, a navigation device, a readable storage medium and a transcranial magnetic therapy device. Background Art

[0002] With the rapid development of society, the pace of life and work has accelerated, the life pressure of modern people has increased, and the incidence of mental illness and nervous system diseases has also increased, mainly including: depression, anxiety, binocular affective disorder, schizophrenia, somatoform disorder, sleep disorder, sequelae of cerebral infarction, post-stroke depression, as well as Alzheimer's disease, vascular cognitive impairment, Parkinson's disease, Meige syndrome, essential tremor, hereditary spinocerebellar ataxia, neuropathic pain and migraine, epilepsy, etc.

[0003] Mental illnesses and neurological diseases are usually treated and regulated by stimulating brain nerves. For example, through transcranial magnetic stimulation (TMS), a powered stimulation coil can be placed above the subject's head to stimulate the cerebral cortex (for example, the motor area) with a magnetic field.

[0004] Amassian et al. proved through experiments in 1987 that the placement direction of the stimulation coil has an impact on the stimulation effect of TMS on the cerebral cortex. In 1993, Hoflide et al. applied TMS to the treatment of mental and neurological diseases such as depression, and the experiment proved that TMS has a certain therapeutic effect on mental and neurological diseases such as depression; in 2005, Edwards et al. proved through experiments that low-intensity repetitive transcranial magnetic stimulation can cause the excitability of cerebral cortical neurons; in 2007, Joo EY et al. found that long-term, low-frequency repetitive transcranial magnetic stimulation has an effect on alleviating the symptoms of epilepsy; in 2015, Ku Y et al. used single-pulse transcranial magnetic stimulation to stimulate the sensory cortex and posterior parietal cortex of the brain, and found that this form of TMS has an intervention effect on the cognitive function of the brain.

[0005] However, there are many difficulties in the clinical application of transcranial magnetic stimulation therapy, which greatly restricts the application and promotion of transcranial magnetic stimulation technology in the treatment of mental and neurological diseases. The main difficulties are: first, the positioning of the TMS stimulation coil depends on the doctor's experience and technology, which is highly subjective. Inaccurate placement of the stimulation coil will affect the treatment effect; second, in the process of placing the stimulation coil, the structural characteristics of the subject's brain are not visible, and each person's head structure has individual differences, which makes the positioning cap insufficiently versatile and too accurate; third, each transcranial magnetic stimulation treatment usually takes 15 to 30 minutes. During this process, if the subject's head moves slightly, the placement of the stimulation coil will change. If the subject's head is fixed, as the stimulation time increases, the muscles contract and tense, and the subject will feel severe discomfort.

[0006] Therefore, it is indeed necessary to provide a posture recognition method, electronic device, navigation device, readable storage medium and transcranial magnetic therapy device that can at least meet the tracking target requirements of the subject to move over a large range during transcranial magnetic stimulation without re-registration. Summary of the invention

[0007] In order to solve at least one aspect of the above problems and defects in the prior art, the present invention provides a posture recognition method for tracking a target, an electronic device, a navigation device, a readable storage medium and a transcranial magnetic therapy device. The technical solution is as follows:

[0008] An object of the present invention is to provide a method for tracking a target's posture recognition.

[0009] Another object of the present invention is to provide an electronic device.

[0010] Another object of the present invention is to provide a navigation device.

[0011] Another object of the present invention is to provide a readable storage medium.

[0012] Another object of the present invention is to provide a transcranial magnetic therapy apparatus.

[0013] According to one aspect of the present invention, a method for tracking a target posture recognition is provided, the method comprising the following steps:

[0014] Based on preset parameters, identify and determine a marking module that matches the preset parameters among a plurality of marking modules located within the field of view of the camera;

[0015] Based on a first spatial mapping relationship between a marking module matching preset parameters and the camera, and a second spatial mapping relationship between a marking module matching preset parameters and a tracking target provided with the multiple marking modules, a position and posture of the tracking target under the camera is obtained.

[0016] Specifically, identifying and determining at least one marking module matching the preset parameters among a plurality of marking modules located within the field of view of the camera based on the preset parameters includes:

[0017] Comparing the identification elements on the marking module identified by the camera with the preset parameters to obtain the marking module matching the preset parameters;

[0018] Based on the determined marking module that matches the preset parameters, the first spatial mapping relationship between the marking module and the camera and the second spatial mapping relationship between the marking module and the tracking target are acquired.

[0019] Preferably, the preset parameters at least include identification elements of all marking modules among the multiple marking modules.

[0020] Further, the identification element includes a color and / or graphic unit of a marking module, and the preset parameter includes a color parameter and / or a shape parameter corresponding to the identification element of the marking module.

[0021] Specifically, comparing the identification element on the marking module recognized by the camera with the preset parameters to obtain the marking module matching the preset parameters includes:

[0022] When the identification element is color, the color on the marking module identified by the camera is compared with the color parameter to obtain a marking module matching the color parameter; and / or

[0023] When the identification element is a graphic unit, the graphic unit on the marking module identified by the camera is compared with the shape parameter to obtain a marking module matching the shape parameter.

[0024] Furthermore, the shape parameter includes a unit style, and the unit style includes a three-dimensional style and a plane style.

[0025] Comparing the graphic unit on the marking module recognized by the camera with the shape parameter to obtain the marking module matching the shape parameter, including:

[0026] The graphic unit on the marking module recognized by the camera is compared with a preset unit pattern to obtain a marking module matching the unit pattern.

[0027] Specifically, each of the plurality of marking modules is provided with a graphic, and the graphic is composed of at least one graphic element.

[0028] The graphic unit is at least one graphic element in the graphic, and the plane pattern corresponds to all the graphic units on the multiple marking modules.

[0029] Specifically, the type of the graphics includes at least one of a checkerboard style, a circular array style, a grid style, a ChArUco style, and a deep learning graphics style.

[0030] The circular array pattern includes a symmetrical spot pattern and an asymmetrical spot pattern.

[0031] The graphic units between adjacent marking modules are arranged as any one of different graphic types, different background colors and different foreground colors or any combination thereof.

[0032] Further, when at least two marking modules among the plurality of marking modules have the same type of graphics, and all graphic elements in the marking modules having the same type of graphics are arranged in different ways, the unit pattern includes graphic layout patterns of all different arrangements.

[0033] Specifically, when at least two marking modules among the plurality of marking modules have the same type of graphics, and all graphic elements in the marking modules having the same type of graphics are arranged in different ways, comparing the graphic unit on the marking module recognized by the camera with a preset unit pattern to obtain a marking module matching the unit pattern, including:

[0034] The shape of at least one graphic element on the same marking module recognized by the camera is compared with the shapes of all graphic elements in at least one preset graphic element, and the layout style of all graphic elements on the same marking module recognized by the camera is compared with the preset layout style to obtain a marking module that matches the preset shape and the preset layout style.

[0035] Furthermore, the color parameters include background color parameters and foreground color parameters.

[0036] The background color and foreground color of the marking module are different colors.

[0037] When the identification element is color, comparing the color on the marking module identified by the camera with the color parameter to obtain a marking module matching the color parameter includes:

[0038] The background color on the marking module recognized by the camera is compared with the background color parameter, and the foreground color on the marking module recognized by the camera is compared with the foreground color parameter to obtain a marking module matching the background color parameter and the foreground color parameter.

[0039] Preferably, when at least two marking modules among the plurality of marking modules within the field of view of the camera match the preset parameters,

[0040] The marking modules are evaluated based on the posture of the recognition element on the same marking module of the at least two marking modules, the distance between the plane with the recognition element on the marking module and the camera, and the size of the recognition element recognized by the camera to screen out the optimal marking module,

[0041] Based on a first spatial mapping relationship between an optimal marking module and a camera and a second spatial mapping relationship between an optimal marking module and a tracking target, a position and posture of the tracking target under the camera is obtained.

[0042] More preferably, the posture of the identification element is the angle between the plane with the identification element on the marking module and the optical axis of the camera,

[0043] The size of the identification element recognized by the camera is the ratio between the actual area of ​​the identification element recognized by the camera and the image area.

[0044] Specifically, the marking module is evaluated based on the posture of the recognition element on the same marking module of the at least two marking modules, the distance between the plane with the recognition element on the marking module and the camera, and the size of the recognition element recognized by the camera to screen out the optimal marking module, including:

[0045] The angle, the distance, and the ratio of the identification elements on all the marking modules in the at least two marking modules are based on respective weights to obtain an angle evaluation value, a distance evaluation value, and a size evaluation value of the identification element on each marking module;

[0046] Adding the angle evaluation value, the distance evaluation value and the size evaluation value of the same identification element to obtain a comprehensive evaluation value of the identification element;

[0047] The comprehensive evaluation values ​​of the identification elements of all the marking modules in at least two marking modules are sorted in descending order, and the marking module corresponding to the maximum comprehensive evaluation value is determined as the optimal marking module.

[0048] Furthermore, based on the first spatial mapping relationship, the second spatial mapping relationship and the real-time position of the marking module recognized by the camera, the position and posture of the tracking target under the camera is obtained.

[0049] Specifically, when the position of the tracking target changes, a marking module recognized by the camera in real time and matching the preset parameters is determined based on the preset preset parameters;

[0050] Based on the first spatial mapping relationship between all the marking modules in the marking module recognized by the camera in real time and the camera, the second spatial mapping relationship between the marking module recognized by the camera in real time and the tracking target, and the real-time position of the marking module, the real-time posture of the tracking target under the camera is obtained.

[0051] According to another aspect of the present invention, an electronic device is provided, wherein:

[0052] The electronic device includes a memory and at least one processor, the memory is communicatively connected to the at least one processor, a program or instruction is stored in the memory, and when the program or instruction is executed by the at least one processor, the electronic device is used to implement any of the above-mentioned methods for tracking target posture recognition.

[0053] According to another aspect of the present invention, a navigation device is provided, wherein:

[0054] The navigation device includes a camera and an electronic device connected to the camera signal, and the electronic device is the electronic device mentioned above.

[0055] According to another aspect of the present invention, a readable storage medium is provided, wherein:

[0056] The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, it executes any of the above-mentioned methods for recognizing the posture of a tracking target.

[0057] According to another aspect of the present invention, a transcranial magnetic therapy apparatus is provided, wherein:

[0058] The transcranial magnetic therapy device includes a coil and a main body connected to the coil, the main body is provided with the electronic device or the readable storage medium, the electronic device is the above-mentioned electronic device, and the readable storage medium is the readable storage medium according to the above-mentioned claim.

[0059] The method for recognizing the position and posture of a tracking target, the electronic device, the navigation device, the readable storage medium and the transcranial magnetic therapy apparatus according to the present invention have at least one of the following advantages:

[0060] (1) The position and posture recognition method, electronic device, navigation device, readable storage medium and transcranial magnetic therapy device provided by the present invention adopt a method of automatic pattern recognition by a structured light camera, so that the tracking target can be moved at any time and at will within the visual range of the camera, and the position and posture of the tracking target can be obtained without re-registration;

[0061] (2) The target position recognition method, electronic device, navigation device, readable storage medium and transcranial magnetic therapy device provided by the present invention increase the range of the camera to recognize the target through the design of multiple marking modules, so that the target can be moved and rotated over a larger range during navigation, control and other processes;

[0062] (3) The target posture recognition method, electronic device, navigation device, readable storage medium and transcranial magnetic therapy device provided by the present invention can automatically recognize graphics in real time, so that when the target is moving or rotating, the camera can automatically select and switch the marking module to be tracked and identified to track the target in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] These and / or other aspects and advantages of the present invention will become apparent and readily understood from the following description of the preferred embodiments in conjunction with the accompanying drawings, in which:

[0064] Figure 1 is a flow chart of a method for tracking a target according to an embodiment of the present invention;

[0065] Figure 2A yes Figure 1 The pattern on the marking module shown is a view of a symmetrical spot pattern;

[0066] Figure 2B yes Figure 1 The pattern on the marking module shown is a view of an asymmetric spot pattern;

[0067] Figure 3 is a structural schematic diagram of a transcranial magnetic therapy apparatus according to another embodiment of the present invention. DETAILED DESCRIPTION

[0068] The technical solution of the present invention is further specifically described below by examples and in conjunction with the accompanying drawings. In the specification, the same or similar reference numerals indicate the same or similar components. The following description of the embodiments of the present invention with reference to the accompanying drawings is intended to explain the overall inventive concept of the present invention and should not be construed as a limitation of the present invention.

[0069] In today's society, people pay much less attention to mental illness than to organic diseases.

[0070] With the rapid development of medical imaging technology and medical image processing technology, image-guided surgery systems have emerged. With the help of three-dimensional reconstruction models of medical images, doctors can intuitively and accurately analyze the structure of organs or tissues and their surrounding tissues, thereby assisting doctors in treating mental illnesses and improving the efficiency of doctors in diagnosing and treating mental illnesses.

[0071] Image-guided surgery systems can use intraoperative images of patients and three-dimensional models of relevant lesions and surrounding tissues to guide the implementation of clinical procedures in real time. During the surgical operation, the image-guided surgery system can accurately display the patient's tissue anatomy and the details of the three-dimensional space around the lesion.

[0072] The image-guided surgery system can also use the head images scanned by medical imaging technology to perform segmentation and three-dimensional reconstruction to establish a three-dimensional head model containing brain tissue, plan stimulation targets on the reconstructed three-dimensional brain model, and then use image registration technology to map the targets on the three-dimensional brain model to the patient's head during surgery, thereby assisting and guiding doctors in locating the targets.

[0073] The image-guided surgery system uses an optically assisted navigation system for navigation, which is currently the most representative TMS navigation system. In 2008, Lars Matthaüs et al. used the Polaris Spectra optical tracking device and the Adept Viper s850 six-axis robot to form a TMS robot treatment system. The TMS robot treatment system uses a robot to clamp the stimulation coil, and the optical tracking device is fixed by a bracket. A marker is fixed on the subject's head to facilitate the optical tracking device to locate the head coordinates. There is currently no public research on the robot system for transcranial magnetic stimulation treatment in China. Research on medical robot technology started relatively late, but has developed rapidly.

[0074] In recent years, with the development of artificial intelligence and the maturity of face matching technology, point cloud matching algorithms have gradually been introduced in navigation. However, due to problems such as low accuracy, there is currently no very mature high-precision commercial medical navigation system (for example, TMS navigation system, transcranial magnetic therapy device, etc.).

[0075] Although the use of an optical navigation and positioning system can realize the visualization of the transcranial magnetic stimulation process and improve the accuracy of the stimulation coil positioning to a certain extent, the current navigation is relatively cumbersome in operation. For example, the subject registration process is required before the treatment begins. Once the camera is not touched, it needs to be re-registered, which is not applicable to subjects with autism.

[0076] In view of this, the present invention provides a new method for tracking the position and posture of a target.

[0077] See also Figure 1 , shows a method for recognizing the position and posture of a tracking target according to an embodiment of the present invention. The method for recognizing the position and posture comprises the following steps:

[0078] Based on the preset parameters, identify and determine a marking module that matches the preset parameters among a plurality of marking modules located within the field of view of the camera;

[0079] Based on a first spatial mapping relationship between a marking module and a camera and a second spatial mapping relationship between at least one marking module and a tracking target provided with a plurality of marking modules, a position and posture of the tracking target under the camera is obtained.

[0080] Those skilled in the art can understand that the tracking target posture recognition method provided by the present invention can obtain the posture of the tracking target in other coordinate systems, for example, the posture of the tracking target in the marker coordinate system can be obtained, and the posture of the tracking target in the visualization model can also be obtained, that is, those skilled in the art can obtain the posture of the tracking target in the required coordinate system or required space through the method or principle of the present invention as needed. This example is only an illustrative example, and those skilled in the art should not be understood as a limitation of the present invention.

[0081] Those skilled in the art can also understand that the tracking target can be a designated tracking object, such as a target point of a subject, facial feature information of a subject, a magnetic stimulation coil, a treatment chair, a treatment bed, or a part of a subject, and of course, can also be any object / human body or all objects / human bodies within the field of view of the camera. This example is only an illustrative example, and those skilled in the art should not understand it as a limitation of the present invention.

[0082] In one example, the field of view of a camera is the viewing angle of the camera.

[0083] The camera may be a black and white-depth camera or an RGB-D (color-depth) camera. Specifically, it may be one or any combination of RGB structured light camera, triangulation ranging camera, RGB-TOF camera, infrared depth camera, stereo vision camera, 3D camera module.

[0084] In one example, the tracking target may be a stimulation module that transmits magnetic stimulation, electrical stimulation, or light stimulation, such as a stimulation coil or a positioning cap. Of course, those skilled in the art will appreciate that the tracking target may also be designed to be the subject's head, arm, heart, etc., as long as it is any part of the human body to be tracked.

[0085] In one example, a plurality of marking modules are arranged on the tracking target, and the marking modules can be arranged as two, three, five or more. The plurality of marking modules are arranged at intervals from each other. Of course, those skilled in the art can understand that when the plurality of marking modules are arranged at intervals from each other, they may not form overlapping areas with each other, and of course, at least two of the plurality of marking modules may form partial overlap, as long as it does not affect the camera recognition.

[0086] In one example, the cross-sectional shape of the marking module can be designed to be L-shaped, wedge-shaped, triangular, rectangular, or the like.

[0087] In an example, the number of marking modules that match the preset parameters may be 1, 2, or more, and the maximum number thereof should be less than or equal to the number of marking modules set on the tracking target.

[0088] In one example, the pose of the tracking target under the camera is obtained based on the first spatial mapping relationship, the second spatial mapping relationship and the real-time position of the marking module recognized by the camera. Those skilled in the art will appreciate that the pose recognition method for the tracking target provided by the present invention can obtain the real-time pose of the tracking target in other coordinate systems. For example, the real-time pose of the tracking target in the marker coordinate system can be obtained, and the real-time pose of the tracking target in the visualization model can also be obtained. That is to say, those skilled in the art can obtain the real-time pose of the tracking target in the required coordinate system or required space through the method or principle of the present invention as needed. This example is only an illustrative example, and those skilled in the art should not understand it as a limitation of the present invention.

[0089] In one example, when the position of the tracking target changes, a marking module recognized by the camera in real time and matching the preset parameters is determined based on the preset preset parameters;

[0090] Based on the first spatial mapping relationship between all the marking modules in the marking module recognized by the camera in real time and the camera, the second spatial mapping relationship between the marking module recognized by the camera in real time and the tracking target, and the real-time position of the marking module, the real-time posture of the tracking target under the camera is obtained.

[0091] The following takes the tracking target as a magnetic stimulation coil, and takes the final position and posture of the tracking target in the camera coordinate system as an example to exemplify the basic principles and steps of the present invention. Those skilled in the art should understand that when the tracking target is an object other than the magnetic stimulation coil, the basic principles and steps of its implementation are roughly the same as those of the position and posture recognition method of the present invention, and will not be repeated here. Those skilled in the art should also understand that when obtaining a position and posture other than the camera coordinate system, as long as the spatial conversion relationship (i.e., spatial mapping relationship) between the pre-obtained space and the space of the tracking target can be obtained, the basic principles of its implementation are roughly the same as those of the position and posture recognition method of the present invention, and will not be repeated here.

[0092] During use, the magnetic stimulation coil needs to be registered first to establish a first spatial mapping relationship between each of the multiple marking modules and the camera. And since each marking module is rigidly connected to the magnetic stimulation coil, the spatial mapping relationship between each marking module and the magnetic stimulation coil is a fixed relationship. In one example, the first spatial mapping relationship between each marking module and the camera is set in the form of a matrix, and of course, the second spatial mapping relationship between each marking module and the magnetic stimulation coil can also be set in the form of a matrix.

[0093] In one example, the first spatial mapping relationship and the second spatial mapping relationship may be spatial transformation matrices (ie, spatial conversion matrices). Spatial transformation (ie, spatial conversion) includes translation, rotation, shearing, scaling, and the like.

[0094] In one example, each marking module is provided with an identification element for camera recognition. There is a background color and a foreground color on the marking module, and the background color and the foreground color are different colors. For example, the foreground color can be designed to be black, and the background color can be designed to be white. Of course, the foreground color can be designed to be white, the background color can be designed to be black, and the background color or the foreground color can also be set to be transparent. Those skilled in the art will understand that the difference in grayscale value between the background color and the foreground color can be designed to be between 1 and 255. This example is only an illustrative example, and those skilled in the art should not be understood as a limitation of the present invention.

[0095] In one example, adjacent marking modules can be distinguished by any one of different unit styles, different background colors, and different foreground colors, or any combination thereof. For example, two adjacent marking modules can be distinguished by different unit styles, or by different background colors, or by different foreground colors, or by a combination of different background colors and foreground colors, or by a combination of different graphic types and different background colors, or by a combination of different graphic types, foreground colors, and background colors. This example is only an illustrative example, and those skilled in the art should not be construed as a limitation of the present invention.

[0096] In one example, the type of graphics set on the marking module includes at least one of a checkerboard pattern, a circular array pattern, a grid pattern, a ChArUco pattern, and a deep learning graphic pattern. A deep learning graphic pattern is a graphic pattern that can be recognized by a camera based on deep learning (e.g., supervised learning, unsupervised learning) training. Preferably, the present invention uses a circular array pattern because the circular array pattern has high accuracy and good robustness, and high-precision recognition can be achieved at a distance of >50cm without a large pattern. Figure 2A and Figure 2B As shown, the circular array pattern includes a symmetrical spot pattern (ie, a symmetrical dot matrix) and an asymmetrical spot pattern (ie, an asymmetrical dot matrix).

[0097] For example, the plurality of marking modules include a first marking module and a second marking module, and the first marking module and the second marking module are arranged on the magnetic stimulation coil at an angle to each other. The type of graphics on the first marking module is a checkerboard pattern, and the type of graphics on the second marking module is a circular array pattern. Of course, those skilled in the art can design the first marking module and the second marking module as a symmetrical dot matrix, wherein the background color of the first marking module is designed to be black, and the foreground color (i.e., the color of the solid dots) is designed to be white. The background color of the second marking module is designed to be white, and the foreground color (i.e., the color of the solid dots) is designed to be black.

[0098] Of course, those skilled in the art can also design multiple marking modules as a first marking module, a second marking module, and a third marking module. The first marking module is adjacent to the second marking module, the second marking module is adjacent to the third marking module, and the first marking module and the third marking module are arranged relatively (i.e., a mode in an adjacent setting). The graphics of the first marking module, the second marking module, and the third marking module are all designed as an asymmetric lattice. Among them, the first marking module and the third marking module are designed to be an asymmetric lattice that is mirror-symmetrical to each other, and the background colors of the first marking module and the third marking module are both designed to be black, and the foreground color (i.e., the color of the solid dots) is designed to be white, and the background color of the second marking module is designed to be white, and the foreground color (i.e., the color of the solid dots) is designed to be black.

[0099] Of course, those skilled in the art can also design multiple marking modules as a first marking module, a second marking module, a third marking module and a fourth marking module, and the first marking module to the fourth marking module are arranged around the periphery of the magnetic stimulation coil. The type of graphics on the first marking module is designed as a ChArUco style, the type of graphics on the second marking module is designed as a symmetrical spot style, the type of graphics on the third marking module is designed as a grid style, and the type of graphics on the fourth marking module is designed as a checkerboard style. This example is only an illustrative example, and those skilled in the art can design and adjust the number of marking modules, the type of graphics, the background color and the foreground color, etc. as needed, and those skilled in the art should not be understood as a limitation of the present invention.

[0100] In one example, identifying and determining at least one marking module matching the preset parameters among a plurality of marking modules located within a field of view of a camera based on preset parameters includes:

[0101] Comparing the identification elements on the marking module identified by the camera with the preset parameters to obtain the marking module matching the preset parameters;

[0102] Based on the determined marking module that matches the preset parameters, the first spatial mapping relationship between the marking module and the camera and the second spatial mapping relationship between the marking module and the tracking target are acquired.

[0103] In one example, the RGB-D camera can capture the identification element on the marking module, and determine the position (i.e., spatial position and posture) of the magnetic stimulation coil according to the position of the captured identification element (which can also be considered as the position of the marking module). Those skilled in the art should understand that by registering the marking module with the RGB camera, parameter calibration between the marking module and the RGB-D camera is achieved. In other words, when the identification element on the marking module is successfully registered with the RGB-D camera, the spatial mapping relationship (i.e., the first spatial mapping relationship) between the identification element on the marking module and the RGB camera can be obtained, and the RGB camera can also obtain the position and posture of the identification element on the marking module in real time during use.

[0104] Therefore, when the marking module that matches the preset parameters is determined, the first spatial mapping relationship and the second spatial mapping relationship corresponding to the marking module are also determined. When in use, the first spatial mapping relationship and the second spatial mapping relationship corresponding to the marking module (or the identification element on the marking module) are searched from a database, for example, based on the determined marking module (or the identification element on the marking module) that matches the preset parameters. Of course, the spatial mapping relationship (i.e., the first spatial mapping relationship) between the identification element and the camera can also be calculated by detecting the identification element and using an existing algorithm. This example is only an illustrative example, and those skilled in the art can replace it with other methods, as long as the mapping relationship between the identification element and the camera can be constructed. Those skilled in the art should not understand this example as a limitation to the present invention.

[0105] In one instance, according to the design of multiple marking modules, preset parameters for identifying and judging the marking modules are preset in, for example, a database, so that the camera can judge whether the identified marking module matches the preset parameters based on the preset parameters, thereby filtering out the marking modules that meet the preset standards (i.e., preset parameters) from the multiple marking modules.

[0106] In one example, the preset parameters include at least the identification elements of each marking module in the plurality of marking modules. Of course, those skilled in the art will appreciate that the preset parameters may also include other elements that can be used for camera recognition in addition to the identification elements on all marking modules, so that when the identification elements on the marking module cannot be recognized or the recognition accuracy after recognition does not meet the user's requirements, they can be replaced.

[0107] In one example, the identification element includes the color and / or graphic unit of the marking module, that is, the identification element can be a color, or a graphic unit, or a combination of a color and a graphic unit. Of course, those skilled in the art can also set the identification element as other components, and connect the components to the camera and the tracking module when in use. For example, the component can be a transmitter, a sticker with an identification code or pattern, etc. The size of the sticker can be similar to the size of the plane of the marking module facing the camera, or it can be other sizes, as long as it can be recognized and / or tracked by the camera.

[0108] In one example, the preset parameters may include color parameters corresponding to the color of the identification element, or may include shape parameters corresponding to the graphic unit of the identification element. Of course, it may also include both color parameters and shape parameters, or may include other parameters in addition to the above parameters, as long as they can be used to distinguish multiple marking modules.

[0109] In one example, the color parameter is represented by at least one of the grayscale value, RGB value, CMY value, HSV value and HIS value of the color. The number of color parameters can be the same as the sum of the number of colors used in multiple marking modules, or more than the sum of the number of colors used in multiple marking modules, and of course less than the sum of the number of colors used. For example, there are a total of two colors, black and white, in the first marking module, the second marking module and the third marking module. At this time, the number of color parameters can be preset to two, namely grayscale values ​​0 and 255, or two grayscale value ranges 0 to 2 and 252 to 255.

[0110] In one example, when the identification element is color, the color on the marking module identified by the camera is compared with the color parameter to obtain a marking module matching the color parameter.

[0111] For example, the identification element on the first marking module is black, the identification element on the second marking module is white, and all colors on the first marking module are different from all colors on the second marking module. For example, the plane of the first marking module facing the camera is set to black, and the plane of the second marking module facing the camera is set to white. Of course, those skilled in the art can design the part of the plane of the first marking module facing the camera to be black, and the rest of the plane to be a color different from all colors in the second marking module as needed. In this case, the plane of the second marking module facing the camera can be designed to be all white, and of course it can also be set to be partially white, and the rest of the plane can be set to a color different from all colors in the first marking module.

[0112] For example, when the camera recognizes an element, the recognized element is compared with a preset color parameter, and it is determined that the recognized element matches the preset grayscale value of black, and the recognized marking module is determined to be the first marking module. Similarly, when the camera recognizes an element, the recognized element is compared with a preset color parameter, and it is determined that the recognized element matches the preset grayscale value of white, and the recognized marking module is determined to be the second marking module. When the camera recognizes two elements, one of the recognized elements is compared with a preset color parameter, and it is determined that the recognized element matches the preset grayscale value of white, and the recognized marking module is determined to be the second marking module. At the same time, the other recognized element is compared with the preset color parameter, and it is determined that the recognized element matches the grayscale value of black, and it is determined that the camera recognizes the first marking module, that is, the camera recognizes the first marking module and the second marking module at the same time. By analogy, the recognized marking module can be determined by different colors, thereby obtaining the corresponding first spatial mapping relationship and second spatial mapping relationship based on the determined recognized marking module.

[0113] In one example, the marking module has a background color and a foreground color, and the background color and the foreground color are different colors from each other. The color parameter includes a background color parameter and a foreground color parameter.

[0114] For example, when the identification element is color, comparing the color of the marking module identified by the camera with the color parameter to obtain a marking module matching the color parameter includes:

[0115] The background color on the marking module recognized by the camera is compared with the background color parameter, and the foreground color on the marking module recognized by the camera is compared with the foreground color parameter to obtain a marking module matching the background color parameter and the foreground color parameter.

[0116] For example, when the background color of the first marking module is set to black and the foreground color is set to white, and the background color of the second marking module is set to white and the foreground color is set to black, the camera can compare the recognized element with the background color parameter and the foreground color parameter. When the foreground color in the element matches the preset foreground color parameter white, and the background color therein matches the preset background color parameter black, it is determined that the marking module recognized by the camera is the second marking module. This example is only an illustrative example, and those skilled in the art should not be understood as a limitation of the present invention.

[0117] In one example, when the identification element is a graphic unit, the graphic unit on the marking module identified by the camera is compared with the shape parameter to obtain a marking module matching the shape parameter.

[0118] In one example, the shape parameter includes a unit pattern. The graphic unit on the marking module recognized by the camera is compared with a preset unit pattern to obtain a marking module matching the unit pattern. The unit pattern can be a three-dimensional pattern or a flat pattern.

[0119] For example, when the graphic unit on the marking module is designed as a three-dimensional style, the shape parameter is a set of three-dimensional styles of all marking modules, and of course, other three-dimensional styles other than the three-dimensional styles on all marking modules can also be included. The three-dimensional style can be at least one of a cone, a cylinder, a hemisphere, etc., that is, the side of the marking module facing the camera can be designed as at least one of a cone, a cylinder, a hemisphere, etc., and accordingly, the shape parameter includes a cone, a cylinder, a hemisphere, etc. corresponding to the three-dimensional style of the graphic unit.

[0120] In one example, a graphic is provided on a plane on a side facing the camera on all the marking modules among the multiple marking modules, and the graphic is composed of at least one graphic element. A graphic unit is at least one graphic element in the graphic. That is to say, the identification element on the marking module can be one graphic element in the graphic, or multiple graphic elements, or all graphic elements.

[0121] In one example, the graphic type includes at least one of a checkerboard pattern, a circular array pattern (including a symmetrical spot pattern and an asymmetrical spot pattern), a grid pattern, a ChArUco pattern, and a deep learning graphic pattern. Accordingly, the unit pattern includes the shapes of all graphic elements in at least one preset graphic element.

[0122] In one example, the plane style in the unit style corresponds to all the graphic units on the multiple marking modules. For example, when the graphic type on the first marking module is set to a checkerboard style, the graphic element in the checkerboard style is a square, that is, the graphic on the first marking module is composed of a plurality of squares, and the graphic unit on the first marking module is set to a black square; the graphic type on the second marking module is set to an asymmetric spot style, and the graphic element in the asymmetric spot style is a black solid circle; the graphic type on the third marking module is set to a ChArUco style, and the graphic element in the ChArUco style is any one of the identification codes. Accordingly, the preset plane style includes black squares, black solid circles, and all identification codes in the ChArUco style (for example, QR codes, bar codes). Of course, the graphic element can be a certain type of graphic, such as a checkerboard style, an asymmetric spot style, and the preset plane style can include a checkerboard style and an asymmetric spot style, and can also include a grid style, etc. This example is only an illustrative example, and those skilled in the art should not be understood as a limitation of the present invention.

[0123] When the camera recognizes an element, compares the element with a preset plane pattern, and determines that the element matches the black solid circle, it is determined that the camera recognizes the second marking module.

[0124] When the camera recognizes an element, compares the element with a preset plane pattern, and determines that the element matches a preset identification code, it is determined that the camera recognizes the third marking module.

[0125] When the camera recognizes two elements, compares the two elements with the preset plane pattern, and determines that one of the elements matches the preset identification code, it is determined that the camera recognizes the third marking module, and when it is determined that the other element matches the preset square, it is determined that the camera recognizes the first marking module. In other words, the camera recognizes the first marking module and the third marking module at the same time.

[0126] By analogy, the camera can determine which marking module or modules it has identified by comparing the identified graphic unit with the preset unit pattern, and thus can determine which first spatial mapping relationship or relationships and second spatial mapping relationships need to be acquired based on the identified marking modules.

[0127] In one example, the preset plane style should at least be the same as the unit style of all the recognition elements in the multiple marking modules. For example, when a checkerboard style, a symmetrical spot style, and an asymmetrical spot style are set in multiple marking modules, the preset unit style includes at least a checkerboard style, a symmetrical spot style, and an asymmetrical spot style, and of course, a grid style and a ChArUco style, and of course, a deep learning unit style. When the graphic units of multiple marking modules are all checkerboard styles, the preset unit style may include the same checkerboard style as the graphic unit, and of course, a grid style, and a circular array style (including a symmetrical spot style and an asymmetrical spot style). This example is only an illustrative example, and those skilled in the art should not be understood as a limitation of the present invention.

[0128] In one example, the multiple marking modules are designed to include a first marking module and a second marking module, and the graphic units on the first marking module and the second marking module are designed as an asymmetric dot matrix (i.e., an asymmetric spot array). In the case where the solid dots in the asymmetric dot matrix (i.e., an asymmetric spot pattern) are the same size as each other or an asymmetric dot matrix contains two asymmetric dot matrix sub-matrices, misidentification may occur. Therefore, the background colors of the first marking module and the second marking module and the foreground colors may be reversed (i.e., the gray value difference is 255), so that the two marking modules can be distinguished by the camera without causing misidentification.

[0129] In one example, a method for acquiring a first spatial mapping relationship (eg, a posture matrix, a spatial transformation matrix) of an asymmetric point matrix in a camera coordinate system includes the following steps:

[0130] The RGB image of the asymmetric dot matrix on the first marking module captured by the camera is grayed out, and then the grayscale image is subjected to white circle detection, and the center of all solid circles in the asymmetric dot matrix is ​​detected using the center detection algorithm, and the eccentricity error is subtracted from the coordinates of each center in the image coordinate system, and the pixel point corresponding to the difference is the center of the corresponding solid circle in the asymmetric dot matrix. Since the distance of each center in the asymmetric dot matrix is ​​known, the PNP (Perspective-n-Point) algorithm can be used to calculate the posture of the asymmetric dot matrix on the first marking module in the camera coordinate system, that is, the spatial transformation matrix or posture matrix M11 between the image coordinate system and the camera coordinate system of the asymmetric dot matrix on the first marking module is calculated.

[0131] Similarly, the RGB image of the asymmetric dot matrix on the second marking module captured by the camera is grayed, and then the grayscale image is subjected to black circle detection, and the center of all solid circles in the asymmetric dot matrix is ​​detected using the center detection algorithm, and the coordinates of each center in the image coordinate system are subtracted from the eccentricity error, and the pixel point corresponding to the difference is the center of the corresponding solid circle in the asymmetric dot matrix. Then the PNP (Perspective-n-Point) algorithm is used to calculate the posture of the asymmetric dot matrix on the second marking module in the camera coordinate system, that is, the spatial transformation matrix or posture matrix M12 between the image coordinate system and the camera coordinate system of the asymmetric dot matrix on the second marking module is calculated.

[0132] The above-mentioned space transformation matrix or posture matrix M11 is the first space mapping relationship between the first marking module and the camera, and the space transformation matrix or posture matrix M12 is the first space mapping relationship between the second marking module and the camera.

[0133] This example is only an illustrative example. Those skilled in the art can use existing methods to establish the first spatial mapping relationship between the marking module and the camera according to actual needs. For example, when multiple marking modules also include a third marking module, and the third marking module also uses a circular array style to design the graphics, then the first spatial transformation relationship between the image coordinate system and the camera coordinate system of the third marking module can be obtained by the method of obtaining the above-mentioned M11 and M12. Of course, other existing methods can also be used as a substitute. When the graphics on the marking module are checkerboard style, grid style, ChArUco style, and deep learning graphic style, existing methods can be used to obtain their first spatial mapping relationship between the image coordinate system and the camera coordinate system. This example is only an illustrative example, and those skilled in the art should not understand it as a limitation of the present invention.

[0134] In one embodiment, multiple marking modules are designed to include a first marking module, a second marking module and a third marking module. The graphics on the first marking module, the second marking module and the third marking module are all designed as asymmetric dot matrix. The first marking module and the third marking module are arranged relative to each other (for example, left and right, front and back, etc.), the second marking module is located between the first marking module and the third marking module, and the first marking module to the third marking module are arranged along the circumference of the magnetic stimulation coil.

[0135] In one example, the background colors of the first marking module and the third marking module are both black and the foreground colors are both white, and the background color of the second marking module is white and the foreground color is black. In order to avoid misidentification by the camera, the layout styles on the first marking module and the third marking module are designed to be mirror-symmetrical to each other.

[0136] In an example, the first spatial transformation relationship (for example, posture matrix, spatial transformation matrix) between the image coordinate system and the camera coordinate system of the first marking module, the second marking module and the third marking module are M21, M22, and M23, respectively, and the basic principles and steps of their acquisition methods are the same as those of the above-mentioned first spatial transformation relationships M11 and M12, and they will not be repeated here.

[0137] In one example, when the graphic unit of the identification element is the entire graphic (i.e., all graphic elements in the graphic), the graphics on the first marking module and the third marking module are designed as rotationally symmetrical layout styles with each other, that is, the graphic on the third marking module can be a graphic in the first marking module rotated by an angle α (0°<α<360°) along the center of the graphic or the center of the plane of the marking module where the graphic is located, and the arrangement of all graphic elements in the resulting graphic is different from the arrangement of all graphic elements in the first marking module, thereby forming the same type of graphics but two different graphic layout styles.

[0138] In one example, the graphic on the third marking module can be a graphic formed by rotating the graphic on the first marking module by 180° along the center of the graphic or the center of the plane of the marking module where the graphic is located, that is, the unit patterns on the first marking module and the third marking module are centrally symmetric (i.e., a form of rotational symmetry). The arrangement of all graphic elements in the resulting graphic is different from the arrangement of all graphic elements in the first marking module, thereby forming the same type of graphic but two different graphic layout styles.

[0139] For example, it can be: a solid circle located in the first row of the graphic in the first marking module is located in the last row of the third marking module in the graphic of the third marking module, and so on from top to bottom, until the solid circle located in the last row of the graphic of the first marking module is located in the first row of the graphic of the third marking module, and the asymmetric spot pattern of the first marking module coincides with the asymmetric spot pattern on the third marking module after rotating 180° clockwise (or counterclockwise) along the center of the asymmetric spot pattern, that is, the asymmetric spot pattern of the first marking module is centrally symmetrical with the asymmetric spot pattern of the graphic on the third marking module (that is, forming the same type of graphic but with different graphic layout styles).

[0140] In one example, the graphic in the first marking module can be rotated 90° clockwise or counterclockwise along the center of the graphic or the center of the plane of the marking module where the graphic is located to form a graphic on the third marking module, that is, the rotation angle between the graphics on the first marking module and the third marking module is 90°, and they are rotationally symmetrical to each other by 90°.

[0141] For example, it may be: a solid circle located in the first row of the graphic in the first marking module is located in the first column on the left side of the third marking module in the graphic of the third marking module (i.e., the solid circle in the first row is rotated counterclockwise to the first column on the left side), and so on, a solid circle in the last row of the graphic in the first marking module is located in the first column on the right side of the graphic of the third marking module, that is, the asymmetric spot pattern of the first marking module coincides with the asymmetric spot pattern on the third marking module after rotating 90° counterclockwise (of course, it can also be clockwise) along the center of the asymmetric spot pattern, so that the asymmetric spot pattern of the first marking module is rotationally symmetrical with the asymmetric spot pattern of the graphic on the third marking module by 90°.

[0142] By changing the above-mentioned layout style, the real-time positions of the graphic units of the first marking module and the third marking module can be distinguished, so that the first marking module located on the left side of the magnetic stimulation coil and the marking module located on the right side of the magnetic stimulation coil can be distinguished. In the subsequent tracking process, the first spatial mapping relationship between the graphic matching the marking module and the camera and the second spatial mapping relationship between the matching marking module and the tracking target can be determined and selected based on the distinction.

[0143] In one example, when at least two marking modules among the plurality of marking modules have the same type of graphics, and all graphic elements in the marking modules having the same type of graphics are arranged differently, the unit pattern includes all graphic layout patterns of different arrangements.

[0144] In one example, when at least two marking modules among a plurality of marking modules have the same type of graphics, and all graphic elements in the marking modules having the same type of graphics are arranged differently, comparing the graphic unit on the marking module recognized by the camera with a preset unit pattern to obtain a marking module matching the unit pattern includes:

[0145] The shape of at least one graphic element on the same marking module recognized by the camera is compared with the shapes of all graphic elements in at least one preset graphic element, and the layout style of all graphic elements on the same marking module recognized by the camera is compared with the preset layout style to obtain a marking module that matches the preset shape and the preset layout style.

[0146] That is, all the marking modules in at least two markings can be distinguished by the different graphic layout styles of all the marking modules in at least two marking modules, so as to obtain the corresponding first spatial mapping relationship and second spatial mapping relationship according to the determined marking modules.

[0147] For example, when there are at least two marking modules having the same type of graphics but different layout styles, they can be distinguished by identifying the pixel coordinates of the same graphic element in the graphics in different marking modules.

[0148] For example, when the graphics between the first marking module and the third marking module are all asymmetric spot patterns, the background colors are all black and the foreground colors are all white, the layout patterns of the graphics on the first marking module and the second marking module are symmetrical with each other by 90° counterclockwise rotation, and the graphic layout pattern on the first marking module is set to a preset layout pattern of an asymmetric spot pattern, when distinguishing the two marking modules, the graphic layout patterns on the first marking module and the second marking module in the captured image are respectively compared with the preset layout patterns to determine that the graphic layout patterns on the first marking module and the second marking module are asymmetric spot patterns. Afterwards, in the image coordinate system (i.e., pixel coordinate system), i.e., with the upper left corner of the image as the origin, the horizontal direction of the image (direction from the origin to the right) as the positive direction of the horizontal axis Xi of the image coordinate system, and the vertical direction of the image (direction from the origin to the bottom) as the positive direction of the longitudinal axis Yi of the image coordinate system, a straight line can be drawn along the center of the nth (e.g., the third) solid circle in the first row of the asymmetric spot pattern on the first marking module and the center of the first solid circle in the row, and it can be determined whether the straight line forms an acute angle or an obtuse angle with the horizontal axis Xi of the image where the first marking module is located, so as to distinguish. Similarly, on the image where the second marking module is located, a straight line can be drawn along the center of the nth solid circle in the first row and the center of the first solid circle in the row, and it can be determined whether the straight line forms an acute angle or an obtuse angle with the horizontal axis Xi of the image where the second marking module is located. For example, when the straight line on the first marking module forms an acute angle with the horizontal axis Xi, it is a figure on the first marking module, and the marking module is determined as the first marking module; when it forms an obtuse angle, it is a figure on the second marking module, and the marking module is determined as the second marking module.

[0149] For example, when the graphics between the first marking module and the third marking module are all asymmetric spot styles, the background colors are all black and the foreground colors are all white, the layouts of the graphics on the first marking module and the second marking module are symmetrical when rotated 90° counterclockwise, and the preset layout style is set as the layout style of the graphics on the first marking module, when it is determined that the layout style and arrangement style of the solid circles on the marking module are the same as the preset layout style, the center of the third solid circle in the first row on the marking module is connected to the center of the first solid circle in the first row to draw a straight line, and the straight line forms an acute angle with the horizontal axis Xi of the image in which it is located, then it is determined that the captured graphic is the graphic on the first marking module, and the marking module is the first marking module. When it is determined that the layout style of the solid circles on the marking module is the same as the preset layout style, but the arrangement is different, and it is determined that the graphic on the marking module is an arrangement formed by rotating the preset layout style 90° clockwise, the first column on the right side of the asymmetric spot pattern on the marking module is recorded as the first row, the first of the first column is recorded as the first of the first row, and the third of the first column is recorded as the third of the first row. Then, the center of the solid circle recorded as the first of the first row is connected with the center of the solid circle recorded as the third of the first row to draw a straight line. The straight line forms an obtuse angle with the horizontal axis Xi of the image in which it is located. Then, it is determined that the captured graphic is the graphic on the second marking module, and the marking module is the second marking module.

[0150] Of course, those skilled in the art can also distinguish the first marking module from the second marking module by comparing based on the number of solid circles in two adjacent rows. For example, if the number of solid circles (i.e., elements) in the first row of the marking module is greater than the number of solid circles in the second row, the layout style on the marking module is determined as the layout style of the first marking module, the marking module is determined as the first marking module, and the other is determined as the second marking module; if the number of solid circles (i.e., elements) in the first row of the marking module is less than the number of solid circles in the second row, the layout style on the marking module is determined as the layout style of the second marking module, the marking module is determined as the second marking module, and the other marking module is determined as the first marking module.

[0151] The above two examples are only an illustration of the present invention and should not be understood as a limitation of the present invention. Those skilled in the art can replace them with existing technologies as needed, as long as they can distinguish between graphics of the same type.

[0152] In one example, the camera can simultaneously determine that all preset parameters match the camera's field of view based on the color and graphic unit and their corresponding preset parameters. Those skilled in the art will appreciate that the identification element that matches the color parameter can be determined based on the color and color parameter first, thereby determining the identification module corresponding to the identification element. When the identification modules that match the color parameter are at least two, the identification element that matches the shape parameter is determined based on the graphic unit and the shape parameter, thereby determining the identification module corresponding to the identification element. When the identification modules that match the shape parameter are at least two, the optimal identification module among the at least two identification modules that match the shape parameter is screened out based on the comprehensive evaluation value. For example, when the colors of multiple marking modules are only black and white, the identification modules that match the color parameter can be screened out based on the color and color parameter first, and then the identification modules that match the shape parameter can be screened out based on the graphic unit and the shape parameter.

[0153] Of course, those skilled in the art can also first determine the identification element that matches the shape parameter based on the graphic unit and the shape parameter, thereby determining the identification module corresponding to the identification element. When the determined identification modules that match the shape parameter are at least two, then determine the identification element that matches the color parameter based on the color and the color parameter, thereby determining the identification module corresponding to the identification element. When the determined identification modules that match the color parameter are at least two, then screen out the best identification module among the at least two identification modules that match the shape parameter based on the comprehensive evaluation value. For example, when the colors of multiple marking modules are colorful, the graphic unit and the shape parameter can be used to screen out the identification module that matches the shape parameter first, and then the identification module that matches the color parameter can be screened out based on the color and the color parameter.

[0154] That is, the identification module can be determined based on color first, and then based on graphic unit; or the identification module can be determined based on graphic unit first, and then based on color. The order of determining color and graphic unit is not limited. Preferably, the determination is based on color first and then based on graphic unit. This example is only an illustrative example, and those skilled in the art should not be understood as a limitation of the present invention.

[0155] In one example, when at least two marking modules among the plurality of marking modules within the field of view of the camera match the preset parameters (such as color parameters and / or shape parameters),

[0156] Each marking module is evaluated based on the posture of the recognition elements on all the marking modules of at least two marking modules, the distance between the plane of the marking module where the recognition elements are located and the camera, and the size of the recognition elements recognized by the camera to screen out the optimal marking module,

[0157] Based on a first spatial mapping relationship between an optimal marking module and a camera and a second spatial mapping relationship between an optimal marking module and a tracking target, a position and posture of the tracking target under the camera is obtained.

[0158] In one example, the posture of the graphic is the angle between the plane with the graphic on the marking module and the optical axis of the camera, and the size of the identification element recognized by the camera is the ratio between the actual area of ​​the identification element recognized by the camera in the image coordinate system and the image area. This example is only an illustrative example, and those skilled in the art should not be understood as a limitation of the present invention.

[0159] In one example, the marking modules are evaluated based on the posture of the recognition element on the same marking module of the at least two marking modules, the distance between the plane with the recognition element on the marking module and the camera, and the size of the recognition element recognized by the camera to screen out the optimal marking module, including:

[0160] The angle, the distance, and the ratio of the identification elements on all the marking modules in the at least two marking modules are based on respective weights to obtain an angle evaluation value, a distance evaluation value, and a size evaluation value of the identification element on each marking module;

[0161] Adding the angle evaluation value, the distance evaluation value and the size evaluation value of the same identification element to obtain a comprehensive evaluation value of the identification element;

[0162] The comprehensive evaluation values ​​of the identification elements of all the marking modules in at least two marking modules are sorted in descending order, and the marking module corresponding to the maximum comprehensive evaluation value is determined as the optimal marking module.

[0163] In one example, the formula for the comprehensive evaluation value score is expressed as:

[0164] score = W1A + W2D + W3S;

[0165] Among them, A represents the angle evaluation value, W1 represents the weight of the angle evaluation value, D represents the distance evaluation value, W2 represents the weight of the distance evaluation value, S represents the size evaluation value, W2 represents the weight of the size evaluation value, and W1+W2+W3=1.

[0166] In one example, the range of W1 is set to 0.1 to 0.35, the range of W2 is set to 0.1 to 0.35, and the range of W3 is set to 0.1 to 0.35.

[0167] Those skilled in the art should understand that the selection of each weight should be set according to the requirements of the application scenario. This example is only an illustrative example and should not be construed as a limitation to the present invention.

[0168] In one example, the actual area of ​​the recognition element recognized by the camera is the area occupied by the recognized part of the recognition element in the image. The original area of ​​the recognition element is the area occupied by the recognition element in the image when it is completely captured by the camera.

[0169] In one example, when calculating the comprehensive evaluation value of the identification element, the angle evaluation value, the distance evaluation value, and the size evaluation value may be normalized first. The angle evaluation value may use 360° as the normalization standard, for example, the angle may be normalized by dividing the angle by 360°. The distance evaluation value may use the farthest distance captured by the camera as the normalization standard. The size evaluation value may use the area occupied by the identification element in the image when it is completely captured by the camera as the normalization standard. Those skilled in the art may understand that the purpose of normalization is to normalize the angle evaluation value, the distance evaluation value, and the size evaluation value to an order of magnitude close to each other, so that the optimal marking module may be obtained based on the comprehensive evaluation value, thereby making the obtained position and posture of the tracked target more accurate.

[0170] According to another embodiment of the present invention, an electronic device is provided. The electronic device (not shown) includes a memory (not shown) and at least one processor (not shown). The memory is communicatively connected to at least one processor, for example, the memory can be communicatively connected to one, two or more processors. In one example, the communication connection can be connected through a physical device or line, or through a wireless signal, that is, as long as communication can be achieved between the memory and the processor. In one example, a program or instruction is stored in the memory, and when the program or instruction is executed by the at least one processor, the electronic device is used to implement the above-mentioned method for tracking the posture recognition of the target.

[0171] In one example, the processor may be a microprocessor, such as a general-purpose processor such as a graphics processing unit (GPU), a central processing unit (CPU), a digital signal processor (DSP), etc. In one example, the processor may also be a microprocessor core implemented by a hardware circuit, such as a microprocessor core implemented in a hardware logic component by a reconfigurable logic, and the hardware logic component includes a field programmable gate array (FPGA), a complex programmable logic device (CPLD), an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), a system on a chip (SOC), etc.

[0172] In one example, the processor may also be a virtual processor, which may be a virtual processor with the characteristics of an Intel x86 processor, or a virtual processor with the characteristics of a PowerPC processor. Preferably, the processor is a graphics processor. In one example, the processor may be a single-core processor or a multi-core processor.

[0173] In one example, the memory includes a volatile memory (i.e., a random access memory) and a non-volatile memory. The volatile memory includes a main memory, a cache, etc., and the non-volatile memory includes an auxiliary memory, etc. In one example, the memory can be set as a remote memory, and the remote memory can be connected to the processor via a network (wired network or wireless network). The network includes, but is not limited to, a wide area network, a local area network, a metropolitan area network, a personal area network, the Internet, a satellite communication network, and any combination thereof.

[0174] In one example, the processor executes a program based on the program obtained from the memory to create a corresponding task thread and execute the thread. In one example, the processor obtains the program from the external memory based on the read instruction in the memory to create a corresponding task thread and execute the thread. The above thread is used to implement the posture recognition method of the tracking target.

[0175] According to another embodiment of the present invention, a navigation device is provided. The navigation device includes a camera 20 (such as Figure 3 The navigation device further comprises an electronic device connected to the camera signal (such as a wired connection or a wireless connection), and the electronic device is the electronic device mentioned above. The navigation device also includes a navigation interface, and the navigation device performs navigation according to the real-time position and posture of the tracking target provided by the electronic device. During the navigation process, the navigation interface provides doctors with more accurate and convenient navigation guidance, improves treatment efficiency and treatment effect, and improves the difficulty of navigation operation. The complex operation process is reduced.

[0176] According to another embodiment of the present invention, a readable storage medium is provided. The "readable storage medium" of an embodiment of the present invention refers to any medium that participates in providing a program or instruction to a processor for execution. The medium can take a variety of forms, including but not limited to non-volatile media, volatile media and transmission media. Non-volatile media include, for example, optical disks or disks, such as storage devices. Volatile media include dynamic memory, such as main memory. Transmission media include coaxial cables, copper wires and optical fibers, including wires containing buses. Transmission media can also take the form of sound waves or light waves, such as sound waves or light waves generated during radio frequency (RF) and infrared (IR) data communications. Common forms of readable storage media include, for example, floppy disks, flexible disks, hard disks, magnetic tapes, any other magnetic media, CD-ROMs, DVDs, any other optical media, punch cards, paper tapes, any other physical media with hole patterns, RAMs, PROMs and EPROMs, FLASH-EPROMs, any other memory chips or boxes, carriers as described below, or any other media from which a computer can read.

[0177] The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, the above-mentioned posture recognition method of the tracking target is executed.

[0178] In one example, the readable storage medium is arranged on a server in the form of a memory, such as a cloud server. The cloud server is also provided with a processor, and the processor executes the program or instruction stored in the memory. The processor may be a central processing unit (CPU). In one example, the cloud server may be a virtual server formed by mapping a physical server through virtualization technology. Among them, there may be one or more physical servers, and when there are multiple physical servers, the cloud server may be a virtual server formed by mapping a server cluster through virtualization technology. The virtual server may also be one or more. In one example, the cloud server may be provided to users through a cloud platform.

[0179] like Figure 3As shown, according to another embodiment of the present invention, a transcranial magnetic therapy instrument 100 is provided. The transcranial magnetic therapy instrument includes a coil (not shown, such as a magnetic stimulation coil) and a main body 10 connected to the coil, and the main body is provided with an electronic device or a readable storage medium, the electronic device is the above-mentioned electronic device, and the readable storage medium is the above-mentioned readable storage medium. The main body also includes at least one of a display module 11, a support frame 12, a mechanical arm (not shown) and a treatment chair (or a treatment bed, not shown). The transcranial magnetic therapy instrument provided by the present invention can be used for repetitive transcranial magnetic stimulation (rTMS). Repetitive transcranial magnetic stimulation (rTMS) is a painless and non-invasive neuromodulation technology that directly acts on the cerebral cortex through electromagnetic-electric conversion to produce a series of physiological and biochemical changes. From the invention of the first transcranial magnetic stimulator by Professor Baker of the United Kingdom in 1985, to the successful development of the first transcranial magnetic stimulator in China by Professor Liao Jiahua's team in 1988, to the re-emergence of the four new brain science technologies in the 21st century, transcranial magnetic stimulation technology has undergone more than 30 years of in-depth research and attempts, and its clinical value has been confirmed by experts from various disciplines. This technology was first used as an important auxiliary treatment for depression. With the in-depth understanding of rTMS, its indications have been continuously expanded. It has now been widely used in a variety of diseases involving clinical departments such as neurology, neurosurgery, psychiatry, rehabilitation medicine, and pain medicine (different departments of application), including stroke, brain trauma, spinal cord injury, neuropathic pain, tinnitus, Parkinson's disease, etc. (disease categories).

[0180] The method for recognizing the position and posture of a tracking target, the electronic device, the navigation device, the readable storage medium and the transcranial magnetic therapy apparatus according to the present invention have at least one of the following advantages:

[0181] (1) The position and posture recognition method, electronic device, navigation device, readable storage medium and transcranial magnetic therapy device provided by the present invention adopt a method of automatic graphic recognition, so that the tracking target can be moved at any time and at will within the visible range of the camera, and the position and posture of the tracking target can be obtained without re-registration;

[0182] (2) The target position recognition method, electronic device, navigation device, readable storage medium and transcranial magnetic therapy device provided by the present invention increase the range of the camera to recognize the target through the design of multiple marking modules, so that the target can be moved and rotated over a larger range during navigation, control and other processes;

[0183] (3) The target posture recognition method, electronic device, navigation device, readable storage medium and transcranial magnetic therapy device provided by the present invention can automatically recognize graphics in real time, so that when the target is moving or rotating, the camera can automatically select and switch the marking module to be tracked and identified to track the target in real time.

[0184] Although some embodiments of the present general inventive concept have been shown and described, it will be appreciated by those skilled in the art that changes may be made to these embodiments without departing from the principles and spirit of the present general inventive concept, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for tracking a target's posture recognition, the method comprising the following steps: Based on preset parameters, identify and determine a marking module that matches the preset parameters among a plurality of marking modules located within the field of view of the camera; Based on a first spatial mapping relationship between a marking module matching preset parameters and the camera, and a second spatial mapping relationship between a marking module matching preset parameters and a tracking target provided with the multiple marking modules, a position and posture of the tracking target under the camera is obtained.

2. The posture recognition method according to claim 1, wherein: Identifying and determining at least one marking module matching the preset parameters among a plurality of marking modules located within the field of view of a camera based on preset parameters includes: Comparing the identification elements on the marking module identified by the camera with the preset parameters to obtain the marking module matching the preset parameters; Based on the determined marking module that matches the preset parameters, the first spatial mapping relationship between the marking module and the camera and the second spatial mapping relationship between the marking module and the tracking target are acquired.

3. The posture recognition method according to claim 2, wherein: The preset parameters at least include identification elements of all marking modules among the multiple marking modules.

4. The posture recognition method according to claim 3, wherein: The identification element includes a color and / or a graphic unit of the marking module, and the preset parameter includes a color parameter and / or a shape parameter corresponding to the identification element of the marking module.

5. The posture recognition method according to claim 4, wherein: Comparing the identification element on the marking module identified by the camera with the preset parameters to obtain the marking module matching the preset parameters, including: When the identification element is color, the color on the marking module identified by the camera is compared with the color parameter to obtain a marking module matching the color parameter; and / or When the identification element is a graphic unit, the graphic unit on the marking module identified by the camera is compared with the shape parameter to obtain a marking module matching the shape parameter.

6. The posture recognition method according to claim 5, wherein: The shape parameters include unit styles, and the unit styles include three-dimensional styles and plane styles. Comparing the graphic unit on the marking module recognized by the camera with the shape parameter to obtain the marking module matching the shape parameter, including: The graphic unit on the marking module recognized by the camera is compared with a preset unit pattern to obtain a marking module matching the unit pattern.

7. The posture recognition method according to claim 6, wherein: Each of the plurality of marking modules is provided with a graphic, and the graphic is composed of at least one graphic element. The graphic unit is at least one graphic element in the graphic, and the plane pattern corresponds to all the graphic units on the multiple marking modules.

8. The posture recognition method according to claim 7, wherein: The type of the graphics includes at least one of a checkerboard pattern, a circular array pattern, a grid pattern, a ChArUco pattern, and a deep learning graphics pattern. The circular array pattern includes a symmetrical spot pattern and an asymmetrical spot pattern. The graphic units between adjacent marking modules are arranged as any one of different graphic types, different background colors and different foreground colors or any combination thereof.

9. The posture recognition method according to claim 8, wherein: When at least two marking modules among the plurality of marking modules have the same type of graphics, and all graphic elements in the marking modules having the same type of graphics are arranged in different ways, the unit pattern includes graphic layout patterns of all different arrangements.

10. The posture recognition method according to claim 9, wherein: When at least two marking modules among the plurality of marking modules have the same type of graphics, and all graphic elements in the marking modules having the same type of graphics are arranged in different ways, comparing the graphic units on the marking modules recognized by the camera with a preset unit pattern to obtain a marking module matching the unit pattern, including: The shape of at least one graphic element on the same marking module recognized by the camera is compared with the shapes of all graphic elements in at least one preset graphic element, and the layout style of all graphic elements on the same marking module recognized by the camera is compared with the preset layout style to obtain a marking module that matches the preset shape and the preset layout style.

11. The posture recognition method according to claim 5, wherein: The color parameters include background color parameters and foreground color parameters. The background color and foreground color of the marking module are different colors. When the identification element is color, comparing the color on the marking module identified by the camera with the color parameter to obtain a marking module matching the color parameter includes: The background color on the marking module recognized by the camera is compared with the background color parameter, and the foreground color on the marking module recognized by the camera is compared with the foreground color parameter to obtain a marking module matching the background color parameter and the foreground color parameter.

12. The location recognition method according to claim 2, wherein: When at least two of the multiple marking modules located within the camera's field of view match the preset parameters, The marking modules are evaluated based on the posture of the recognition element on the same marking module of the at least two marking modules, the distance between the plane with the recognition element on the marking module and the camera, and the size of the recognition element recognized by the camera to screen out the optimal marking module, Based on a first spatial mapping relationship between an optimal marking module and a camera and a second spatial mapping relationship between an optimal marking module and a tracking target, a position and posture of the tracking target under the camera is obtained.

13. The posture recognition method according to claim 12, wherein: The posture of the identification element is the angle between the plane with the identification element on the marking module and the optical axis of the camera. The size of the identification element recognized by the camera is the ratio between the actual area of ​​the identification element recognized by the camera and the image area.

14. The posture recognition method according to claim 13, wherein: The marking module is evaluated based on the posture of the recognition element on the same marking module of the at least two marking modules, the distance between the plane with the recognition element on the marking module and the camera, and the size of the recognition element recognized by the camera to screen out the optimal marking module, including: The angle, distance and proportion of the identification elements on all the marking modules in the at least two marking modules are calculated based on respective weights to obtain an angle evaluation value, a distance evaluation value and a size evaluation value of the identification element on each marking module; Adding the angle evaluation value, the distance evaluation value and the size evaluation value of the same identification element to obtain a comprehensive evaluation value of the identification element; The comprehensive evaluation values ​​of the recognition elements of all the marking modules in at least two marking modules are sorted in descending order, and the marking module corresponding to the maximum comprehensive evaluation value is determined as the optimal marking module.

15. The posture recognition method according to claim 1, wherein: Based on the first spatial mapping relationship, the second spatial mapping relationship and the real-time position of the marking module recognized by the camera, the position and posture of the tracking target under the camera is obtained.

16. The posture recognition method according to claim 15, wherein: When the position of the tracking target changes, determining, based on preset parameters, a marking module recognized by the camera in real time and matching the preset parameters; Based on the first spatial mapping relationship between all the marking modules in the marking modules recognized by the camera in real time and the camera, the second spatial mapping relationship between the marking modules recognized by the camera in real time and the tracking target, and the real-time position of the marking module, the real-time posture of the tracking target under the camera is obtained.

17. An electronic device, characterized in that: The electronic device includes a memory and at least one processor, the memory is communicatively connected to the at least one processor, a program or instruction is stored in the memory, and when the program or instruction is executed by the at least one processor, the electronic device is used to implement the posture recognition method of the tracking target as described in any one of claims 1-16.

18. A navigation device, characterized in that: The navigation device includes a camera and an electronic device connected to the camera signal, and the electronic device is the electronic device according to claim 17.

19. A readable storage medium, characterized in that: The readable storage medium stores a program or instruction, and when the program or instruction is executed by the processor, it executes the posture recognition method of the tracking target described in any one of claims 1-16.

20. A transcranial magnetic therapy device, characterized in that: The transcranial magnetic therapy device includes a coil and a main body connected to the coil, and the main body is provided with the electronic device or the readable storage medium. The electronic device is the electronic device according to claim 17, and the readable storage medium is the readable storage medium according to claim 19.

Images