A positioning guide method and device, electronic equipment and storage medium

Abstract

The application discloses a positioning guidance method and device, electronic equipment and storage medium, and belongs to the technical field of positioning guidance. The method comprises the following steps: acquiring a marker feature of a first marker; acquiring image coordinate values of each reflective ball in a second marker relative to an image coordinate system; determining a static coordinate value of the second marker relative to a first marker coordinate system; identifying a second rotation and translation matrix of the first marker coordinate system and an optical positioning system; determining a theoretical coordinate value of the second marker relative to the optical positioning system according to the static coordinate value; determining a transformation relationship between the image and the optical positioning system according to the theoretical coordinate value and the image coordinate value, and performing positioning guidance based on the transformation relationship. According to the technical scheme, the coordinate relationship between the first marker and the second marker is established, the coordinate value of the second marker can be calculated by using the first marker, and the positioning guidance precision is improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of positioning guidance, and particularly relates to a positioning guidance method and device, an electronic device and a storage medium. BACKGROUND

[0002] With the increasing maturity of positioning technology, more and more fields begin to use positioning technology to assist in performing tasks, and people have higher requirements for positioning accuracy. The guidance of positioning has become more and more important.

[0003] The current positioning guidance mode mainly uses the method of pasting markers to mark the positioning target or the positioning range. The coordinate values of the markers are recognized in real time by an optical positioning system, and the coordinate conversion relationship between the optical positioning system and the image is calculated to achieve the purpose of positioning guidance by using the image view. However, the current positioning method has high requirements for the positioning target. Not only the pose of the positioning target cannot change, but also the markers pasted on the positioning target cannot be displaced, fall off, and deformed. Since the above requirements are not easy to be realized in actual application, the use of the prior art for positioning guidance is prone to cause deviation of the positioning accuracy. SUMMARY

[0004] The purpose of the embodiments of the application is to provide a positioning guidance method and device, an electronic device and a storage medium, which can solve the problem that the current positioning guidance using pastable markers is prone to cause deviation of the positioning accuracy. By establishing the coordinate relationship between the first marker and the second marker, the coordinate values of the second marker can be calculated without considering whether the pose of the positioning target changes and whether the second marker falls off, thereby improving the accuracy and reliability of the positioning guidance.

[0005] In a first aspect, the embodiments of the application provide a positioning guidance method, which comprises:

[0006] obtaining a marker feature of a first marker, and obtaining image coordinate values of each reflective ball in a second marker collected after the second marker is fixed relative to an image coordinate system;

[0007] identifying the static coordinate values of the second marker relative to the first marker coordinate system after the first marker is fixed;

[0008] identifying the second rotation and translation matrix of the first marker coordinate system relative to the optical positioning system coordinate system after the pose changes, and determining the theoretical coordinate values of the second marker relative to the optical positioning system coordinate system according to the static coordinate values;

[0009] According to the theoretical coordinate value and the image coordinate value, a transformation relationship of the image relative to the optical positioning system coordinate system is determined, and positioning guidance is performed based on the transformation relationship.

[0010] Further, after the first marker is fixed, a static coordinate value of the second marker relative to the first marker coordinate system is determined, including:

[0011] After the first marker is fixed, a relative coordinate value of the second marker relative to the optical positioning system coordinate system is determined, and a first rotation and translation matrix of the first marker coordinate system relative to the optical positioning system coordinate system is determined.

[0012] According to the relative coordinate value and the first rotation and translation matrix, a static coordinate value of the second marker relative to the first marker coordinate system is determined.

[0013] Further, a theoretical coordinate value of the second marker relative to the optical positioning system coordinate system is determined according to the static coordinate value, including:

[0014] A second rotation and translation matrix after the pose change is read.

[0015] According to the static coordinate value and the second rotation and translation matrix, a theoretical coordinate value of the second marker relative to the optical positioning system coordinate system is determined.

[0016] Further, according to the theoretical coordinate value and the image coordinate value, a transformation relationship of the image relative to the optical positioning system coordinate system is determined, including:

[0017] The theoretical coordinate value and the image coordinate value are matched, and a singular value decomposition algorithm is used to determine the transformation relationship of the image relative to the optical positioning system coordinate system.

[0018] Further, the image coordinate value of each reflective ball in the second marker collected after the second marker is fixed relative to the image coordinate system includes:

[0019] The two-dimensional image original data of the positioning target is imported into a three-dimensional model reconstruction system, and a three-dimensional visual digital model is established.

[0020] According to the image positions of each reflective ball of the second marker in the two-dimensional image original data, position information of each reflective ball in the second marker in the three-dimensional visual digital model is determined.

[0021] According to the position information in the three-dimensional visual digital model, the image coordinate value of each reflective ball in the second marker relative to the image coordinate system is determined.

[0022] Further, the first marker is a rigid tool in the shape of "X", which is fixed with the positioning target.

[0023] Further, the second marker is pasted on the surface of the positioning target.

[0024] In a second aspect, an embodiment of the present application provides a positioning and guiding device, which comprises:

[0025] a marker acquisition module, configured to acquire a marker feature of the first marker, and acquire image coordinate values of each reflective ball in the second marker relative to an image coordinate system after the second marker is fixed;

[0026] a static coordinate value determination module, configured to identify the first marker after being fixed, and determine a static coordinate value of the second marker relative to a first marker coordinate system;

[0027] a coordinate value updating module, configured to determine a second rotation and translation matrix of the first marker coordinate system relative to the optical positioning system coordinate system, and determine a theoretical coordinate value of the second marker relative to the optical positioning system coordinate system according to the static coordinate value;

[0028] a positioning and guiding module, configured to determine a transformation relationship of the image relative to the optical positioning system coordinate system according to the theoretical coordinate value and the image coordinate value, and perform positioning and guiding based on the transformation relationship.

[0029] In a third aspect, an embodiment of the present application provides an electronic device, which comprises a processor, a memory, and a program or instruction stored in the memory and executable on the processor, and the program or instruction is executed by the processor to implement the steps of the method in the first aspect.

[0030] In a fourth aspect, an embodiment of the present application provides a readable storage medium, which stores a program or instruction, and the program or instruction is executed by a processor to implement the steps of the method in the first aspect.

[0031] In a fifth aspect, an embodiment of the present application provides a chip, which comprises a processor and a communication interface, the communication interface is coupled with the processor, and the processor is configured to run a program or instruction to implement the method in the first aspect.

[0032] In the embodiment of the present application, the marker feature of the first marker is acquired, and the image coordinate values of each reflective ball in the second marker collected after the second marker is fixed relative to the image coordinate system are acquired; the static coordinate value of the second marker relative to the first marker coordinate system is determined after the first marker is fixed; the second rotation and translation matrix of the first marker coordinate system relative to the optical positioning system coordinate system is determined after the pose is recognized; the theoretical coordinate value of the second marker relative to the optical positioning system coordinate system is determined according to the static coordinate value; the transformation relationship of the image relative to the optical positioning system coordinate system is determined according to the theoretical coordinate value and the image coordinate value, and positioning guidance is performed according to the transformation relationship. By establishing the coordinate relationship between the first marker and the second marker, the coordinate value of the first marker can be directly positioned by the optical positioning system coordinate system, and then the coordinate value of the second marker is calculated, thereby avoiding the problem of inaccurate positioning caused by displacement, falling off and deformation of the second marker, and effectively improving the positioning accuracy and reliability of the positioning guidance. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 is a flowchart of a positioning guidance method provided by an embodiment of the present application;

[0034] Figure 2 is a flowchart of a positioning guidance method provided by an embodiment of the present application;

[0035] Figure 3 is a structural diagram of a positioning guidance device provided by an embodiment of the present application;

[0036] Figure 4 is a structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0037] In order to make the objectives, technical solutions and advantages of the present application clearer, the specific embodiments of the present application are described in detail below with reference to the drawings. It can be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application. In addition, it should be noted that, for the convenience of description, only the parts related to the present application are shown in the drawings, but not all the contents. Before discussing the example embodiments in more detail, it should be mentioned that some example embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The processes can be terminated when the operations are completed, but can also have additional steps not included in the drawings. The processes can correspond to methods, functions, procedures, subroutines, subprograms, etc.

[0038] With reference to the drawings, the technical solutions in the embodiments of the present application will be clearly described below. Obviously, the described embodiments are only some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art are within the scope of the present application.

[0039] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of a kind and do not limit the number of objects, for example, the first object can be one or more. In addition, "and / or" in the specification and claims indicates at least one of the connected objects, and the character " / ", generally indicates that the objects before and after are in a "or" relationship.

[0040] The positioning guidance method, device, electronic device and storage medium provided by the embodiments of the present application will be described in detail below with reference to the drawings and specific embodiments and their application scenarios.

[0041] Embodiment one

[0042] Figure 1 is a flowchart of the positioning guidance method provided by the first embodiment of the present application. The method is executed by a host computer. As shown in Figure 1 , the method specifically comprises the following steps:

[0043] S101, obtaining the marker characteristics of the first marker; and obtaining the image coordinate values of each reflective ball in the second marker relative to the image coordinate system after the second marker is fixed and collected;

[0044] Firstly, the use scenario of the technical solution can be a scenario that needs to use positioning guidance. Specifically, it can be a scenario of accurately corresponding the image data of the positioning target and the anatomical structure, and it can also be a scenario of tracking the execution tool and then guiding the task process. By using positioning guidance to guide the task process, the accuracy of positioning the positioning target can be improved.

[0045] Based on the above use scenario, it can be understood that the execution subject of the present application can be a development system in the host computer, such as a control software and a processor with the function of issuing commands to the lower computer, etc., which is not limited here.

[0046] In the scheme, the host computer can be a computer capable of directly issuing control commands, and various signal changes can be displayed on the screen. The command issued by the host computer is first given to the lower computer, which then interprets the corresponding time sequence signal according to the command to directly control the corresponding device. The lower computer reads the device state data from time to time and converts it into a digital signal to feed back to the host computer. Specifically, the host computer in the scheme can issue control and calculation instructions to the lower computer to control the lower computer to calculate the coordinate value of the marker.

[0047] In the scheme, the marker can be an object arranged around the positioning target and capable of being used for positioning by the positioning guide system. Specifically, the positioning guide system includes an optical positioning system and an image system, and the marker needs to be capable of being recognized by the optical positioning system and presented in the image. The marker feature of the first marker can be a feature capable of identifying and distinguishing the first marker from other markers, specifically, one or more of the shape, color, and structure of the first marker. The first marker can be an "X"-shaped rigid marker made of metal and plastic and the like, which is not easy to deform. During the positioning guide process, the marker is fixed near the positioning target and the position of the marker cannot be moved. The "X"-shaped rigid marker can be composed of an "X"-shaped rigid support and a spherical or square solid body with a reflective property fixed at the four end points of the support. The second marker can be a spherical or square solid body with a reflective effect pasted on the surface of the positioning target during the execution process and capable of being recognized by the optical positioning system and the image. Specifically, it can be a reflective marker ball and the like. The surface of the reflective marker ball is composed of tens of thousands of microbeads, which can reflect infrared light and accurately position each marker point.

[0048] In the scheme, the image can be a two-dimensional digital tomographic image sequence formed by a computer for target positioning, in a non-invasive manner, processing three-dimensional body data to transform it into an image with a stereoscopic effect, specifically, it can be through image three-dimensional reconstruction software, using MRI (Magnetic Resonance Imaging) and CT (Computed Tomography) image data for three-dimensional reconstruction, and obtaining the stereoscopic image of the positioning target. The principle of MRI imaging is that the energy or signal released has different attenuations in different structural environments inside the material. Through the detection of the electromagnetic wave emitted by the external magnetic field, the position and type of the atomic nucleus constituting the object can be known, and the structure image inside the object can be drawn. The principle of CT imaging is to use an X-ray beam to scan a certain thickness of the layer of the positioning target, and the X-ray that passes through the layer is received by the detector, converted into visible light, and then converted into an electrical signal by photoelectric conversion. After analog / digital conversion, it is converted into a digital signal and input into a computer for processing and drawing images. The image can be used to observe the structure inside the positioning target by the operator, position the execution tool and the positioning target, and then guide the task process.

[0049] In the scheme, the host computer can obtain the marking features of the first marker through the image, specifically, it can obtain the shape, position and number of the reflective markers of the marker. The second marker can be fixed on the surface near the positioning target by using detachable glue and the like. The image coordinate system can be a three-dimensional coordinate system reconstructed by importing the image data into the three-dimensional reconstruction software, and the three-dimensional reconstruction software can be Mimics, Simpleware, 3D-Doctor and the like. The image coordinate value of each reflective ball in the second marker relative to the image coordinate system is obtained through the three-dimensional reconstruction software.

[0050] Based on the above embodiment, optionally, the first marker is an "X" shaped rigid tool fixed with the positioning target.

[0051] In the scheme, the "X" shape rigid tool can be made of rigid material, the overall structure is "X" shape, and the positioning tool is provided with reflective markers. Specifically, the reflective markers can be reflective marker balls installed by punching at four end points of the structure. The marker balls are used for positioning and identification of the positioning guide system, and the positions of each marker ball at the end points remain unchanged. The "X" shape rigid tool is provided with a base for supporting the tool and fixing the tool with the positioning target, and the tool itself is fixed with the base and cannot rotate. The rigid material can be a material that is not easy to deform under external force, specifically, a material with high hardness such as plastic, metal and synthetic material. The material requires light weight and moderate volume.

[0052] In the scheme, the fixed position of the first marker can be near the positioning target. Specifically, the fixed position is moderately distant from the positioning target, and the positioning accuracy of the positioning guide system to the first marker is maintained as much as possible without affecting the execution personnel's execution vision and task process.

[0053] In the scheme, the "X" shape rigid tool is used as the first marker, which can avoid the deformation and failure of the marker caused by the contact between the execution tool and the marker during the execution process. Meanwhile, the marker structure is set to "X" shape, and reflective marker balls can be arranged at each end point of the marker, which is equivalent to fixing a rigid tool and obtaining four positioning coordinates at the same time, thereby improving the positioning accuracy. The first marker is fixed near the positioning target by using the Kirschner wire, which can effectively prevent the marker from falling off and shifting, thereby improving the positioning accuracy.

[0054] Based on the above embodiment, optionally, the second marker is pasted on the surface of the positioning target.

[0055] In the scheme, the second marker can be a marker with a reflective property, capable of being positioned by an optical positioning system and image recognition. Specifically, it can be a reflective marker ball coated with a reflective material on the surface. The reflective marker ball can reflect infrared light emitted by the positioning guide system back into the infrared light in the lens of the positioning guide system, which is received by the sensor matrix on the multiple lenses, and the three-dimensional coordinates of the ball center are obtained. The size of the second marker is related to the area of the positioning target. If the positioning target area is smaller, the second marker is smaller. The second marker is pasted near the positioning target by detachable glue. The detachable glue can be a glue with safety, green and absorbable characteristics for the positioning target. Specifically, the detachable glue can directly contact the surface of the positioning target without affecting the state of the positioning target and the task progress. When positioning the positioning target, at least three second markers are needed to obtain the three-dimensional coordinates, and the distance between two markers cannot be too close, otherwise it is easy to cause the system to fail to correctly identify. At the same time, the markers need to be arranged in a non-linear and asymmetric manner, so that the system can correctly identify the direction and coordinate value.

[0056] In the scheme, the second marker is pasted on the surface near the positioning target, which can be positioned at a position close to the positioning target, improving the positioning guide accuracy. At the same time, since the second marker is pasted by detachable glue, when the second marker affects the field of view of the execution personnel, it can be unloaded at any time, avoiding the influence of the marker on the task progress.

[0057] Based on the above embodiment, optionally, the image coordinate values of each reflective ball in the second marker relative to the image coordinate system are acquired after the second marker is fixed, including:

[0058] The two-dimensional image original data of the positioning target is imported into a three-dimensional model reconstruction system to establish a three-dimensional visual digital model;

[0059] According to the image positions of each reflective ball in the second marker in the two-dimensional image original data, the position information of each reflective ball in the second marker in the three-dimensional visual digital model is determined;

[0060] According to the position information in the three-dimensional visual digital model, the image coordinate values of each reflective ball in the second marker relative to the image coordinate system are determined.

[0061] In the present scheme, three-dimensional reconstruction refers to establishing a mathematical model suitable for computer representation and processing of a three-dimensional object, which is the basis for processing, operating and analyzing its properties in a computer environment, and is also a key technology for establishing a virtual reality representing the objective world in a computer. In computer vision, three-dimensional reconstruction refers to the process of reconstructing three-dimensional information from a single view or multiple views. Since the information of a single view is not complete, three-dimensional reconstruction needs to use experience knowledge, while multi-view three-dimensional reconstruction (similar to human binocular positioning) is relatively easy, and the method is to calibrate the camera first, i.e. to calculate the relationship between the image coordinate system of the camera and the world coordinate system, and then use the information in multiple two-dimensional images to reconstruct three-dimensional information. The three-dimensional visual digital model can be a sequence of two-dimensional image digital tomograms extracted by image recognition, forming three-dimensional body data, and using the data to transform the two-dimensional image into an image with intuitive three-dimensional effect. Specifically, the upper computer can obtain the two-dimensional image of the positioning target, and then issue an instruction to the lower computer. The lower computer imports the original data of the two-dimensional image into a three-dimensional model reconstruction system, and the system reconstructs a three-dimensional visual digital model through coding.

[0062] In the present scheme, the two-dimensional image original data can be the positioning target data and the image position data of each reflective ball of the second marker. By coding, the two-dimensional image is corresponded to the three-dimensional visual digital model to determine the position information of each reflective ball in the second marker in the three-dimensional visual digital model. According to the position information in the three-dimensional visual digital model, the image coordinate value of each reflective ball in the second marker relative to the image coordinate system is determined. The target positioning influence coordinate system can be the coordinate system of the three-dimensional visual digital model. By importing the two-dimensional image data into a positioning guide software system, a three-dimensional visual digital model is reconstructed, and the image coordinate value of each reflective ball in the second marker relative to the image coordinate system is extracted by algorithm, which is recorded as .

[0063] In the present scheme, by reconstructing a three-dimensional visual digital model using two-dimensional image data, the image coordinate value of each reflective ball in the second marker relative to the image coordinate system can be obtained, and the positioning of the positioning target can be obtained using the image coordinate value, thereby improving the safety of execution.

[0064] S102, after recognizing that the first marker is fixed, determining the static coordinate value of the second marker relative to the first marker coordinate system;

[0065] In the present solution, the first marker coordinate system can be a coordinate system established by the first marker itself, specifically, it can be a tool file for manufacturing the first marker using an optical positioning system, which contains coordinate values of each reflective marker ball of the first marker in its own coordinate system. Since the optical positioning system has software with the functions of identifying the first marker and manufacturing the tool file, the software can be used to define a coordinate system for the first marker, which is the first marker coordinate system, and calculate the coordinate values of the marker balls installed at each end point of the first marker in the first marker coordinate system, denoted as . The static coordinate values of the second marker relative to the first marker coordinate system can be that the relative positions of each reflective marker ball in the second marker to the marker balls at all end points of the first marker remain unchanged, that is, the relative distances of the marker balls in the two types of markers remain unchanged, so that after the second marker is detached or unloaded, the coordinate values of the first marker can be obtained by the optical positioning system, and the coordinate values of the second marker can be calculated using the coordinate values of the first marker.

[0066] Based on the above embodiments, optionally, after identifying that the first marker is fixed, the static coordinate values of the second marker relative to the first marker coordinate system are determined, including:

[0067] identifying the relative coordinate values of the second marker relative to the optical positioning system coordinate system after the first marker is fixed; and identifying a first rotation and translation matrix of the first marker coordinate system relative to the optical positioning system coordinate system;

[0068] determining the static coordinate values of the second marker relative to the first marker coordinate system according to the relative coordinate values and the first rotation and translation matrix.

[0069] In the present solution, the relative coordinate values of the second marker relative to the optical positioning system coordinate system can be that the coordinate values of the second marker are identified and calculated using the optical positioning system, specifically, the optical positioning system can use several infrared cameras to emit infrared light and perform covering shooting on the space, and the reflective marker balls in the second marker reflect the infrared light, the camera captures the infrared light reflected in the space, and an algorithm is used to calculate the relative position changes of these points in the space to obtain the relative coordinate values of the second marker relative to the optical positioning system coordinate system, which are denoted as . The first rotation and translation matrix can be a rotation and translation matrix of the first marker coordinate system to the optical positioning system coordinate system, specifically, it can be the coordinate values of the first marker in the tool file and the coordinate values of the first marker in the optical positioning instrument coordinate system in the current frame , calculate the distance between each pair of marker balls in the same coordinate system and sort them according to the distance, so that the coordinates of the marker balls in different coordinate systems correspond one by one , and then the first rotation and translation matrix is obtained by using the singular value decomposition algorithm, denoted as .

[0070] In the scheme, since the first rotation and translation matrix is the rotation and translation matrix from the first marker coordinate system to the optical positioning system coordinate system, the static coordinate value can be obtained by using the first rotation and translation matrix. Specifically, the optical positioning system can obtain the coordinate value of the second marker in the coordinate system of the first marker Ptool-2nd-in-1st by using the point coordinate transformation formula , according to the relative coordinate value of the second marker relative to the optical positioning system coordinate system and the first rotation and translation matrix. Since the above process is to calculate the coordinate value of the second marker in the coordinate system of the first marker, in order to ensure the reliability of the calculation result, the pose of the positioning target must be kept unchanged before the process is completed.

[0071] In the scheme, by calculating the first rotation and translation matrix of the first marker coordinate system and the optical positioning system coordinate system, the conversion relationship between the first marker coordinate system and the optical positioning system coordinate system can be obtained, and the static coordinate value of the second marker relative to the first marker coordinate system can be calculated according to the relative coordinate value of the second marker relative to the optical positioning system coordinate system and the first rotation and translation matrix. Since the first marker is fixed with the positioning target, after the second marker is removed or unloaded, the coordinate value of the second marker can be obtained by calculating the coordinate value of the first marker, which avoids the problems such as marker falling off and marker affecting the field of view of the execution personnel caused by the movement of the pose of the positioning target, and improves the reliability of the positioning guidance.

[0072] S103, identifying the pose change, calculating a second rotation and translation matrix of the first marker coordinate system and the optical positioning system coordinate system; and determining a theoretical coordinate value of the second marker relative to the optical positioning system coordinate system according to the static coordinate value;

[0073] In the scheme, the coordinate value of the first marker in its own coordinate system will not change, while the coordinate value of the first marker in the optical positioning system coordinate system will change after the pose of the positioning target moves. Therefore, the second rotation and translation matrix can be a new rotation and translation matrix from the first marker coordinate system to the optical positioning system coordinate system, and specifically, the coordinate value of the first marker in the tool file and the new coordinate value of the first marker in the optical positioning system coordinate system in the current frame , calculate the distance between each pair of marker balls in the same coordinate system and reorder according to the distance, so that the coordinate values of the marker balls in different coordinate systems correspond one by one , and then a second rotation and translation matrix can be obtained by using a singular value decomposition algorithm, which can be denoted as When the position of the positioning target changes, the marker balls may fall off or fail, etc. After the optical positioning system identifies the change in the position, the second rotation and translation matrix can be used to determine the theoretical coordinate value of the second marker relative to the coordinate system of the optical positioning system according to the static coordinate value of the second marker. Specifically, the coordinate value of the second marker in the coordinate system of the optical positioning system at the current position can be denoted as , and the point coordinate formula can be used to calculate the theoretical coordinate value.

[0074] S104, according to the theoretical coordinate value and the image coordinate value, determine the transformation relationship of the image coordinate system relative to the coordinate system of the optical positioning system, and use it for positioning guidance.

[0075] In the present solution, the transformation relationship of the image coordinate system relative to the coordinate system of the optical positioning system can be obtained by corresponding the theoretical coordinate value of the second marker relative to the coordinate system of the optical positioning system with the image coordinate value of the marker, and using a singular value algorithm to solve the two sets of coordinate values, to obtain the transformation relationship of the image coordinate system to the coordinate system of the optical positioning system, and use it for positioning guidance.

[0076] In the present solution, the positioning guidance can be positioning the positioning target before performing a task, and feeding back the position of the positioning target to the execution control end to guide the operation process of the task execution. Specifically, the image data of the positioning target in multiple modes can be three-dimensionally reconstructed and visually processed to obtain a three-dimensional model; a registration operation is performed by the optical positioning system to unify the three-dimensional model, the actual body position of the positioning target, and the real-time position of the execution tool in space in one coordinate system; the position of the execution tool in space is collected and displayed in real time by using the three-dimensional positioning system, and the execution personnel observe the relative position relationship between the execution tool and the target position in the three-dimensional model to perform the task operation on the positioning target.

[0077] Based on the above embodiments, according to the theoretical coordinate value and the image coordinate value, the transformation relationship of the image coordinate system relative to the coordinate system of the optical positioning system can be determined, including:

[0078] The theoretical coordinate value and the image coordinate value are matched, and a singular value decomposition algorithm is used to determine the transformation relationship of the image coordinate system relative to the coordinate system of the optical positioning system.

[0079] In the scheme, the theoretical coordinate value can be based on the pose of the current positioning target, the coordinate value of the second marker in the optical positioning system coordinate system, and specifically can be calculated using the second rotation translation matrix and the coordinate of the first marker under the optical system. The image coordinate value can be the coordinate value of the same marker in the image corresponding to the theoretical coordinate value, and specifically can be the coordinate value of the marker in the three-dimensional visualization model of the image. The positioning guide system corresponds the theoretical coordinate value and the image coordinate value, and determines the transformation relationship of the image coordinate system relative to the optical positioning system coordinate system using the singular value decomposition algorithm.

[0080] In the scheme, by matching the theoretical coordinate value of the second marker in the optical positioning system coordinate system after the pose change with the image coordinate value of the marker, and determining the transformation relationship of the image coordinate system relative to the optical positioning system coordinate system using the singular value decomposition algorithm, the purpose of real-time positioning of the second marker and feeding back the positioning result to the image can be achieved, and the accuracy of positioning guidance is improved.

[0081] The technical scheme provided by the embodiment comprises: acquiring the marker feature of the first marker; acquiring the image coordinate value of each reflective ball in the second marker relative to the image coordinate system after the second marker is fixed; identifying the static coordinate value of the second marker relative to the first marker coordinate system after the first marker is fixed; identifying the second rotation translation matrix of the first marker coordinate system and the optical positioning system coordinate system after the pose change; determining the theoretical coordinate value of the second marker relative to the optical positioning system coordinate system according to the static coordinate value; and determining the transformation relationship of the image coordinate system relative to the optical positioning system coordinate system according to the theoretical coordinate value and the image coordinate value, and performing positioning guidance based on the transformation relationship. Through the above positioning guidance method, the problem that the positioning accuracy is prone to deviation when the current positioning guidance is performed using the pasteable marker can be solved. By establishing the coordinate relationship between the first marker and the second marker, the coordinate value of the second marker can be calculated without considering whether the pose of the positioning target changes and whether the second marker falls off, and the accuracy and reliability of the positioning guidance are improved.

[0082] Embodiment Two

[0083] Figure 2 is a flowchart of a positioning guidance method provided by Embodiment Two of the present application. As shown in Figure 2 , the specific steps include the following steps:

[0084] S201, acquiring the marker feature of the first marker; and acquiring the image coordinate value of each reflective ball in the second marker relative to the image coordinate system after the second marker is fixed;

[0085] S202, determining a static coordinate value of the second marker relative to the first marker coordinate system after the first marker is fixed;

[0086] S203, identifying the pose change, and obtaining a second rotation and translation matrix of the first marker coordinate system relative to the optical positioning system coordinate system;

[0087] S204, reading the second rotation and translation matrix after the pose change is identified;

[0088] In the scheme, the positioning and guiding system can identify the second rotation and translation matrix after the pose change is identified. After the target pose changes, the coordinate value of the first marker in the optical positioning system coordinate system also changes. The second rotation and translation matrix can be a rotation and translation matrix calculated by the optical positioning system according to the new coordinate value of the first marker in the optical positioning system coordinate system and the coordinate value of the first marker in the first marker coordinate system.

[0089] S205, determining a theoretical coordinate value of the second marker relative to the optical positioning system coordinate system according to the static coordinate value and the second rotation and translation matrix;

[0090] In the scheme, the static coordinate value can be the coordinate value of the second marker in the first marker coordinate system. Specifically, the coordinate value can be calculated by the first rotation and translation matrix and the point coordinate transformation formula. The theoretical coordinate value of the second marker relative to the optical positioning system coordinate system can be calculated by the optical positioning system through calculating the new coordinate value of the first marker in the optical positioning system coordinate system, using the point coordinate calculation formula and the second rotation and translation matrix.

[0091] S206, determining the transformation relationship of the image coordinate system relative to the optical positioning system coordinate system according to the theoretical coordinate value and the image coordinate value, and performing positioning and guiding based on the transformation relationship.

[0092] The technical scheme provided by the embodiments of the present application can identify the pose change of the positioning target, calculate the second rotation and translation matrix of the first marker coordinate system relative to the optical positioning system coordinate system, read the second rotation and translation matrix after the pose change is identified, and determine the theoretical coordinate value of the second marker relative to the optical positioning system coordinate system according to the static coordinate value and the second rotation and translation matrix. Therefore, the theoretical coordinate value of the second marker relative to the optical positioning system coordinate system can be calculated without optical positioning of the second marker. The problems caused by deformation, displacement, and falling of the second marker and the influence on the task progress are avoided, the reliability of positioning and guiding is improved, and the success rate of task execution is improved.

[0093] Embodiment three

[0094] Figure 3 is a structural schematic diagram of a positioning guide device provided by Embodiment Three of the present application. As shown in the figure, it specifically comprises the following: Figure 3

[0095] The marker acquisition module 301 is configured to acquire a marker feature of a first marker, and acquire image coordinate values of each reflective ball in a second marker collected after the second marker is fixed relative to an image coordinate system;

[0096] The static coordinate value determination module 302 is configured to identify the second marker relative to a first marker coordinate system after the first marker is fixed, and determine a static coordinate value of the second marker;

[0097] The coordinate value updating module 303 is configured to determine a second rotation and translation matrix of the first marker coordinate system relative to an optical positioning system coordinate system, and determine a theoretical coordinate value of the second marker relative to the optical positioning system coordinate system according to the static coordinate value;

[0098] The positioning guide module 304 is configured to determine a transformation relationship of the image relative to the optical positioning system coordinate system according to the theoretical coordinate value and the image coordinate value, and perform positioning guide based on the transformation relationship.

[0099] Further, the static coordinate value determination module is specifically configured to:

[0100] identify a relative coordinate value of the second marker relative to the optical positioning system coordinate system after the first marker is fixed, and identify a first rotation and translation matrix of the first marker coordinate system relative to the optical positioning system coordinate system;

[0101] determine the static coordinate value of the second marker relative to the first marker coordinate system according to the relative coordinate value and the first rotation and translation matrix.

[0102] Further, the coordinate value updating module is specifically configured to:

[0103] read a second rotation and translation matrix after recognizing the pose change;

[0104] determine the theoretical coordinate value of the second marker relative to the optical positioning system coordinate system according to the static coordinate value and the second rotation and translation matrix.

[0105] Further, the positioning guide module is specifically configured to:

[0106] match the theoretical coordinate value and the image coordinate value, and determine the transformation relationship of the image relative to the optical positioning system coordinate system by using a singular value decomposition algorithm.

[0107] ​Further, the marker acquisition module is specifically used for:

[0108] The two-dimensional image original data of the positioning target is imported into a three-dimensional model reconstruction system to establish a three-dimensional visual digital model.

[0109] According to the image positions of the reflective balls in the second marker in the two-dimensional image original data, the position information of the reflective balls in the second marker in the three-dimensional visual digital model is determined.

[0110] According to the position information in the three-dimensional visual digital model, the image coordinate values of the reflective balls in the second marker relative to the image coordinate system are determined.

[0111] Further, the first marker is an "X"-shaped rigid tool fixed with the positioning target.

[0112] Further, the second marker is pasted on the surface of the positioning target.

[0113] The technical scheme provided in the embodiment includes a marker acquisition module, which is used to acquire the marker features of the first marker and the image coordinate values of the reflective balls in the second marker relative to the image coordinate system after the second marker is fixed; a static coordinate value determination module, which is used to determine the static coordinate values of the second marker relative to the first marker coordinate system after the first marker is fixed; a coordinate value updating module, which is used to determine the second rotation and translation matrix of the first marker coordinate system relative to the optical positioning system coordinate system and to determine the theoretical coordinate values of the second marker relative to the optical positioning system coordinate system according to the static coordinate values; and a positioning guide module, which is used to determine the transformation relationship of the image relative to the optical positioning system coordinate system according to the theoretical coordinate values and the image coordinate values and to perform positioning guidance. Through the above positioning guidance device, the problem that the positioning accuracy is prone to deviation when the pastable marker is used for positioning guidance can be solved, and the coordinate values of the second marker can be calculated without considering whether the pose of the positioning target changes and whether the second marker falls off, thereby improving the positioning accuracy and reliability.

[0114] The positioning guiding device in the embodiments of the present applicationapplicationbe a device, a component in a terminal, an integrated circuit, or a chip. The deviceapplicationbe a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic deviceapplicationbe a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc., and the non-mobile electronic deviceapplicationbe a server, a network attached storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc., and the embodiments of the present application do not make a specific limitation.

[0115] The positioning guiding device in the embodiments of the present applicationapplicationbe a device with an operating system. The operating systemapplicationbe an Android operating system, an ios operating system, or other possible operating systems, and the embodiments of the present application do not make a specific limitation.

[0116] The positioning guiding device provided in the embodiments of the present applicationapplicationbe capable of implementing the method embodiments. Figures 1 to 2 The processes implemented by the method embodimentsapplicationbe described above, and thus are not described herein again to avoid repetition.

[0117] Embodiment Four

[0118] As shown in Figure 4 , the embodiments of the present application further provide an electronic device 400, which includes a processor 401, a memory 402, and a program or instruction stored in the memory 402 and executable on the processor 401. The program or instruction is executed by the processor 401 to implement the processes of the above positioning guiding method embodiments and achieve the same technical effects. The processes are described above, and thus are not described herein again to avoid repetition.

[0119] It should be noted that the electronic device in the embodiments of the present applicationapplicationinclude the above-mentioned mobile electronic device and non-mobile electronic device.

[0120] Embodiment Five

[0121] The embodiments of the present application further provide a readable storage medium, which stores a program or instruction. The program or instruction is executed by a processor to implement the processes of the above positioning guiding method embodiments and achieve the same technical effects. The processes are described above, and thus are not described herein again to avoid repetition.

[0122] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0123] Embodiment six

[0124] The embodiment of the present application further provides a chip, which comprises a processor and a communication interface, the communication interface is coupled with the processor, the processor is used for running programs or instructions to realize the processes of the positioning guiding method embodiments and achieve the same technical effects. To avoid repetition, details are not described here.

[0125] It should be understood that the chip mentioned in the embodiment of the present application can also be referred to as a system-level chip, a system chip, a chip system, or a system-on-chip chip, etc.

[0126] It should be noted that in this document, the term "comprising" or "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such a process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of additional identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to the order of performing the functions as shown or discussed, but can also include performing the functions in a substantially simultaneous manner or in reverse order, for example, the described method can be performed in an order different from that described, and various steps can also be added, omitted or combined. In addition, the features described with reference to some examples can be combined in other examples.

[0127] From the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment method can be realized by means of software and necessary general hardware platform, of course, it can also be realized by hardware, but in many cases, the former is a better embodiment. Based on such understanding, the technical solutions of the present application can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a plurality of instructions for making a terminal (which can be a mobile phone, computer, server, or network equipment, etc.) execute the method described in each embodiment of the present application.

[0128] The embodiments of the present application are described above with reference to the drawings, but the present application is not limited to the specific embodiments described above, and the specific embodiments described above are merely illustrative, but not restrictive, and those of ordinary skill in the art can make many forms without departing from the purpose of the present application and the scope protected by the claims under the inspiration of the present application, which all belong to the protection of the present application.

[0129] The above are only the preferred embodiments of the present application and the technical principles used. The present application is not limited to the specific embodiments described herein, and various obvious changes, re-adjustments and replacements made by those skilled in the art will not deviate from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and more other equivalent embodiments can be included without departing from the concept of the present application, and the scope of the present application is determined by the scope of the claims.

Claims

1. A positioning guide method characterized by, The method is executed by a host computer, and the method comprises: Obtaining a marker feature of a first marker; and importing two-dimensional image raw data into a three-dimensional model reconstruction system to establish a three-dimensional visual digital model, determining position information of each reflective sphere in the second marker in the three-dimensional visual digital model according to image positions of each reflective sphere in the second marker in the two-dimensional image raw data, and determining image coordinate values of each reflective sphere in the second marker relative to an image coordinate system according to the position information in the three-dimensional visual digital model, the image coordinate system being a coordinate system of the three-dimensional visual digital model, the two-dimensional image raw data comprising image position data of a positioning target and image position data of each reflective sphere of the second marker, and the second marker being pasted on a surface of the positioning target; After recognizing that the first marker is fixed, determining a static coordinate value of the second marker relative to a first marker coordinate system, the static coordinate value being constant, and the second marker being capable of being recognized by an optical positioning system; After recognizing a pose change, calculating a second rotation and translation matrix of the first marker coordinate system and an optical positioning system coordinate system, and determining a theoretical coordinate value of the second marker relative to the optical positioning system coordinate system according to the static coordinate value; According to the theoretical coordinate value and the image coordinate value, determining a transformation relationship of the image coordinate system relative to the optical positioning system coordinate system, and performing positioning guidance based on the transformation relationship.

2. The method of claim 1, wherein, After recognizing that the first marker is fixed, determining a static coordinate value of the second marker relative to a first marker coordinate system, comprises: After recognizing that the first marker is fixed, determining a relative coordinate value of the second marker relative to an optical positioning system coordinate system, and recognizing a first rotation and translation matrix of the first marker coordinate system and the optical positioning system coordinate system; According to the relative coordinate value and the first rotation and translation matrix, determining the static coordinate value of the second marker relative to the first marker coordinate system.

3. The method of claim 1, wherein, According to the static coordinate value, determining a theoretical coordinate value of the second marker relative to the optical positioning system coordinate system, comprises: Reading a second rotation and translation matrix after recognizing a pose change; According to the static coordinate value and the second rotation and translation matrix, determining the theoretical coordinate value of the second marker relative to the optical positioning system coordinate system.

4. The method of claim 1, wherein, According to the theoretical coordinate value and the image coordinate value, determining a transformation relationship of the image coordinate system relative to the optical positioning system coordinate system, comprises: Matching the theoretical coordinate value and the image coordinate value, and determining the transformation relationship of the image coordinate system relative to the optical positioning system coordinate system by using a singular value decomposition algorithm.

5. The method according to any one of claims 1-4, characterized in that, The first marker is an "X"-shaped rigid tool fixed with the positioning target.

6. A positioning guide device, characterized in that The device is configured in a host computer, and the device comprises: The marker acquisition module is configured to acquire a marker feature of a first marker, and to import two-dimensional image raw data into a three-dimensional model reconstruction system to establish a three-dimensional visual digital model. Position information of each reflective sphere in the second marker in the three-dimensional visual digital model is determined according to image positions of the reflective spheres in the second marker in the two-dimensional image raw data. Image coordinate values of each reflective sphere in the second marker relative to an image coordinate system are determined according to the position information in the three-dimensional visual digital model. The image coordinate system is a coordinate system of the three-dimensional visual digital model. The two-dimensional image raw data includes image position data of a positioning target and image position data of each reflective sphere of the second marker. The second marker is attached to a surface of the positioning target. The static coordinate value determination module is configured to determine a static coordinate value of the second marker relative to a coordinate system of the first marker after the first marker is fixed. The static coordinate value remains unchanged. The second marker can be recognized by the optical positioning system. The coordinate value updating module is configured to calculate a second rotation and translation matrix of the coordinate system of the first marker and a coordinate system of the optical positioning system after a pose change, and to determine a theoretical coordinate value of the second marker relative to the coordinate system of the optical positioning system according to the static coordinate value. The positioning guidance module is configured to determine a transformation relationship of the image coordinate system relative to the coordinate system of the optical positioning system according to the theoretical coordinate value and the image coordinate value, and to perform positioning guidance based on the transformation relationship.

7. An electronic device, comprising: The computer program product includes a processor, a memory, and a program or instructions stored on the memory and executable on the processor. When the program or instructions are executed by the processor, the steps of the positioning guidance method according to any one of claims 1-5 are implemented.

8. A readable storage medium, characterized by, The readable storage medium stores a program or instructions. When the program or instructions are executed by the processor, the steps of the positioning guidance method according to any one of claims 1-5 are implemented.

Images