Machine vision navigation device and method for electrode implantation

Abstract

This invention provides a machine vision navigation device and method for electrode implantation. The device includes: a displacement mechanism; a first connector disposed on the displacement mechanism; a first imaging mechanism connected to the first connector, the optical axis of the first imaging mechanism being perpendicular to the implantation area; a second imaging mechanism connected to the first connector and located on one side of the first imaging mechanism; and a third imaging mechanism connected to the first connector and located on the other side of the first imaging mechanism. By integrating the first, second, and third imaging mechanisms, omnidirectional visual coverage from global to local is achieved. The first imaging mechanism performs global observation of the implantation area. The second and third imaging mechanisms form a binocular vision system, performing local three-dimensional observation of the vicinity of the target implantation point, accurately capturing changes near the implantation point in real time, thereby precisely guiding the electrode tip to the implantation point surface.

Description

Technical Field

[0001] This invention relates to the field of brain-computer interface technology, and in particular to a machine vision navigation device and method for electrode implantation. Background Technology

[0002] Electrode implantation refers to the placement of neural electrodes into the cerebral cortex, and is an important technology in the fields of brain-computer interfaces and neuroscience. In order to ensure that the electrodes enter the designated brain region and avoid risky tissues such as blood vessels, navigation technology is required to precisely guide the electrode tip to the implantation point on the cortical surface, and then insert it into the cortex from the implantation point.

[0003] Machine vision navigation methods offer advantages such as non-contact operation, real-time performance, low cost, and high accuracy. For navigation tasks involving electrode implantation, the surface of a designated area of ​​the cerebral cortex is observed before implantation begins, and local observations are also conducted near the implantation point during the implantation process.

[0004] Current navigation devices struggle to simultaneously achieve global observation before implantation and local observation during implantation, or require a large number of cameras and complex mechanical structures. Therefore, there is an urgent need for a novel machine vision navigation device and corresponding method for electrode implantation, capable of precisely guiding the electrode tip to the target implantation point on the surface of the cerebral cortex. Summary of the Invention

[0005] This invention provides a machine vision navigation device and method for electrode implantation, which solves the problem that existing navigation devices are difficult to achieve both global observation before implantation and local observation during implantation, or require a large number of cameras and complex mechanical structures.

[0006] This invention provides a machine vision navigation device for electrode implantation, comprising:

[0007] Displacement mechanism;

[0008] A first connector is disposed on the displacement mechanism, the displacement mechanism being used to adjust the distance of the first connector relative to the implantation area;

[0009] A first imaging mechanism is connected to the first connector, and the optical axis of the first imaging mechanism is perpendicular to the implantation area for observing the implantation area;

[0010] The second shooting mechanism is connected to the first connector and located on one side of the first shooting mechanism. The optical axis of the second shooting mechanism is tilted to one side of the optical axis of the first shooting mechanism.

[0011] A third shooting mechanism is connected to the first connector and located on the other side of the first shooting mechanism. The optical axis of the third shooting mechanism is tilted to the other side of the optical axis of the first shooting mechanism.

[0012] The optical axis of the second imaging mechanism approximately intersects with the optical axis of the third imaging mechanism to form binocular vision for the electrodes.

[0013] According to the present invention, a machine vision navigation device for electrode implantation includes a first imaging mechanism comprising a first camera and a first lens; the first camera is disposed in the middle of the first connector, and the first lens is disposed at the imaging end of the first camera; the first camera forms a first rectangular field of view through the first lens.

[0014] The second shooting mechanism includes: a second camera and a second lens; the second camera is disposed on the left side of the first connector, and the second lens is disposed at the shooting end of the second camera; the second camera forms a second rectangular field of view through the second lens;

[0015] The third shooting mechanism includes a third camera and a third lens; the third camera is located on the right side of the first connector, and the third lens is located at the shooting end of the third camera; the third camera forms a third rectangular field of view through the third lens.

[0016] According to the present invention, a machine vision navigation device for electrode implantation is provided, wherein the size of the first rectangular field of view is larger than the size of the second rectangular field of view and the third rectangular field of view.

[0017] A machine vision navigation device for electrode implantation according to the present invention further includes:

[0018] A first motion mechanism and a second connector, wherein the first motion mechanism is connected to the displacement mechanism via the second connector to drive the displacement mechanism to perform three-dimensional motion relative to the implantation area.

[0019] A machine vision navigation device for electrode implantation according to the present invention further includes:

[0020] A second motion mechanism is used to connect the electrode to drive the electrode to perform three-dimensional motion relative to the implantation area;

[0021] The controller is electrically connected to the first motion mechanism, the second motion mechanism, the displacement mechanism, the first imaging mechanism, the second imaging mechanism, and the third imaging mechanism to control the first motion mechanism, the second motion mechanism, and the displacement mechanism based on the images of the implantation area acquired by the first imaging mechanism, the second imaging mechanism, and the third imaging mechanism.

[0022] According to the present invention, a machine vision navigation device for electrode implantation is provided, wherein the first motion mechanism is a multi-axis robotic arm, the second motion mechanism is a three-axis motion table, and the displacement mechanism is a linear displacement slide.

[0023] The present invention also provides a method for using the aforementioned machine vision navigation device for electrode implantation, comprising:

[0024] Based on the position of maximum focus and the two-dimensional coordinates in the coordinate system of the first imaging mechanism, establish the three-dimensional coordinates of the implantation point in the navigation coordinate system;

[0025] Based on the three-dimensional coordinates of the implantation point in the navigation coordinate system and the transformation relationship from the navigation coordinate system to the motion coordinate system, the position of the displacement mechanism is adjusted by the first motion mechanism so that the implantation point is located in the field of view of the second and third imaging mechanisms.

[0026] Using the binocular vision provided by the second and third imaging mechanisms, the second motion mechanism is controlled to drive the electrode tip to approach the implantation point.

[0027] According to the method provided by the present invention, before the step of establishing the three-dimensional coordinates of the implantation point based on the position of maximum focus sharpness and the two-dimensional coordinates in the coordinate system of the first imaging mechanism, the method further includes:

[0028] The displacement mechanism is moved to the first position, and the position of the displacement mechanism is adjusted by the first motion mechanism to place the implantation area within the field of view and optical depth of field of the first imaging mechanism.

[0029] The first image is acquired using the first imaging device, and the implantation point is selected in the first image;

[0030] The control displacement mechanism moves to a second position and then moves from the second position to a third position, acquiring multiple second images during the movement from the second position to the third position; wherein, the first position is located between the second position and the third position;

[0031] Calculate the focus sharpness of the implantation point in multiple second images and determine the location with the maximum focus sharpness.

[0032] According to the method provided by the present invention, the step of obtaining two-dimensional coordinates in the coordinate system of the first imaging mechanism includes:

[0033] Based on the two-dimensional coordinates of the implantation point in the first image and the calibration parameters of the first imaging mechanism, the two-dimensional coordinates in the coordinate system of the first imaging mechanism are determined.

[0034] The machine vision navigation device and method for electrode implantation provided by this invention achieves omnidirectional visual coverage from global to local perspectives by integrating a first imaging mechanism, a second imaging mechanism, and a third imaging mechanism. The optical axis of the first imaging mechanism is perpendicular to the implantation area, facilitating the planning and selection of multiple implantation points. The second and third imaging mechanisms form a binocular vision system, performing local three-dimensional observation of the vicinity of the target implantation point, accurately capturing changes in the vicinity of the implantation point in real time, thereby precisely guiding the electrode tip to the implantation point surface. This design not only improves the accuracy and safety of implantation but also significantly enhances the efficiency and success rate of the surgery. Simultaneously, the entire machine vision navigation device uses only the first, second, and third imaging mechanisms, avoiding complex motion mechanisms and expensive sensors, thus reducing the cost of the electrode implantation system and improving its long-term reliability and stability. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0036] Figure 1 This is a schematic diagram of the machine vision navigation device provided in an embodiment of the present invention.

[0037] Figure 2 This is a schematic diagram provided by an embodiment of the present invention, showing how the implanted area is placed within the field of view and optical depth of the first imaging mechanism by adjusting the three-dimensional pose of the machine vision navigation device.

[0038] Figure 3 This is a flowchart illustrating the method using a machine vision navigation device provided in an embodiment of the present invention.

[0039] Figure 4 This is a schematic diagram of the first image provided in the embodiment of the invention.

[0040] Figure 5 This is a schematic diagram provided by an embodiment of the invention, showing how the implantation point is placed within the field of view of the second and third imaging mechanisms by adjusting the three-dimensional pose of the machine vision navigation device.

[0041] Figure 6 This is a schematic diagram of the three-dimensional alignment movement of the control electrode tip provided in the embodiment of the invention.

[0042] Figure label:

[0043] 100. First shooting mechanism; 110. First camera; 120. First lens; 130. First rectangular field of view; 200. Second shooting mechanism; 210. Second camera; 220. Second lens; 230. Second rectangular field of view; 300. Third shooting mechanism; 310. Third camera; 320. Third lens; 330. Third rectangular field of view; 400. Displacement mechanism; 500. Implantation area; 510. Implantation point; 520. First image; 600. First connector; 700. Electrode; 800. Second connector. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0045] The following is combined Figures 1-6 This invention describes the machine vision navigation device and method for electrode implantation provided by the present invention.

[0046] This invention provides a machine vision navigation device for electrode implantation, such as... Figure 1 and Figure 2 As shown, the machine vision navigation device includes a displacement mechanism 400, a first connector 600, a first imaging mechanism 100, a second imaging mechanism 200, and a third imaging mechanism 300. A first connector 600 is disposed on a displacement mechanism 400, which is used to adjust the distance of the first connector 600 relative to the implantation area 500. A first imaging mechanism 100 is connected to the first connector 600, and the optical axis of the first imaging mechanism 100 is perpendicular to the implantation area 500 for observing the implantation area 500. A second imaging mechanism 200 is connected to the first connector 600 and is located on one side of the first imaging mechanism 100, and the optical axis of the second imaging mechanism 200 is inclined to one side of the optical axis of the first imaging mechanism 100. A third imaging mechanism 300 is connected to the first connector 600 and is located on the other side of the first imaging mechanism 100, and the optical axis of the third imaging mechanism 300 is inclined to the other side of the optical axis of the first imaging mechanism 100. The optical axes of the second imaging mechanism 200 and the third imaging mechanism 300 approximately intersect to form binocular vision targeting the tip of the electrode 700 or the vicinity of the implantation point.

[0047] In this embodiment, the displacement mechanism 400 is the basic moving part of the device, used to precisely adjust the position and distance of the first connector 600 (and all imaging mechanisms connected to it) relative to the implantation area 500 (i.e., the surgical target area). By adjusting the displacement mechanism 400, it can be ensured that the imaging mechanism is in the optimal observation position, thereby obtaining the clearest and most accurate image.

[0048] The first imaging mechanism 100 is connected to the first connector 600, and its optical axis is perpendicular to the implantation area 500, providing a frontal view of the implantation area 500 for easy observation. The second imaging mechanism 200 is located to one side of the first imaging mechanism 100, and its optical axis is tilted to the optical axis of the first imaging mechanism 100. The third imaging mechanism 300 is located to the other side of the first imaging mechanism 100, and its optical axis is tilted in the opposite direction to that of the second imaging mechanism 200. The second imaging mechanism 200 and the third imaging mechanism 300 together form a binocular vision, providing a binocular view of the implanted electrode 700 for measuring the three-dimensional relative position of the electrode 700 and the implantation point.

[0049] The machine vision navigation device for electrode implantation provided in this embodiment achieves omnidirectional visual coverage from global to local perspectives by integrating a first imaging mechanism 100, a second imaging mechanism 200, and a third imaging mechanism 300. The optical axis of the first imaging mechanism 100 is perpendicular to the implantation area 500, facilitating the planning and selection of multiple implantation points 510. The second imaging mechanism 200 and the third imaging mechanism 300 form a binocular vision system, performing local three-dimensional observation of the vicinity of the target implantation point 510, accurately capturing changes in the vicinity of the implantation point 510 in real time, thereby precisely guiding the tip of the electrode 700 to the surface of the implantation point 510. This design not only improves the accuracy and safety of implantation but also significantly enhances the efficiency and success rate of the surgery. Simultaneously, the entire machine vision navigation device uses only the first imaging mechanism 100, the second imaging mechanism 200, and the third imaging mechanism 300, avoiding complex motion mechanisms and expensive sensors, thus reducing the cost of the electrode 700 implantation system and improving the long-term reliability and stability of the system.

[0050] In this embodiment, as Figure 1 and Figure 2As shown, the first shooting mechanism 100 includes: a first camera 110 and a first lens 120; the first camera 110 is disposed in the middle of the first connector 600, and the first lens 120 is disposed at the shooting end of the first camera 110; the first camera 110 forms a first rectangular field of view 130 through the first lens 120; the second shooting mechanism 200 includes: a second camera 210 and a second lens 220; the second camera 210 is disposed on the left side of the first connector 600, and the second lens 220 is disposed at the shooting end of the second camera 210; the second camera 210 forms a second rectangular field of view 230 through the second lens 220; the third shooting mechanism 300 includes: a third camera 310 and a third lens 320; the third camera 310 is disposed on the right side of the first connector 600, and the third lens 320 is disposed at the shooting end of the third camera 310; the third camera 310 forms a third rectangular field of view 330 through the third lens 320.

[0051] Specifically, the first imaging mechanism 100 includes a first camera 110 and a first lens 120. The first lens 120 forms a first rectangular field of view 130, the size of which is 14×14 mm, determined by the camera's CCD physical size (7×7 mm), pixel resolution (2048×2048), and the lens's optical depth of field (1 mm) and magnification (0.5x). The shallow optical depth of field of the first lens 120 makes the image sharpness near the focal plane highly sensitive to depth changes, facilitating accurate measurement of the depth of the implantation point in the coordinate system of the first imaging mechanism.

[0052] The second shooting mechanism 200 includes a second camera 210 and a second lens 220. The second lens 220 forms a second rectangular field of view 230, the size of which is 3.5×3.5 mm, determined by the physical size of the camera's CCD (7×7 mm), pixel resolution (2048×2048), and the optical depth of field (0.5 mm) and magnification (2x) of the lens.

[0053] The third imaging mechanism 300 includes a third camera 310 and a third lens 320. The third lens 320 forms a third rectangular field of view 330, which is also 3.5 × 3.5 mm in size and configured identically to the second imaging mechanism 200. The third imaging mechanism 300 works in conjunction with the second imaging mechanism 200 to provide a stereoscopic view of the implantation area 500, enhancing accuracy and safety.

[0054] In this embodiment, the first rectangular field of view 130 is the largest (14×14 mm) and is used to provide a comprehensive overview of the implantation area 500. The second rectangular field of view 230 and the third rectangular field of view 330 are smaller (3.5×3.5 mm) but have higher magnification and are used for detailed observation of the end of the electrode 700 and the local implantation area 500.

[0055] Based on the above embodiments, in some embodiments, such as Figure 1 and Figure 2 As shown, the machine vision navigation device further includes a first motion mechanism and a second connector 800. The first motion mechanism is connected to the displacement mechanism 400 via the second connector 800 to drive the displacement mechanism 400 to perform three-dimensional motion relative to the implantation area 500.

[0056] In addition, the machine vision navigation device also includes a second motion mechanism and a controller. The second motion mechanism is used to connect to the electrode 700 to drive the electrode 700 to perform three-dimensional motion relative to the implantation area 500; the controller is electrically connected to the first motion mechanism, the second motion mechanism, the displacement mechanism 400, the first imaging mechanism 100, the second imaging mechanism 200, and the third imaging mechanism 300 to control the first motion mechanism, the second motion mechanism, and the displacement mechanism 400 based on the images of the implantation area 500 acquired by the first imaging mechanism 100, the second imaging mechanism 200, and the third imaging mechanism 300.

[0057] In this embodiment, the first motion mechanism is connected to the displacement mechanism 400 via the second connector 800, providing three-dimensional motion capability for the entire machine vision navigation device. This allows the position and angle of the imaging mechanism and the electrode 700 to be adjusted as needed during the implantation process, so as to better observe the implantation area 500 and precisely control the implantation of the electrode 700.

[0058] Preferably, the first motion mechanism is a multi-axis robotic arm with six degrees of freedom (three translational degrees of freedom and three rotational degrees of freedom), which can provide highly flexible and precise motion control. The second connector 800 serves to connect and support, ensuring a stable connection between the first motion mechanism and the displacement mechanism 400.

[0059] The second motion mechanism is used to connect the electrode 700 and drive the electrode 700 to move in three dimensions relative to the implantation area 500. This helps to precisely control the implantation position and depth of the electrode 700 during the implantation process.

[0060] Preferably, the second motion mechanism is a high-precision three-axis motion stage with three translational degrees of freedom, providing high-precision motion control. It is electrically connected to the first motion mechanism, the second motion mechanism, the displacement mechanism 400, and the three imaging mechanisms (first imaging mechanism 100, second imaging mechanism 200, and third imaging mechanism 300). The controller calculates and controls the motion of each motion mechanism in real time based on the images of the implantation area 500 acquired by the three imaging mechanisms to achieve precise implantation of the electrode 700.

[0061] The first camera 110, the second camera 210, and the third camera 310 are all fixedly mounted on the first connector 600, and their relative positions remain unchanged after installation. This ensures that the geometric relationship of the imaging mechanism remains unchanged during the surgical procedure. The first connector 600 is fixedly mounted on the displacement mechanism 400, and the base of the displacement mechanism 400 is fixedly mounted on the second connector 800. The second connector 800 is then fixedly mounted on the end of the first motion mechanism.

[0062] The tip of electrode 700 is fixedly connected to the second motion mechanism. If it is a rigid electrode, it is directly connected to the second motion mechanism. If it is a flexible electrode, the tip of the flexible electrode is temporarily connected to the tip of the auxiliary implantation needle, and the tip of the flexible electrode is connected to the second motion mechanism through the auxiliary implantation needle.

[0063] A first navigation coordinate system is established on the base of the linear displacement slide to describe the position of the entire machine vision navigation device in space. A first motion coordinate system describes the motion state of the first motion mechanism (i.e., the multi-axis robotic arm). The transformation relationship T1 from the first navigation coordinate system to the first motion coordinate system can be obtained through calibration and the real-time state of the first motion mechanism. The first motion mechanism has at least three translational degrees of freedom and can drive the displacement mechanism 400, the first connecting member 600, and the imaging mechanism as a whole to perform three-dimensional motion relative to the implantation area 500. The second motion mechanism also has at least three translational degrees of freedom and can drive the tip of the electrode 700 to perform high-precision three-dimensional motion.

[0064] This invention also provides a method for using the machine vision navigation device for electrode implantation described above, such as... Figure 3 As shown, the method includes the following steps:

[0065] Step S350: Establish the three-dimensional coordinates of the implantation point based on the position of maximum focus clarity and the two-dimensional coordinates in the coordinate system of the first imaging mechanism.

[0066] Step S360: Based on the three-dimensional coordinates of the implantation point in the navigation coordinate system and the transformation relationship from the navigation coordinate system to the motion coordinate system, the position of the displacement mechanism is adjusted by the first motion mechanism so that the implantation point is located in the field of view of the second and third imaging mechanisms.

[0067] Step S370: Using the binocular vision provided by the second and third imaging mechanisms, control the second motion mechanism to drive the electrode tip to approach the implantation point.

[0068] Specifically, during the process, the first imaging mechanism 100 (including the first camera 110 and the first lens 120) is used to capture images of the implantation area 500. The location with the highest focus sharpness is determined by an image processing algorithm (such as a focus evaluation function).

[0069] By combining the two-dimensional coordinates of the first shooting mechanism 100 and the camera's internal and external parameters (such as focal length, magnification, pixel equivalent, optical center position, etc.), the three-dimensional coordinates of the implantation point 510 in the navigation coordinate system are established.

[0070] Based on the three-dimensional coordinates of the implantation point in the navigation coordinate system and the transformation relationship from the navigation coordinate system to the motion coordinate system, the position of the machine vision navigation device is adjusted by the first motion mechanism (such as a multi-axis robotic arm) so that the implantation point 510 is located in the common field of view of the second imaging mechanism 200 and the third imaging mechanism 300, so as to perform binocular visual observation and electrode 700 implantation control.

[0071] Using binocular vision images provided by the second imaging mechanism 200 and the third imaging mechanism 300, the three-dimensional coordinate information of the implantation point 510 is obtained through a stereo matching algorithm. Based on the obtained three-dimensional coordinate information, the second motion mechanism (such as a high-precision three-axis motion stage) is controlled to drive the electrode 700 to perform three-dimensional motion, so that the tip of the electrode 700 accurately reaches the implantation point 510. During the implantation of the electrode 700, binocular vision is continuously used for monitoring and feedback control to ensure the accuracy and safety of the electrode 700 implantation.

[0072] This method utilizes a machine vision navigation device to achieve precise control of electrode 700 implantation. The three-dimensional coordinates of the implantation point 510 are determined by the first imaging mechanism 100, and then the position of the displacement mechanism 400 is adjusted to place the implantation point 510 within the field of view of the binocular vision system. The stereoscopic view provided by the binocular vision system is then used to control the implantation of the electrode 700. This method not only improves the accuracy and safety of electrode 700 implantation but also reduces implantation risks and operational difficulty.

[0073] In some embodiments, such as Figure 3 As shown, before the step of establishing the three-dimensional coordinates of the implantation point based on the position of maximum focus and the two-dimensional coordinates in the coordinate system of the first imaging mechanism, the following steps are also included:

[0074] Step S310: Control the displacement mechanism to move to the first position, and adjust the position of the displacement mechanism through the first motion mechanism to place the implantation area within the field of view and optical depth range of the first imaging mechanism.

[0075] Step S320: Acquire a first image using the first imaging mechanism, and select an implantation point in the first image.

[0076] Step S330: Control the displacement mechanism to move to the second position, and then move from the second position to the third position to acquire multiple second images during the movement from the second position to the third position; wherein, the first position is located between the second position and the third position.

[0077] Step S340: Calculate the focus sharpness of the implantation point in multiple second images and determine the location with the maximum focus sharpness.

[0078] In this embodiment, the displacement mechanism 400 (such as a linear displacement slide) is moved to an initial position, namely the first position. The position of the displacement mechanism 400 is further adjusted by the first motion mechanism (such as a multi-axis robotic arm) to ensure that the implantation area 500 is completely within the field of view and optical depth range of the first imaging mechanism 100 (such as a camera and lens combination).

[0079] An image of the implantation region 500 is captured using a first imaging device 100, resulting in a first image 520. On the first image 520, an implantation point 510 is selected via a human-computer interaction interface or an automated algorithm. The implantation point 510 is typically determined based on implantation planning and anatomical structures in the image.

[0080] The control displacement mechanism 400 moves from the second position to the third position. Note that the first position should be located between the second and third positions, but not necessarily at their midpoint.

[0081] During the movement of the displacement mechanism 400, multiple images are continuously captured by the first imaging mechanism 100, forming a second image sequence. These images reflect the changes in focus sharpness as the position of the displacement mechanism 400 changes.

[0082] Multiple acquired second images are processed to calculate the focus sharpness evaluation index (such as grayscale variance, gradient magnitude, etc.) for each image. The focus sharpness evaluation indexes of different images are compared to determine the image with the highest focus sharpness and its corresponding displacement mechanism position 400.

[0083] After determining the position with the greatest focus sharpness, steps S350 to S370 can be performed.

[0084] In some embodiments, the step of obtaining the two-dimensional coordinates in the coordinate system of the first imaging mechanism includes: determining the two-dimensional coordinates in the coordinate system of the first imaging mechanism based on the two-dimensional coordinates of the implantation point in the first image and the calibration parameters of the first imaging mechanism.

[0085] In this embodiment, the implantation point 510 is selected on the image acquired by the first imaging mechanism 100 through a human-computer interaction interface or an automatic algorithm. Its position on the image is the two-dimensional coordinate (image coordinate) of the implantation point 510.

[0086] Before image capture and coordinate calculation, the first imaging mechanism 100 (camera and lens combination) needs to be calibrated. After the camera is calibrated, the two-dimensional coordinates of the first imaging mechanism 100 can be determined by using the camera's intrinsic and extrinsic parameters and the image coordinates of the implantation point 510 through methods such as inverse projection transformation.

[0087] In one specific embodiment, such as Figure 1 , Figure 2 , Figure 4 , Figure 5 and Figure 6 As shown, the moving part of the linear displacement slide reaches the first position, which is close to the middle of the movement stroke. Preferably, the first position is 7.5 cm.

[0088] like Figure 2 As shown, by using the first motion mechanism, the three-dimensional pose of the machine vision navigation device is adjusted to T2, so that the optical axis of the first lens 120 is approximately perpendicular to the implantation area 500, and the implantation area 500 is within the optical depth of field of the first lens 120, so that the implantation area 500 can be clearly observed and imaged by the first camera 110.

[0089] like Figure 4 As shown, a first image 520 containing the implantation region 500 is acquired using a first camera 110. Based on the specific requirements for electrode 700 implantation, N implantation points 510 are selected on the first image 520 using either automatic or manual selection. The two-dimensional coordinates of the i-th implantation point 510 are I... i (i=1,2,…,N).

[0090] The moving part of the linear displacement slide moves to the second position, and then begins to move to the third position in a stepping or continuous mode. During the movement of the moving part of the linear displacement slide from the second position to the third position, the first camera 110 continuously captures the latest image in real time, sequentially at j observation positions {z}. m}(j=1,2,…,M) Calculate the focus sharpness value {f} near N implantation points 510 in the latest image. i,j}(i=1,2,…,N,j=1,2,…,M).

[0091] The first position is located between the second and third positions, and the distance from the third position to the second position is greater than the optical depth of field range of the first lens 120.

[0092] For the i-th implantation point 510, the focus sharpness value fi,j of the i-th implantation point 510 shows a trend of first increasing and then decreasing as j increases. Calculate the linear displacement slide position z that maximizes the focus sharpness of the i-th implantation point 510. max,i .

[0093] Based on the two-dimensional coordinates of the i-th implantation point 510 in the image space and the calibrated intrinsic parameters of the first camera 110, calculate the XY coordinates (x, y, y) of the i-th implantation point 510 in the coordinate system of the first camera 110. i ,y i ).

[0094] According to the above (x) i ,y i ) and z max,i The three-dimensional coordinates P of the i-th implantation point 510 in the first navigation coordinate system are calculated through the calibrated rigid body transformation. i Finally, the three-dimensional coordinates of N implantation points 510 in the first navigation coordinate system are obtained {P}. i}

[0095] like Figure 5 As shown, in order to implant the electrode 700 into the i-th implantation point 510, the three-dimensional coordinates P of the i-th implantation point 510 in the first navigation coordinate system are first determined. i The transformation relationship T1 from the first navigation coordinate system to the first motion coordinate system is used to adjust the three-dimensional pose of the machine vision navigation device to T using the first motion mechanism. i This places the i-th implantation point 510 near the common field of view center of the second camera 210 and the third camera 310.

[0096] Finally, as Figure 6 As shown, the tip of electrode 700 is driven into the common field of view of the second camera 210 and the third camera 310 by the second motion mechanism. Using the binocular vision measurement results provided by the second camera 210 and the third camera 310, the tip of electrode 700 is controlled to perform three-dimensional alignment movement to accurately reach the i-th implantation point 510 of the implantation area 500.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for controlling a machine vision navigation device for electrode implantation, characterized in that, The machine vision navigation device for electrode implantation includes: First motion mechanism and displacement mechanism; A first connector is disposed on the displacement mechanism, the displacement mechanism being used to adjust the distance of the first connector relative to the implantation area; A first imaging mechanism is connected to the first connector, and the optical axis of the first imaging mechanism is perpendicular to the implantation area for observing the implantation area; The second shooting mechanism is connected to the first connector and located on one side of the first shooting mechanism. The optical axis of the second shooting mechanism is tilted to one side of the optical axis of the first shooting mechanism. A third shooting mechanism is connected to the first connector and located on the other side of the first shooting mechanism. The optical axis of the third shooting mechanism is tilted to the other side of the optical axis of the first shooting mechanism. Wherein, the optical axis of the second imaging mechanism and the optical axis of the third imaging mechanism approximately intersect to form binocular vision for the electrode tip or implantation point; The method includes: The displacement mechanism is controlled to move to a first position, and the position of the displacement mechanism is adjusted by the first motion mechanism to place the implantation area within the field of view and optical depth range of the first imaging mechanism; The first image is acquired using the first imaging mechanism, and the implantation point is selected in the first image; The displacement mechanism is controlled to move to a second position and then to a third position, acquiring multiple second images during the movement from the second position to the third position; wherein the first position is located between the second and third positions; Calculate the focus sharpness of the implantation point in multiple second images and determine the location of maximum focus sharpness; Based on the position of maximum focus and the two-dimensional coordinates in the coordinate system of the first imaging mechanism, establish the three-dimensional coordinates of the implantation point in the navigation coordinate system; Based on the three-dimensional coordinates of the implantation point in the navigation coordinate system and the transformation relationship from the navigation coordinate system to the motion coordinate system, the position of the displacement mechanism is adjusted by the first motion mechanism so that the implantation point is located in the field of view of the second and third imaging mechanisms. Using the binocular vision provided by the second and third imaging mechanisms, the second motion mechanism is controlled to drive the electrode tip to approach the implantation point.

2. The method according to claim 1, characterized in that, The step of obtaining the two-dimensional coordinates in the coordinate system of the first imaging mechanism includes: The two-dimensional coordinates of the implantation point in the first image and the calibration parameters of the first imaging mechanism are used to determine the two-dimensional coordinates in the coordinate system of the first imaging mechanism.

3. The method according to claim 1, characterized in that, The first shooting mechanism includes: a first camera and a first lens; the first camera is disposed in the middle of the first connector, and the first lens is disposed at the shooting end of the first camera; the first camera forms a first rectangular field of view through the first lens; The second shooting mechanism includes: a second camera and a second lens; the second camera is disposed on the left side of the first connector, and the second lens is disposed at the shooting end of the second camera; the second camera forms a second rectangular field of view through the second lens; The third shooting mechanism includes a third camera and a third lens; the third camera is located on the right side of the first connector, and the third lens is located at the shooting end of the third camera; the third camera forms a third rectangular field of view through the third lens.

4. The method according to claim 3, characterized in that, The size of the first rectangular field of view is larger than the size of the second rectangular field of view and the third rectangular field of view.

5. The method according to claim 1, characterized in that, The machine vision navigation device for electrode implantation also includes: The second connector connects the first motion mechanism to the displacement mechanism to drive the displacement mechanism to perform three-dimensional motion relative to the implantation area.

6. The method according to claim 5, characterized in that, The machine vision navigation device for electrode implantation also includes: A second motion mechanism is used to connect the electrode to drive the electrode to perform three-dimensional motion relative to the implantation area; The controller is electrically connected to the first motion mechanism, the second motion mechanism, the displacement mechanism, the first imaging mechanism, the second imaging mechanism, and the third imaging mechanism to control the first motion mechanism, the second motion mechanism, and the displacement mechanism based on the images of the implantation area acquired by the first imaging mechanism, the second imaging mechanism, and the third imaging mechanism.

7. The method according to claim 6, characterized in that, The first motion mechanism is a multi-axis robotic arm, the second motion mechanism is a three-axis motion table, and the displacement mechanism is a linear displacement slide.

Images