Surgical exterior mirror system with motion focus following function

Abstract

The invention discloses a surgical exterior mirror system with a motion focus following function. The surgical exterior mirror system comprises a box body, a mechanical arm, a camera system, a focusing controller and a display screen, the mechanical arms are arranged on the box body and comprise a large mechanical arm, a first small mechanical arm, a second small mechanical arm and a camera branch arm which are connected in sequence, and the mechanical arms are connected through mechanical arm joints; the camera system is used for acquiring image information of a surgical field; the focusing controller is used for controlling the movement of an optical module in the camera system; and the display screen displays the maximum-definition image obtained by the camera system. According to the invention, more convenient operation is provided for doctors. In an operation, a doctor obtains deeper information of an operation area through movement of the exterior mirror, and in the movement process, the exterior mirror tracks an operation object area, so that the view can be ensured not to be lost. In the process of tracking the operation area, when the position of the exterior mirror is changed, the focusing position can be quickly and automatically adjusted, the image definition of the exterior mirror is ensured, and the efficiency of obtaining the operation area image by the exterior mirror is improved.

Description

Technical Field

[0001] The invention belongs to the technical field of medical devices and relates to a surgical exoscopic system with a motion follow-focus function. Background Art

[0002] Visual instruments are essential to ensure high-quality surgical results in minimally invasive surgery. In recent years, exoscopes, as an important auxiliary means in modern surgical operations, have been increasingly used in neurosurgery. Compared with traditional surgical microscopes, exoscopes have advantages such as comfortable operation, deep depth of field, wide field of view, and long focal length. In addition, as the resolution of the display screen of the exoscope continues to improve, the ability to accurately identify fine vascular and neuroanatomical structures can be further improved, which will further help improve the safety of surgery.

[0003] Exoscopic technology is a technological innovation in surgery that can magnify and image the surgical area during surgery. During the operation, the surgical area contains complex structures such as blood vessels and nerves, and a single perspective cannot obtain in-depth details. When doctors adjust the exoscopic scope, they are likely to lose sight of the target area, and the clarity of the image obtained cannot be guaranteed.

[0004] Therefore, there is an urgent need for a surgical exoscopic system with a motion focus function to address the deficiencies in the existing technology and provide more stable imaging quality to meet the requirements of modern surgery. Summary of the invention

[0005] The existing exoscopic system needs to acquire images of the surgical area for a long time during operation to achieve high-resolution long-term imaging. However, the blood vessels and nerves in the surgical area are intricate. The image presented from a single angle has a blind spot. When the doctor adjusts the position of the exoscopic system, it is easy to lose the image of the key parts of the surgical area, and the clarity of the final image cannot be verified. The process of manually adjusting the optical path to obtain a clear image takes a lot of time, affecting the efficiency of the operation. In order to solve the above problems, the present invention discloses a surgical exoscopic system with a motion focus function.

[0006] The specific plan is as follows:

[0007] A surgical exoscopic system with a motion focus function comprises a housing, a mechanical arm, a camera system, a focus controller and a display screen; the mechanical arm is arranged on the housing, and comprises a mechanical upper arm, a first mechanical lower arm, a first mechanical lower arm and a camera arm connected in sequence, and each mechanical arm is connected via a mechanical arm joint; the camera system is connected to the camera arm via a camera arm joint, and comprises an image sensor and an optical module composed of a zoom imaging optical path, and is used to obtain image information of a surgical field; the focus controller is connected to the camera system, and is used to control the movement of the optical module in the camera system; the display screen is connected to the housing via a display rotating bracket, can perform fixed-axis rotation, and display the maximum definition image obtained by the camera system.

[0008] Furthermore, the box is provided with a motion detection and tracking unit, a robot control unit and an autofocus unit;

[0009] The motion detection and tracking unit is used to keep the camera system focused on the surgical area during the movement of the robotic arm and obtain a central image of the surgical area; the motion detection and tracking unit includes a motion detection module, a feature extraction module and an information processing module, the feature extraction module is used to distinguish feature information of key parts, the motion detection module is used to obtain position information of key parts during the movement, and the information processing module is used to calculate position deviation and inversely solve the movement angle and displacement distance of the robotic arm.

[0010] Furthermore, the feature extraction module is expressed by HOG features (Histogram of Oriented Gradient), and specifically includes the following steps:

[0011] (1) Grayscale the input image

[0012] Perform gamma transformation on the input multi-channel color image input. The formula is as follows:

[0013]

[0014] Among them, output is the gamma transform output image; γ is the gamma coefficient; ∈ is the fixed value compensation to prevent the grayscale from being 0;

[0015] Grayscale to get the grayscale image gray;

[0016] gray(i,j)=0.299×output(i,j)+0.578×output(i,j)+0.114×output(i,j)(2) Calculate the grayscale image gradient

[0017] For the grayscale image gray, use the convolution kernel of the sobel operator in the X and Y directions to get g x and gy :

[0018] Then calculate the gradient amplitude g, the formula is as follows:

[0019]

[0020] Calculate the gradient direction θ, the formula is as follows:

[0021]

[0022] (3) Divide into cells, blocks, and wins

[0023] cell is the basic pixel block, and the default pixel block is 8×8 as one cell; block is composed of integer multiples of cells, and the default is 2×2 cells as one block; win is the bounding box of the target object, and the bounding box size is arbitrary. The default pixel block is 16×128, and the default sliding generates 7×15=105 blocks;

[0024] (4) Calculate the gradient histogram

[0025] Calculate the gradient histogram of each cell. The horizontal axis of the histogram ranges from 0° to 180° and is divided into bins of 20°.

[0026] The pixel gradient direction of the cell falls in the corresponding bin of the histogram, and the pixel gradient amplitude is weighted and accumulated to the bin position, and then counted into a histogram;

[0027] The histogram of each cell is a 1×9 vector, so each block is a 2×2×1×9=1×36 vector;

[0028] In order to improve the lighting robustness, the block vector is normalized again. Here, the L2 normalization formula is selected as follows:

[0029]

[0030] where v o is the output vector, v i is the input vector, ε is the constant compensation, to avoid the denominator being 0, take ε=1e -5 ; (5) Calculate feature descriptors

[0031] The HOG feature descriptor of the target object is calculated according to step S4 with win as the unit; the cell does not slide; the block slides within the win with a pixel step of 8;

[0032] Under the default parameters, the dimension of the HOG feature descriptor is calculated to be 7×15×1×36=1×3780.

[0033] 5. According to claim 3, a surgical exoscopic system with a motion focus function is characterized in that the motion detection module is implemented by a KCF algorithm (Kernelized Correlation Filters). On the one hand, the KCF tracking algorithm has a variety of feature extraction algorithms to choose from, and here the 36-dimensional HOG feature is selected; on the other hand, in order to improve real-time performance, Fourier diagonalization is used to accelerate the calculation, and the specific steps are as follows:

[0034] S1. Before motion detection begins, interactively specify the bounding box of the detected target in the first frame, and take 2.5 times the size of the bounding box as the region of interest (ROI);

[0035] S2. Perform cosine weighting on the samples in the ROI and calculate the HOG features, recorded as x:

[0036] x=[x1,x2,…,x n ] T ,

[0037] Using the permutation matrix P:

[0038]

[0039] Perform 1 left multiplication on x and 1 circular shift:

[0040] Px = [x n ,x1,…,x n-1 ] T

[0041] Left-multiply the features of x n times:

[0042] {P u x|u=0,…,n-1}

[0043] Get the circulant matrix C(x), abbreviated as X:

[0044]

[0045] S3. Use the two-dimensional Gaussian kernel function to construct a relatively smooth training label vector y for x′. The Gaussian kernel function is as follows:

[0046]

[0047] Where x is the first row vector of the circulant matrix X, x′ is the other row vectors of the circulant matrix X, σ is the standard deviation of the Gaussian kernel, which represents the bandwidth parameter of the HOG feature space, and takes σ = 0.5;

[0048] S4. Use Fourier diagonalization to calculate the kernel correlation coefficient of x, x′ The formula is as follows:

[0049]

[0050] in is the inverse Fourier transform, is the vector after Fourier transformation of x and then taking the complex conjugate; the ^ indicates the vector after Fourier transformation, and the * represents the vector after taking the complex conjugate;

[0051] The training results in an initial nonlinear regressor model, referred to as the model, the same below:

[0052]

[0053] in is the Fourier transform vector of the label vector y, λ is the regularization coefficient to prevent the model from overfitting, and λ=1e -5 ;

[0054] S5: When detecting the target in the current frame image, first use the model of the previous frame to predict the initial position of the target, set an area of ​​the same size as the ROI at the initial position, perform cosine weighting on the samples in the area, calculate the HOG features, obtain z, and then use the current frame model trained in step S4 and

[0055] S6. Calculate the frequency domain response matrix The formula is as follows:

[0056]

[0057] right Optimize to get the maximum response value and the maximum response position Idx, given the threshold T, if Then Idx is the target position of the current frame; if If the target is lost, a full-image search strategy can be adopted to reselect a target area and repeat step S1;

[0058] S7, update the model, the model of the current frame is recorded as α′, and the model used in the next frame is α new , according to the current frame and the initial frame model α old The interpolation formula is as follows:

[0059] α new =mα old +(1-m)α′

[0060] Where m is the learning rate, 0 <m<1;

[0061] S8. Repeat steps S5 - S7 to update the tracking position information.

[0062] The robotic arm control unit is used to provide robotic arm motion control, attitude adjustment, automatic positioning, balance control, safety functions, braking functions, sensor feedback, and integration functions, and at the same time receive the movement information fed back by the autofocus unit and send it to the focus controller.

[0063] The autofocus unit is used to process the obtained image information, calculate the moving distance of the optical module by calculating the image sharpness, and feed it back to the robotic arm control unit. The robotic arm control unit adjusts the air gap between different functional lens groups in the optical module through a servo motor for focusing.

[0064] The focus controller receives the information from the autofocus unit and the robotic arm control unit and controls the movement and focusing of the imaging system.

[0065] Furthermore, the autofocus unit searches for the position with the maximum sharpness value by the hill - climbing method, which specifically includes the following steps:

[0066] S1. Before autofocus starts, calculate the current image sharpness evaluation value f0. The image sharpness evaluation function f is a function of the pixel point position (x, y).

[0067] S2. Set the initial step size L1. The initial step size corresponds to the moving distance of the optical module, and the moving direction corresponds to the axial moving direction of the optical module. When the optical module moves according to the initial step size, calculate the current sharpness evaluation value f1.

[0068] S3. Compare the magnitudes of f0 and f1 values. If f0 > f1, the moving direction is incorrect, move in the opposite direction by the step size and recalculate. If f0 < f1, the moving direction is correct.

[0069] S4. Calculate the coefficient k value.

[0070] S5. Recalculate the moving step size L2 = L1 + k * L1. After the external mirror optical module moves L2, calculate the sharpness evaluation value f2.

[0071] S6. Repeat step S3, set the step size extreme value L min , when moving i times, if L i < L min , then the focusing is completed, and the step size L i at this time is the final moving amount.

[0072] The beneficial effects of the present invention are as follows:

[0073] The present invention provides doctors with more convenient operation. During the operation, the doctor obtains deeper information of the operation area by moving the exoscopic mirror. During the movement, the exoscopic mirror tracks the operation area to ensure that the field of view is not lost. At the same time, in the process of tracking the operation area, when the position of the exoscopic mirror changes, the focus position can be adjusted quickly and automatically, ensuring the clarity of the exoscopic mirror image and improving the efficiency of the exoscopic mirror in obtaining the image of the operation area. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 It is a schematic diagram of the three-dimensional structure of the present invention.

[0075] Figure 2 It is a schematic diagram of the motion focus tracking method of the present invention.

[0076] Figure 3 These are the convolution kernels of Sobel in the X and Y directions in the present invention, the left picture is in the X direction, and the right picture is in the Y direction.

[0077] List of reference numerals:

[0078] 1. Box; 2. Operation screen; 3. Display; 4. Mechanical arm; 5. Mechanical forearm joint; 6. Mechanical forearm; 7. Mechanical forearm joint; 8. Mechanical forearm; 9. Mechanical forearm joint; 10. Camera sub-arm; 11. Camera sub-arm joint; 12. Camera system. DETAILED DESCRIPTION

[0079] The present invention will be further explained below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the following specific embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0080] like Figure 1 As shown, the present invention provides a surgical exoscopic system with a motion focus function, comprising a box 1, a mechanical arm, a camera system 12, a focus controller and a display screen 3; the mechanical arm is arranged on the box 1, comprising a mechanical arm 4, a first mechanical arm 6, a first mechanical arm 8 and a camera arm 10 connected in sequence, and each mechanical arm is connected through a mechanical arm joint; the camera system is connected to the camera arm 10 through a camera arm joint 11, comprising an image sensor and an optical module composed of a zoom imaging optical path, for acquiring image information of the surgical field; the focus controller is connected to the camera system, for controlling the movement of the optical module in the camera system; the display screen 3 is connected to the box 1 through a display rotating bracket, can perform fixed-axis rotation, and display the maximum clarity image obtained by the camera system.

[0081] In this embodiment, the housing 1 is provided with a motion detection and tracking unit, a robotic arm control unit and an autofocus unit; the motion detection and tracking unit is used to make the camera system 12 always focus on the surgical area during the movement of the robotic arm to obtain the central image of the surgical area; the motion detection and tracking unit includes a motion detection module, a feature extraction module and an information processing module, the feature extraction module is used to distinguish the feature information of the key parts, the motion detection module is used to obtain the position information of the key parts during the movement, and the information processing module is used to calculate the position deviation and inversely solve the movement angle and displacement distance of the robotic arm; the robotic arm control unit is used to provide robotic arm motion control, posture adjustment, automatic positioning, balance control, safety function, braking function, sensor feedback and integration function, and at the same time receive the movement information fed back by the autofocus unit and send it to the focus controller; the autofocus unit is used to process and process the obtained image information, obtain the moving distance of the optical module by calculating the image clarity, and feed it back to the robotic arm control unit, and the robotic arm control unit adjusts the air gap between the different functional lens groups in the optical module through the servo motor to focus; the focus controller receives the information of the autofocus unit and the robotic arm control unit and controls the movement and focus of the camera system.

[0082] In this embodiment, the feature extraction module is expressed by HOG features (Histogram of Oriented Gradient), and specifically includes the following steps:

[0083] (1) Grayscale the input image

[0084] Perform gamma transformation on the input multi-channel color image input. The formula is as follows:

[0085]

[0086] Among them, output is the gamma transform output image; γ is the gamma coefficient; ∈ is the fixed value compensation to prevent the grayscale from being 0;

[0087] Grayscale to get the grayscale image gray;

[0088] gray(i,j)=0.299×output(i,j)+0.578×output(i,j)+0.114×output(i,j)(2) Calculate the grayscale image gradient

[0089] For the grayscale image gray, use the convolution kernel of the sobel operator in the X and Y directions to get g x and g y :

[0090] Then calculate the gradient amplitude g, the formula is as follows:

[0091]

[0092] Calculate the gradient direction θ, the formula is as follows:

[0093]

[0094] (3) Divide into cells, blocks, and wins

[0095] cell is the basic pixel block, and the default pixel block is 8×8 as one cell; block is composed of integer multiples of cells, and the default is 2×2 cells as one block; win is the bounding box of the target object, and the bounding box size is arbitrary. The default pixel block is 16×128, and the default sliding generates 7×15=105 blocks;

[0096] (4) Calculate the gradient histogram

[0097] Calculate the gradient histogram of each cell. The horizontal axis of the histogram ranges from 0° to 180° and is divided into bins of 20°.

[0098] The pixel gradient direction of the cell falls in the corresponding bin of the histogram, and the pixel gradient amplitude is weighted and accumulated to the bin position, and then counted into a histogram;

[0099] The histogram of each cell is a 1×9 vector, so each block is a 2×2×1×9=1×36 vector;

[0100] In order to improve the lighting robustness, the block vector is normalized again. Here, the L2 normalization formula is selected as follows:

[0101]

[0102] where v o is the output vector, v i is the input vector, ε is the constant compensation, to avoid the denominator being 0, take ε=1e -5 ; (5) Calculate feature descriptors

[0103] The HOG feature descriptor of the target object is calculated according to step S4 with win as the unit; the cell does not slide; the block slides within the win with a pixel step of 8;

[0104] Under the default parameters, the dimension of the HOG feature descriptor is calculated to be 7×15×1×36=1×3780.

[0105] In this embodiment, the motion detection module is implemented by the KCF algorithm (Kernelized Correlation Filters). On the one hand, the KCF tracking algorithm has a variety of feature extraction algorithms to choose from. Here, the 36-dimensional HOG feature is constructed. On the other hand, in order to improve real-time performance, Fourier diagonalization is used to accelerate the calculation. The specific steps are as follows:

[0106] S1. Before motion detection begins, interactively specify the bounding box of the detected target in the first frame, and take 2.5 times the size of the bounding box as the region of interest (ROI);

[0107] S2. Perform cosine weighting on the samples in the ROI and calculate the HOG features, recorded as x:

[0108] x=[x1,x2,…,x n ] T ,

[0109] Using the permutation matrix P:

[0110]

[0111] Multiply x by one left bit and perform one circular shift:

[0112] Px = [x n ,x1,…,x n-1 ] T

[0113] Left-multiply the features of x n times:

[0114] {P u x|u=0,…,n-1}

[0115] Get the circulant matrix C(x), abbreviated as X:

[0116]

[0117] S3. Use the two-dimensional Gaussian kernel function to construct a relatively smooth training label vector y for x′. The Gaussian kernel function is as follows:

[0118]

[0119] Where x is the first row vector of the circulant matrix X, x′ is the other row vectors of the circulant matrix X, σ is the standard deviation of the Gaussian kernel, which represents the bandwidth parameter of the HOG feature space, and takes σ = 0.5;

[0120] S4. Use Fourier diagonalization to calculate the kernel correlation coefficient of x, x′ , the formula is as follows:

[0121]

[0122] in is the inverse Fourier transform, is the vector after Fourier transformation of x and then taking the complex conjugate; the ^ indicates the vector after Fourier transformation, and the * represents the vector after taking the complex conjugate;

[0123] The training results in an initial nonlinear regressor model, referred to as the model, the same below:

[0124]

[0125] in is the Fourier transform vector of the label vector y, λ is the regularization coefficient to prevent the model from overfitting, and λ=1e -5 ;

[0126] S5: When detecting the target in the current frame image, first use the model of the previous frame to predict the initial position of the target, set an area of ​​the same size as the ROI at the initial position, perform cosine weighting on the samples in the area, calculate the HOG features, obtain z, and then use the current frame model trained in step S4 and

[0127] S6. Calculate the frequency domain response matrix The formula is as follows:

[0128]

[0129] right Optimize to get the maximum response value and the maximum response position Idx, given the threshold T, if Then Idx is the target position of the current frame; if If the target is lost, a full-image search strategy can be adopted to reselect a target area and repeat step S1;

[0130] S7, update the model, the model of the current frame is recorded as α′, and the model used in the next frame is α new , according to the current frame and the initial frame model α old The interpolation formula is as follows:

[0131] α new =mα old +(1-m)α′

[0132] Where m is the learning rate, 0 <m<1;

[0133] S8. Repeat steps S5-S7 to update the tracking location information.

[0134] As Figure 2 shown, in order to achieve real-time locking of the key parts of the surgical area during the movement of the external endoscope system and maintain the clarity and non-deviation of the surgical image, the present invention is implemented by the automatic control of the robotic arm and the automatic tracking algorithm.

[0135] In this embodiment, at the beginning of the operation, at the initial position, the imaging system 12 obtains an image of the surgical area and transmits it back to the autofocus unit inside the box. The autofocus unit processes the obtained image. In this embodiment, the autofocus unit obtains the distance that the optical module needs to move through the hill-climbing method, and the control unit issues the optical module movement distance information to the focus controller, and the focus controller controls the movement of the internal optical module of the imaging system until an image with the maximum clarity is obtained.

[0136] Among them, the autofocus unit searches for the position with the maximum clarity value through the hill-climbing method. The hill-climbing method simulates a blind person climbing a mountain. Starting from the starting point, it climbs the slope along a certain direction with a certain large step size to change the focal length, and compares by calculating the clarity evaluation value each time. When the descending edge of the slope is detected, the step size is reduced and the slope is climbed in the reverse direction. This process is repeated until the step size is reduced to the preset termination step size and the climbing stops. The peak position during the last climbing process is the maximum value searched. This method includes the following steps:

[0137] S1. Before the autofocus starts, calculate the current image clarity evaluation value f0. The image clarity evaluation function f is a function of the pixel position (x, y).

[0138] S2. Set the initial step size L1. The initial step size corresponds to the movement distance of the optical module in the external endoscope, and the movement direction corresponds to the axial movement direction of the optical module. When the optical module moves according to the initial step size, calculate the current clarity evaluation value f1;

[0139] S3. Compare the magnitudes of f0 and f1 values. If f0 > f1, the movement direction is incorrect and it is necessary to move the step size in the reverse direction and recalculate. If f0 < f1, the movement direction is correct.

[0140] S4. Calculate the coefficient k value,

[0141] S5. Recalculate the movement step size L2 = L1 + k * L1. After the external endoscope optical module moves L2, calculate the clarity evaluation value f2.

[0142] S6. Repeat step S3, set the step size extreme value L min , when moving i times, if L i < L min , then the autofocus is completed, and the step size L i at this time is the final movement amount.

[0143] In this embodiment, the camera system obtains images with maximum definition and outputs the information to a display screen.

[0144] In this embodiment, during the operation, the doctor needs different observation angles to obtain more details of the surgical area. In this embodiment, a voice interaction module is used to interact with the instructions of medical staff. The voice interaction module includes the medical staff issuing control instructions through the voice system, and the control unit changes the spatial position and lens orientation of the camera system by controlling the movement of the robotic arm. During the movement, the control unit transmits the rotation angle of each connected joint to the robotic arm.

[0145] In this embodiment, it is achieved by moving and rotating the robotic arm, which includes a robotic arm 4, a robotic arm joint 5, a first robotic forearm 6, a second robotic forearm 8, a first robotic forearm joint 7, a second robotic forearm joint 9, a camera arm 10, a camera arm joint 11 and a camera system 12.

[0146] Specifically, the mechanical arm 4 is fixed on the box body, and the mechanical arm 6 is connected to the mechanical arm 4 through the mechanical arm joint 5. The mechanical arm 8 is connected to the first mechanical arm 6 through the first mechanical arm joint 7. The camera arm 10 is connected to the second mechanical arm 8 through the second mechanical arm joint 9. The camera system 12 is connected to the camera arm through the camera arm joint 11. The joints in this embodiment are spherical joints, which can rotate around three mutually perpendicular axes and provide three degrees of freedom, so the mechanical arm in this embodiment is flexible and has a large range of motion.

[0147] The external mirror completes the automatic focusing of the initial position. The feature extraction module uses a machine vision algorithm to detect the surgical object. In this embodiment, the YOLOv5 algorithm is used for target detection. YOLOv5 can detect multiple objects in the image and output their information, including features and position information.

[0148] In this embodiment, after the robot arm receives the control command, the joint rotation angle θ is divided into several θ i The angle of joint rotation in the first movement is θ i .

[0149] When the designated position is reached, the autofocus unit controls the movement of the optical module again through the algorithm to obtain a clear image of the surgical target area. The feature extraction module detects the image again, obtains the position information of the key area through feature recognition, and sends the position information to the information processing module.

[0150] The information processing module determines whether the key part exceeds the center frame. When the position of the key part in the image changes significantly, the information processing module calculates the offset of the image and inversely calculates the rotation angle and displacement distance of the robot arm joint. The control unit controls the movement of the robot arm based on the data obtained by the inverse solution so that the key part is always in the center frame of the screen.

[0151] When the feature extraction module cannot detect the key parts in the current image through feature recognition, the robot arm returns to the position of the previous step. Reduce the joint rotation angle θ i , the control unit controls the rotation of the robot arm joint again. Repeat the steps of autofocus and feature extraction. This measure can ensure that the image of the key parts is always presented in the center frame area of ​​the screen during the movement of the robot arm, and avoid the situation where the robot arm loses image information.

[0152] The technical means disclosed in the scheme of the present invention are not limited to the technical means disclosed in the above-mentioned implementation mode, but also include technical schemes composed of any combination of the above-mentioned technical features. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications are also regarded as the protection scope of the present invention.

Claims

1. A surgical exoscopic system with a motion focus function, characterized in that: The invention comprises a housing (1), a mechanical arm, a camera system (12), a focus controller and a display screen (3); the mechanical arm is arranged on the housing (1), and comprises a mechanical arm (4), a first mechanical arm (6), a first mechanical arm (8) and a camera arm (10) which are connected in sequence, and each mechanical arm is connected via a mechanical arm joint; the camera system is connected to the camera arm (10) via a camera arm joint (11), and comprises an image sensor and an optical module composed of a zoom imaging optical path, and is used to obtain image information of a surgical field; the focus controller is connected to the camera system, and is used to control the movement of the optical module in the camera system; the display screen (3) is connected to the housing (1) via a display rotating bracket, and can rotate along a fixed axis, and displays an image with maximum clarity obtained by the camera system.

2. The surgical exoscopic system with motion focus function according to claim 1, characterized in that: The box (1) is provided with a motion detection and tracking unit, a mechanical arm control unit and an automatic focusing unit.

3. The surgical exoscopic system with motion focus function according to claim 2, characterized in that: The motion detection and tracking unit is used to make the camera system (12) always focus on the surgical area during the movement of the robotic arm, so as to obtain a central image of the surgical area; the motion detection and tracking unit comprises a motion detection module, a feature extraction module and an information processing module, the feature extraction module is used to distinguish feature information of key parts, the motion detection module is used to obtain position information of key parts during the movement, and the information processing module is used to calculate the position deviation and inversely solve the movement angle and displacement distance of the robotic arm.

4. The surgical exoscopic system with a motion focus function according to claim 3, characterized in that: The feature extraction module is expressed by HOG features, and specifically includes the following steps: (1) Grayscale the input image Perform gamma transformation on the input multi-channel color image input. The formula is as follows: Among them, output is the gamma transform output image; γ is the gamma coefficient; ∈ is the fixed value compensation to prevent the grayscale from being 0; Grayscale to get the grayscale image gray; gray(i,j)=0.299×output(i,j)+0.578×output(i,j)+0.114×output(i,j)(2) Calculate the grayscale image gradient For the grayscale image gray, use the convolution kernel of the sobel operator in the X and Y directions to get g x and g y : Then calculate the gradient amplitude g, the formula is as follows: Calculate the gradient direction θ, the formula is as follows: (3) Divide into cells, blocks, and wins cell is the basic pixel block, and the default pixel block is 8×8 as one cell; block is composed of integer multiples of cells, and the default is 2×2 cells as one block; win is the bounding box of the target object, and the bounding box size is arbitrary. The default pixel block is 16×128, and the default sliding generates 7×15=105 blocks; (4) Calculate the gradient histogram Calculate the gradient histogram of each cell. The horizontal axis of the histogram ranges from 0° to 180° and is divided into bins of 20°. The pixel gradient direction of the cell falls in the corresponding bin of the histogram, and the pixel gradient amplitude is weighted and accumulated to the bin position, and then counted into a histogram; The histogram of each cell is a 1×9 vector, so each block is a 2×2×1×9=1×36 vector; To improve the illumination robustness, the vectors of the blocks are further normalized. Here, the formula for L2 normalization is as follows: where v o is the output vector, v i is the input vector, ε is the constant compensation, to avoid the denominator being 0, take ε=1e -5 ; (5) Calculate feature descriptors The HOG feature descriptor of the target object is calculated in units of win according to step S4; the cells do not slide; the blocks slide within win with a pixel step of 8. Under the default parameters, the dimension of the calculated HOG feature descriptor is 7×15×1×36 = 1×3780.

5. The surgical exoscopic system with motion focus function according to claim 3, characterized in that: The motion detection module is implemented by the KCF algorithm. On the one hand, there are multiple feature extraction algorithms available for the KCF tracking algorithm. Here, a 36-dimensional HOG feature is constructed. On the other hand, to improve the real-time performance, Fourier diagonalization is used to accelerate the calculation. The specific steps are as follows: S1. Before the motion detection starts, interactively specify the bounding box of the detection target in the first frame, and take 2.5 times the size of the bounding box as the region of interest ROI. S2. Cosine-weight the samples within the ROI and calculate the HOG feature, denoted as x. x=[x1,x2,…,x n ] T , Use the permutation matrix P: Perform one left multiplication on x to complete one cyclic shift. Px=[x n ,x1,…,x n-1 ] T Perform n left multiplications on the x feature. {P u y|u=0,…,n-1} Obtain the circulant matrix C(x), simply denoted as X. S3. Through the two-dimensional Gaussian kernel function, construct a relatively smooth training label vector y for x′. The Gaussian kernel function is as follows: Among them, x is the first row vector of the circulant matrix X quantity, and x′ is the other row vectors of the circulant matrix X quantity. σ is the Gaussian kernel standard deviation, which represents the bandwidth parameter of the HOG feature space here, and σ = 0.5 is taken. S4. Use Fourier diagonalization to calculate the kernel correlation coefficient of x, x′ The formula is as follows: in is the inverse Fourier transform, is the vector obtained by taking the complex conjugate of x after Fourier transformation; the vector with ^ indicates the vector after Fourier transformation, and the vector with * indicates the vector after taking the complex conjugate; Train an initial non-linear regressor model, simply referred to as the model, the same below. in is the Fourier transform vector of the label vector y, λ is the regularization coefficient to prevent the model from overfitting, and λ=1e -5 ; S5: When detecting the target in the current frame image, first use the model of the previous frame to predict the initial position of the target, set an area of ​​the same size as the ROI at the initial position, perform cosine weighting on the samples in the area, calculate the HOG features, obtain z, and then use the current frame model trained in step S4 and S6. Calculate the frequency domain response matrix The formula is as follows: right Optimize to get the maximum response value and the maximum response position Idx, given the threshold T, if Then Idx is the target position of the current frame; if If the target is lost, a full-image search strategy can be adopted to reselect a target area and repeat step S1; S7, update the model, the model of the current frame is recorded as α′, and the model used in the next frame is α new , according to the current frame and the initial frame model α old The interpolation formula is as follows: a new =ma old +(1-m)α′ Among them, m is the learning rate, 0 < m < 1. S8. Repeat steps S5 - S7 to update the tracking position information.

6. The surgical exoscopic system with a motion focus function according to claim 3, characterized in that: The robotic arm control unit is used to provide robotic arm motion control, attitude adjustment, automatic positioning, balance control, safety functions, braking functions, sensor feedback, and integration functions, and at the same time receive the movement information fed back by the autofocus unit and send it to the focus controller.

7. The surgical exoscopic system with a motion focus function according to claim 6, characterized in that: The autofocus unit is used to process the obtained image information. By calculating the image sharpness, obtain the moving distance of the optical module and feedback it to the robotic arm control unit. The robotic arm control unit adjusts the air gap between different functional lens groups in the optical module through the servo motor for focusing.

8. The surgical exoscopic system with a motion focus function according to claim 7, characterized in that: The focus controller receives the information from the autofocus unit and the robotic arm control unit and controls the movement and focusing of the imaging system.

9. The surgical exoscopic system with a motion focus function according to claim 8, characterized in that: The autofocus unit searches for the position with the maximum sharpness value by the hill climbing method, specifically including the following steps: S1. Before the autofocus starts, calculate the current image sharpness evaluation value f0. The image sharpness evaluation function f is a function of the pixel point position (x, y). S2. Set the initial step size L1. The initial step size corresponds to the moving distance of the optical module, and the moving direction corresponds to the axial moving direction of the optical module. When the optical module moves according to the initial step size, calculate the current sharpness evaluation value f1. S3. Compare the magnitudes of f0 and f1 values. If f0 > f1, the moving direction is incorrect, move in the opposite direction by the step size and recalculate. If f0 < f1, the moving direction is correct. S4, calculate the coefficient k value, S5, recalculate the moving step length L2 = L1 + k*L1, and after the exterior mirror optical module moves L2, calculate the clarity evaluation value f2; S6, repeat step S3, set the step length extreme value L min , after moving i times, if L i <L min , then the step length L at which the focus is completed i The final movement amount.

Images