Surgical system master-slave operation judgment method and device and storage medium

Abstract

The application relates to the technical field of medical equipment, in particular to a surgical system master-slave operation judgment method and device and a storage medium. The surgical system master-slave operation judgment method comprises the following steps: obtaining a face image of an operator; performing feature recognition according to the face image to obtain a glasses area; obtaining multiple edge contours of the glasses area through edge detection according to the glasses area; screening all the edge contours to obtain a marked edge contour; obtaining a contour fitting circle through circle fitting according to the marked edge contour; obtaining the radius of the contour fitting circle, and screening the contour fitting circle according to all the radii and a preset radius threshold to obtain a marked fitting circle; judging whether the operator looks at the screen according to the marked fitting circle; and when the operator looks at the screen and a double-click master hand signal of the operator is obtained, master-slave operation is performed. The technical scheme of the application can effectively reduce the misoperation in the surgical process.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical equipment, in particular to a surgical system master-slave operation judgment method and device and a storage medium. BACKGROUND

[0002] With the continuous progress of science and technology and the continuous innovation of medical technology, surgical systems have made great progress and have been widely used in laparoscopic surgery to assist doctors in performing complex surgical actions, reduce tremor errors during surgery, and help doctors perform precise, stable, and controllable operations.

[0003] A laparoscopic surgery system mainly includes a surgical operation end, an image system, and a surgical execution end. When performing stereoscopic image guided surgery, the operator needs to wear pre-prepared 3D glasses to operate, and needs to pay high attention during operation to ensure accurate and stable control. The execution end is controlled by the operation end to operate, but in the actual operation process, the operator's attention may be distracted due to external personnel or environmental factors, and at this time the hand has not left the surgical operation end, so the extra action of the hand may cause misoperation and cause damage or equipment failure. SUMMARY

[0004] The problem solved by the present application is how to reduce misoperation during surgery.

[0005] To solve the above problems, the present application provides a surgical system master-slave operation judgment method, device and storage medium.

[0006] In a first aspect, the present application provides a surgical system master-slave operation judgment method, comprising:

[0007] Obtaining a face image of an operator;

[0008] Performing feature recognition according to the face image to obtain a glasses region;

[0009] According to the glasses region, a plurality of edge contours of the glasses region are obtained through edge detection;

[0010] All the edge contours are screened to obtain a marked edge contour;

[0011] According to the marked edge contour, a contour fitting circle is obtained through circle fitting;

[0012] According to the contour fitting circle, a radius of the contour fitting circle is obtained, and all the radii and a preset radius threshold are used to screen the contour fitting circle to obtain a marked fitting circle;

[0013] Fitting a circle according to the mark, judging whether the operator looks at the screen or not;

[0014] When the operator looks at the screen and a double-click master hand signal of the operator is acquired, master-slave operation is performed, otherwise, the master-slave operation is stopped.

[0015] Optionally, the feature recognition according to the face image comprises:

[0016] The pixel value of each pixel point of the face image is obtained according to the face image;

[0017] The face image is binarized according to a preset binarization threshold to obtain a binary face image;

[0018] The glasses region is obtained according to the binary face image and a preset glasses template image through a template matching algorithm.

[0019] Optionally, the glasses region is obtained according to the binary face image and a preset glasses template image through a template matching algorithm, comprising:

[0020] The size of the preset glasses template image is determined as a matching window size, and a step length is set as 1;

[0021] The face image is selected as a plurality of matching regions through sliding according to the matching window and the step length;

[0022] The matching coefficient of each matching region is obtained through a matching relationship according to each matching region and the preset glasses template;

[0023] The matching region with a matching coefficient less than a preset matching threshold is determined as the glasses region.

[0024] Optionally, the edge contour is selected to obtain a mark edge contour, comprising:

[0025] The number of pixel points in the edge contour is determined according to the edge contour;

[0026] The edge contour area is determined according to the number of pixel points;

[0027] The median edge contour area is determined according to all the edge contour areas;

[0028] The edge contour is selected according to the median edge contour area to obtain the mark edge contour.

[0029] Optionally, the contour fitting circle is obtained through circle fitting according to the mark edge contour, comprising:

[0030] According to each of the mark edge profile, an edge pixel point corresponding to the mark edge profile is obtained;

[0031] According to the edge pixel point, a mark fitting circle corresponding to the mark edge profile is obtained through circle fitting.

[0032] Optionally, the filtering of the profile fitting circles according to all the radii and a preset radius threshold value to obtain a mark fitting circle comprises:

[0033] According to each of the radii, the radius threshold value and a radius deviation relationship, a radius difference of each of the profile fitting circles is obtained;

[0034] When the radius difference is less than a preset difference value, the profile fitting circle is determined as a mark fitting circle;

[0035] The radius deviation relationship satisfies:

[0036]

[0037] Wherein, R i is the radius difference of the i-th profile fitting circle, r i is the radius of the i-th profile fitting circle, and d is a radius threshold value.

[0038] Optionally, the judging of whether the operator looks at the screen according to the mark fitting circle comprises:

[0039] When the number of the mark fitting circles is less than 3, it is judged that the operator does not look at the screen;

[0040] When the number of the mark fitting circles is greater than or equal to 3, any three of the mark fitting circles are obtained;

[0041] It is judged whether the positional relationship of the three mark fitting circles satisfies a preset condition, if yes, it is judged that the operator looks at the screen, otherwise, it is judged that the operator does not look at the screen.

[0042] Optionally, the judging of whether the positional relationship of the three mark fitting circles satisfies a preset condition, if yes, it is judged that the operator looks at the screen, otherwise, it is judged that the operator does not look at the screen, comprises:

[0043] According to the three mark fitting circles, respective corresponding center positions and radii of the circles are obtained;

[0044] According to the three center positions and a center position relationship, a center distance of each two of the mark fitting circles is determined;

[0045] When two of the center distances are equal, and the difference value of the three radii is less than a preset value, it is judged that the operator looks at the screen.

[0046] Otherwise, it is judged that the operator does not look at the screen.

[0047] In a second aspect, an electronic device includes a memory and a processor;

[0048] The memory is configured to store a computer program;

[0049] The processor is configured to implement the surgical system master-slave operation judgment method of the first aspect when executing the computer program.

[0050] In a fourth aspect, a computer readable storage medium stores a computer program, and when the computer program is executed by a processor, the surgical system master-slave operation judgment method of the first aspect is implemented.

[0051] The surgical system master-slave operation judgment method, device and storage medium have the following advantages: the feature recognition is performed on the face image collected by the infrared camera, the glasses area in the face image is obtained, the 3D glasses of the operator have a mark point, the mark point is searched in the glasses area, the range of data processing is reduced, the accuracy of the mark point searching is improved, and the line of sight attention position of the operator can be accurately judged. The edge detection algorithm is used to perform edge detection on the glasses area, a plurality of edge contours are obtained, the plurality of edge contours are screened, the mark edge contour is obtained, the edge contour that does not conform to the mark point contour in the glasses is removed, the interference of the non-mark point contour on the subsequent image analysis and processing result is reduced, the data amount of processing is reduced, the efficiency of the mark point is improved, the mark edge contour obtained after the screening is fitted by a circle, a contour fitting circle is obtained, the mark fitting circle that conforms to the mark point in the 3D glasses worn by the operator is screened through the radius of the contour fitting circle, and then whether the operator looks at the screen is judged according to the mark fitting circle. When the operator looks at the screen and the operator double-clicks the master hand, the master-slave operation can be performed, otherwise, the master-slave operation is not performed, and the master-slave operation is stopped when the line of sight of the operator leaves the screen. The misoperation caused by not paying attention to the screen can be effectively avoided, the safety in the surgical process is improved, and the occurrence of equipment damage and accidental injury is effectively avoided. BRIEF DESCRIPTION OF DRAWINGS

[0052] Figure 1 FIG. 1 is a flowchart of a surgical system master-slave operation judgment method according to an embodiment of the present application;

[0053] Figure 2 FIG. 3 is a schematic diagram of a 3D glasses mark point according to an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the above objectives, characteristics and advantages of the present application more apparent, concrete embodiments of the present application will be described in detail below with reference to the drawings. Although some embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms, and should not be interpreted as being limited to the embodiments set forth herein, but rather, these embodiments are provided in order to more thoroughly and completely understand the present application. It should be understood that the drawings and embodiments of the present application are for exemplary purposes only, and are not intended to limit the scope of protection of the present application.

[0055] It should be understood that each of the steps described in the method embodiments of the present application can be performed in different orders, and / or in parallel. In addition, the method embodiments can include additional steps and / or omit the steps shown. The scope of the present application is not limited in this respect.

[0056] The term "comprising" and variations thereof as used herein are open-ended, that is "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments"; the term "optional" means "optional in at least some embodiments". Related definitions will be given in the description below. It should be noted that the concepts of "first", "second", etc. mentioned in the present application are only used to distinguish different devices, modules or units, and are not intended to limit the order or interdependence of the functions performed by these devices, modules or units.

[0057] It should be noted that the modification of "one" or "multiple" mentioned in the present application is illustrative and not limiting, and those skilled in the art should understand that unless otherwise explicitly indicated in the context, it should be understood as "one or more".

[0058] In the surgical system of the present embodiment, an infrared camera or other acquisition device can be provided on the operation screen, and a marker point is provided on the 3D glasses worn by the operator, as shown in Figure 2 The three marker points (three circular markers on the 3D glasses) are arranged in an isosceles triangle, and the color of the marker points has a significant color difference from the overall color of the 3D glasses, for example, the 3D glasses are generally black or dark gray, and the color of the marker points is white. In this way, accurate marker point contour features can be obtained during image processing, facilitating the determination of the marker points.

[0059] The master-slave operation judgment method of the surgical system of the present embodiment mainly determines whether the operator is looking at the screen by finding the marker points and judging the position relationship of the markers, and then determines whether to perform master-slave operation.

[0060] As shown in Figure 1To solve the above technical problems, the embodiment of the present application provides a surgical system master-slave operation judgment method, comprising:

[0061] Step S1, obtaining a face image of an operator;

[0062] Specifically, the on-site image is collected by an infrared camera and the like, and face recognition is performed on the on-site image to determine whether there is a face in the on-site image; when no face is recognized, it is determined that the operator is not looking at the screen; when a face is recognized in the on-site image, a face image with the face is obtained for subsequent operation.

[0063] Step S2, performing feature recognition on the face image to obtain a glasses region;

[0064] Specifically, the operator watches the 3D effect in the screen by wearing 3D glasses, and operates according to the situation watched by the 3D glasses; through feature recognition on the face image, when the recognized region is matched with the shape of the 3D glasses, the region is determined as the glasses region; the line-of-sight state of the operator can be accurately determined according to the glasses region, and then it is determined whether the operator is watching the screen.

[0065] Since there may be multiple personnel wearing 3D glasses watching the screen around, only the 3D glasses of the actual operator have three circular marker points set in advance, so multiple regions matched with the shape of the 3D glasses may be obtained, which are collectively referred to as the glasses region, that is, the glasses region may include multiple independent regions.

[0066] Step S3, obtaining multiple edge contours of the glasses region through edge detection according to the glasses region;

[0067] Specifically, the multiple edge contours may include the edge contour of the glasses, the edge contour of the marker points, and the edge contour of some light spots. It can be understood that generally, the edge contour area of the glasses structure is large, the edge contour area of the light spot is small, and the edge contour area of the marker point is moderate.

[0068] Specifically, the edge detection algorithm such as edge detection operator (Canny), difference operator (Roberts), and gradient operator (Prewitt) is used to perform edge detection on the glasses region to obtain multiple edge contours in the glasses region.

[0069] Step S4, screening all the edge contours to obtain a marker edge contour;

[0070] Further, by screening the edge contour, too large and too small areas in the edge contour area are screened, and some edge contours with moderate area are obtained, i.e., marker edge contours, and further screening areas conforming to the marker points in the marker edge contours can improve the efficiency and accuracy of extracting the marker points.

[0071] Step S5, according to the marker edge contour, a contour fitting circle is obtained through circle fitting;

[0072] Step S6, a radius of the contour fitting circle is obtained according to the contour fitting circle, and the contour fitting circle is screened according to all the radii and a preset radius threshold to obtain a marker fitting circle;

[0073] Specifically, circle fitting is performed on each marker edge contour screened to obtain a contour fitting circle approximating the marker edge contour, and a radius of the contour fitting circle is determined, and a difference between each radius and the preset radius threshold is calculated, and when the difference satisfies a preset range, the contour fitting circle corresponding to the radius is determined as the marker fitting circle. The preset radius threshold is determined according to the radius of the marker point, and specifically can be equal to the radius of the marker point. The marker fitting circle screened through step S6 is close to the contour of the marker point, and the range of the marker point is further narrowed.

[0074] Step S7, according to the marker fitting circle, whether the operator looks at the screen is determined;

[0075] Step S8, when the operator looks at the screen and a double-click master hand signal of the operator is acquired, master-slave operation is performed, otherwise, the master-slave operation is stopped.

[0076] Specifically, the line of sight attention area of the operator is determined according to the positional relationship of the marker fitting circle, so as to determine whether the operator watches the screen, for example, if the marker fitting circle has a fitting circle conforming to the positional relationship of three marker points, it can be determined that the 3D glasses of the operator are directly opposite to the operation screen, i.e., the line of sight of the operator is focused on the screen; when the operator watches the screen and the double-click master hand signal of the operator is detected, the master-slave operation is performed, and as long as the above conditions are not met at the same time, the master-slave operation is stopped, which can effectively avoid the misoperation when the operator does not watch the screen.

[0077] In the embodiment, the feature recognition is performed on the face image collected by the infrared camera to obtain the glasses region in the face image. Since the 3D glasses of the operator have a mark point, the mark point is searched in the glasses region, the range of data processing is reduced, the accuracy of mark point searching is improved, and then the line of sight attention position of the operator can be accurately judged. The edge detection algorithm is used to perform edge detection on the glasses region to obtain a plurality of edge contours. The plurality of edge contours are screened to obtain a mark edge contour. The edge contour that does not conform to the mark point contour in the glasses is removed, the interference of the non-mark point contour on the subsequent image analysis and processing result is reduced, the data amount of processing is reduced, the efficiency of searching the mark point is improved, the mark edge contour obtained after screening is fitted by a circle to obtain a contour fitting circle, and the mark fitting circle that conforms to the mark point in the 3D glasses worn by the operator is screened according to the radius of the contour fitting circle. Then, whether the operator looks at the screen is judged according to the mark fitting circle. When the operator looks at the screen and the operator double-clicks the master hand, the master-slave operation can be performed. Otherwise, the master-slave operation is not performed. When the line of sight of the operator is away from the screen, the master-slave operation is stopped. The misoperation caused by not paying attention to the screen can be effectively avoided, the safety in the operation process is improved, and the damage and injury of the equipment are effectively avoided.

[0078] In an optional embodiment, the feature recognition according to the face image to obtain the glasses region comprises:

[0079] The pixel value of each pixel point of the face image is obtained according to the face image.

[0080] The face image is binarized according to a preset binarization threshold to obtain a binary face image.

[0081] The glasses region is obtained by a template matching algorithm according to the binary face image and a preset glasses template image.

[0082] In an optional embodiment, the glasses region is obtained by a template matching algorithm according to the binary face image and a preset glasses template image, which comprises:

[0083] The size of the preset glasses template image is determined as the size of a matching window, and the step length is set to 1.

[0084] The face image is selected as a plurality of matching regions by sliding according to the matching window and the step length.

[0085] The matching coefficient of each matching region is obtained by a matching relationship according to each matching region and the preset glasses template.

[0086] The matching region with a matching coefficient less than a preset matching threshold is determined as the glasses region.

[0087] Specifically, the pixel value of each pixel point in the face image is obtained, the pixel value is compared with a preset binarization threshold, the pixel point is revalued according to the comparison result, and the pixel point is revalued as 0 if the pixel value is greater than or equal to the binarization threshold, or revalued as 255 if the pixel value is less than the binarization threshold. For example, if the pixel value of the current pixel point is 89 and the binarization threshold is 125, the current pixel value is less than the binarization threshold, and the current pixel point is revalued as 255. If the current pixel value is 220, the pixel value is revalued as 0. Finally, a binary face image is generated according to all the revalued pixel values.

[0088] Further, the size of the matching window is set to the size of the preset glasses template image, and the binary face image is matched by a sliding window with a step of 1. The matching coefficient of each matching region and the preset glasses template image is calculated according to the matching relationship, and when the matching coefficient is less than a preset matching threshold, it is determined that the matching region matches the preset glasses template, and the matching region is determined as the glasses region. In actual operation, there may be multiple personnel wearing 3D glasses watching the screen, and only the 3D glasses of the actual operator have three circular marker points set in advance, so multiple matching regions that meet the matching requirements may be obtained, and all the matching regions are determined as the glasses region, so that accurate marker points are obtained from the glasses region.

[0089] Further, the matching relationship satisfies:

[0090]

[0091] Wherein, M is the matching coefficient, n is the number of pixel points of the preset glasses template image, x i is the pixel value of the i-th pixel point of the preset glasses template image, y i is the pixel value of the i-th pixel point of the matching region.

[0092] In this optional embodiment, the face image is binarized to effectively simplify the image, avoid the interference of noise points in the image on subsequent extraction of marker fitting circles, highlight the features of 3D glasses and marker points set on the 3D glasses in the face image, reduce the noise in the image, and reduce the storage space of the image and improve the speed of image processing. According to the matching of the binarized face image to the glasses region, the range of searching for the marker points can be further narrowed down through the glasses region, and the operator can be accurately determined whether he is watching the screen through the found marker points.

[0093] In an optional embodiment, the filtering of all the edge contours to obtain the marker edge contour comprises:

[0094] determining a number of pixels in the edge contour according to the edge contour;

[0095] determining an edge contour area according to the number of pixels;

[0096] determining a median edge contour area according to all the edge contour areas;

[0097] screening the edge contour according to the median edge contour area to obtain the marker edge contour.

[0098] Specifically, the number of pixels in the edge contour is determined according to the edge contour. Since the size of each pixel is the same, the number of pixels in the area is set as the edge contour area of the edge contour. All the edge contour areas are sorted, the median edge contour area is determined as the median edge contour area, and the edge contour is screened according to the median edge contour area. Specifically, the median edge contour area is taken as a reference, and a plurality of edge contours close to the median edge contour area are screened as the marker edge contour. Specifically, a preset number of edge contours greater than the median edge contour area and a preset number of edge contours less than the median edge contour area can be selected. For example, if the preset number is 5, the 5 edge contours with the edge contour area greater than the median edge contour area are determined as the marker edge contour, and the 5 edge contours with the edge contour area less than the median edge contour area are also determined as the marker edge contour.

[0099] In the optional embodiment, the edge contour is screened according to the area of the edge contour to obtain the marker edge contour close to the marker point area, thereby reducing the data amount of subsequent image processing and improving the processing speed. At the same time, the edge contour with a large difference from the marker point area is removed, thereby reducing the interference of the edge contour which is not the 3D glasses marker point on subsequent data processing and improving the accuracy of the marker fitting circle obtained finally.

[0100] In an optional embodiment, the marker edge contour is obtained by circle fitting according to the marker edge contour, comprising:

[0101] According to each of the marker edge contour, the edge pixel point corresponding to the marker edge contour is obtained;

[0102] According to the edge pixel point, the marker fitting circle corresponding to the marker edge contour is obtained by circle fitting.

[0103] Specifically, a plurality of pixels of the marker edge contour are obtained according to the marker edge contour. The contour fitting circle of the marker edge contour, the coordinate of the center position of the fitting circle and the radius of the contour fitting circle are obtained by the least square method, which is a prior art and will not be described here. The contour fitting circle is more consistent with the shape of the marker point, and the marker point can be more accurately obtained by the contour fitting circle.

[0104] In an optional embodiment, the screening of the contour fitting circles according to all the radii and a preset radius threshold value to obtain the marker fitting circles comprises:

[0105] According to each of the radii, the radius threshold value and a radius deviation relationship, a radius difference of each of the contour fitting circles is obtained;

[0106] When the radius difference is less than a preset difference value, the contour fitting circle is determined as a marker fitting circle;

[0107] The radius deviation relationship satisfies:

[0108]

[0109] wherein R i is the radius difference of the i-th contour fitting circle, r i is the radius of the i-th contour fitting circle, and d is a radius threshold value.

[0110] Exemplarily, when the radius of contour fitting circle 1 is 5, the radius of contour fitting circle 2 is 7, the radius threshold value is 4 and the preset difference value is 2, the radius deviation relationship obtains that the radius deviation of contour fitting circle 1 is 1, and the radius deviation of contour fitting circle 2 is 3. Since the radius deviation of contour fitting circle 1 is 1 which is less than the preset difference value 2, contour fitting circle 1 is determined as a marker fitting circle, while the radius deviation of contour fitting circle 2 is 3 which is greater than the preset difference value 2, so contour fitting circle 2 is not a marker fitting circle.

[0111] In the optional embodiment, the contour fitting circles are screened by the radius threshold value to find the marker fitting circles which conform to the shape of the marker point. The radius threshold value can be set as the radius of the marker point, so that the screened marker fitting circles are similar to the marker point, and the implementation of whether the operator looks at the screen can be accurately determined according to the obtained marker fitting circles.

[0112] In an optional embodiment, the judging of whether the operator looks at the screen according to the marker fitting circles comprises:

[0113] When the number of the marker fitting circles is less than 3, it is judged that the operator does not look at the screen;

[0114] When the number of the marker fitting circles is greater than or equal to 3, any three of the marker fitting circles are obtained;

[0115] It is judged whether the positional relationship of the three marker fitting circles satisfies a preset condition. If yes, it is judged that the operator looks at the screen, otherwise, it is judged that the operator does not look at the screen.

[0116] In an optional embodiment, the judging whether the positional relationship of the three marker fitting circles meets the preset condition includes:

[0117] According to the three marker fitting circles, the respective corresponding center positions and radii are obtained;

[0118] According to the three center positions and the center position relationship, the center distance of each two marker fitting circles is determined;

[0119] When two of the center distances are equal, and the difference between the three radii is less than a preset value, it is judged that the operator is watching the screen;

[0120] Otherwise, it is judged that the operator is not watching the screen.

[0121] Specifically, due to angle deviation and other reasons, the actual obtained marker fitting circles are not necessarily three, and can be less than three. When the operator realizes leaving the screen, due to the occlusion of the marker points caused by the rotation of the face, or the area of the marker points in the image changes, the obtained marker fitting circles that meet the requirements are less than three, and it is further judged that the operator is not watching the screen.

[0122] Further, the eyeglass region can have contour features that are the same as or similar to the shape and size of the marker points, so that the obtained marker fitting circles are greater than three. Therefore, when the number of marker fitting circles is greater than or equal to three, any three marker fitting circles are selected from all the marker fitting circles to judge whether the operator is watching the screen. When it is judged that the operator is not watching the screen, three marker fitting circles are selected again, and the above judgment is performed again. In this way, the three marker fitting circles in all the marker fitting circles are traversed until the judgment result is that the operator is watching the screen, and the traversal is stopped. If all the combinations of the three marker fitting circles judge that the operator is not watching the screen, it is finally determined that the operator is not watching the screen.

[0123] Specifically, the positions and radii of the centers of any three marker fitting circles are obtained, the distance between the centers of two marker fitting circles is calculated according to the positional relationship, the values of the three center distances can be obtained, and it is judged whether the radii of the three marker fitting circles are similar. When the error is less than a set error value, it is judged that the radii of the three marker fitting circles are similar, and the three marker fitting circles are determined as marker points, such as Figure 2As shown, the three circles in the 3D glasses are the mark points of the 3D glasses, since the mark points in the 3D glasses are arranged in an isosceles triangle, that is, the three circle centers formed by the three mark points have two equal lengths, therefore, when the two circle center distances are equal, it can be determined that the operator is viewing the screen, thereby judging that the operator is viewing the screen, if all the three mark fitting circles cannot satisfy the above two conditions at the same time, it is determined that the operator is not viewing the screen.

[0124] In the optional embodiment, when the number of mark fitting circles is greater than or equal to 3, the position relationship and the radius of the three mark fitting circles are used to judge whether the operator is viewing the screen, only when the position relationship and the radius all satisfy the conditions, it is proved that the implementation of the operator is perpendicular to the screen, thereby judging that the operator is viewing the screen, the line of sight attention point of the operator can be accurately judged through the number and position relationship of the mark fitting circles, and then whether the screen is focused is used as the condition of whether to perform the master-slave operation, which can effectively reduce the misoperation caused by the operator not viewing the screen, and improve the safety of the operation.

[0125] The electronic device provided by the embodiment of the application comprises a memory and a processor.

[0126] The memory is used for storing a computer program.

[0127] The processor is used for implementing the surgical system master-slave operation judgment method when the computer program is executed.

[0128] The electronic device provided by the embodiment of the application has the similar technical effects of the surgical system master-slave operation judgment method, and details are not repeated here.

[0129] The computer readable storage medium provided by the embodiment of the application has the similar technical effects of the surgical system master-slave operation judgment method, and details are not repeated here.

[0130] The computer readable storage medium provided by the embodiment of the application has the similar technical effects of the surgical system master-slave operation judgment method, and details are not repeated here.

[0131] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing relevant hardware, and the program can be stored in a computer readable storage medium. When the program is executed, the program can include the processes of the above-mentioned embodiment methods. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM), a random access memory (RAM), or the like. In this application, the units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or they can be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment of the present application. In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically independently, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0132] Although the present application is disclosed as above, the protection scope of the present application is not limited to this. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the present application, and these changes and modifications will fall within the protection scope of the present application.

Claims

1. A surgical system master-slave operation judgment method characterized by comprising: The method comprises the following steps: acquiring a face image of an operator, wherein three mark points are arranged on glasses worn by the operator, the three mark points are arranged in an isosceles triangle, and the three mark points are circular marks; performing feature recognition on the face image to obtain a glasses region; obtaining, by edge detection, a plurality of edge contours of the glasses region according to the glasses region; screening all the edge contours to obtain mark edge contours; obtaining a contour fitting circle according to the mark edge contours by circle fitting; obtaining radii of the contour fitting circles according to the contour fitting circles, and screening the contour fitting circles according to all the radii and a preset radius threshold to obtain mark fitting circles; judging whether the operator looks at a screen according to the mark fitting circles; when the operator looks at the screen and a double-click master hand signal of the operator is acquired, performing master-slave operation, otherwise, stopping the master-slave operation.

2. The surgical system master-slave operation judgment method according to claim 1, characterized in that, The feature recognition on the face image to obtain a glasses region comprises the following steps: obtaining pixel values of each pixel point of the face image according to the face image; performing binaryzation processing on the face image according to a preset binaryzation threshold to obtain a binary face image; obtaining the glasses region by a template matching algorithm according to the binary face image and a preset glasses template image.

3. The surgical system master-slave operation judgment method according to claim 2, characterized in that, The obtaining of the glasses region by the template matching algorithm according to the binary face image and the preset glasses template image comprises the following steps: determining the size of the preset glasses template image as a matching window size, and setting a step length as 1; slidingly selecting a plurality of matching regions from the face image according to the matching window and the step length; obtaining a matching coefficient of each matching region by a matching relationship according to each matching region and the preset glasses template; determining the matching region with a matching coefficient less than a preset matching threshold as the glasses region.

4. The surgical system master-slave operation judgment method according to claim 1, characterized in that, The screening of all the edge contours to obtain mark edge contours comprises the following steps: determining the number of pixel points in the edge contour according to the edge contour; determining an edge contour area according to the number of pixel points; determining a median edge contour area according to all the edge contour areas; screening the edge contour according to the median edge contour area to obtain the mark edge contour.

5. The surgical system master-slave operation judgment method according to claim 1, characterized in that, The obtaining of a contour fitting circle according to the mark edge contour by circle fitting comprises the following steps: obtaining edge pixel points corresponding to the mark edge contour according to each mark edge contour; obtaining the contour fitting circle corresponding to the mark edge contour by circle fitting according to the edge pixel points.

6. The surgical system master-slave operation judgment method according to claim 1, characterized in that, The screening of the contour fitting circles according to all the radii and a preset radius threshold to obtain mark fitting circles comprises the following steps: obtaining a radius difference of each contour fitting circle according to each radius, the radius threshold and a radius deviation relationship; determining the contour fitting circle as a mark fitting circle when the radius difference is less than a preset difference value; the radius deviation relationship satisfies: ； wherein R i is the radius difference of the i-th profile-fitted circle, r i is the radius of the i-th profile-fitted circle, and d is a radius threshold.

7. The surgical system master-slave operation judgment method according to claim 1, characterized in that, The judging of whether the operator looks at the screen according to the mark fitting circles comprises the following steps: When the number of the mark fitting circles is less than 3, it is determined that the operator does not watch the screen; When the number of the mark fitting circles is greater than or equal to 3, any three of the mark fitting circles are obtained; It is determined whether the positional relationship of the three mark fitting circles meets a preset condition, if yes, it is determined that the operator watches the screen, otherwise, it is determined that the operator does not watch the screen.

8. The surgical system master-slave operation judgment method according to claim 7, characterized in that, The determination of whether the positional relationship of the three mark fitting circles meets the preset condition, if yes, it is determined that the operator watches the screen, otherwise, it is determined that the operator does not watch the screen, comprises: According to the three mark fitting circles, the corresponding center positions and radii of the circles are obtained; According to the three center positions and the center position relationship, the center distance of each two mark fitting circles is determined; When two of the center distances are equal, and the difference of the three radii is less than a preset value, it is determined that the operator watches the screen; Otherwise, it is determined that the operator does not watch the screen.

9. An electronic device, comprising: comprise a memory and a processor; The memory is used to store a computer program; The processor is used to implement the surgical system master-slave operation judgment method according to any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The storage medium has a computer program stored thereon, and when the computer program is executed by the processor, the surgical system master-slave operation judgment method according to any one of claims 1 to 8 is implemented.

Images