Positioning detection method, device, system, medium and product for medical robot

Abstract

The application discloses a positioning detection method, device and system of a medical robot, a medium and a product. The method comprises the following steps: in response to adjustment of an end effector of the medical robot to a first calibration pose, acquiring a first end image containing the end effector; acquiring a second end image containing the end effector under the first calibration pose, wherein the second end image represents reference visual information in a robot calibration scene; determining a first image similarity according to the first end image and the second end image; and determining an accuracy detection result of end positioning of the medical robot according to the first image similarity. The embodiment solves the problem of feedback delay of traditional positioning detection methods, and improves the completion quality and operation safety of the medical robot in implementing a work task.

Description

Technical Field

[0001] This invention relates to the field of medical robot technology, and in particular to a positioning and detection method, device, system, medium, and product for a medical robot. Background Technology

[0002] Medical robots are high-end medical equipment that deeply integrates multiple disciplines such as medical imaging technology, bio-terminal engineering, and communication technology. Through automated, precise, and intelligent mechanical structures and control systems, they assist or replace medical personnel in completing various tasks. The positioning accuracy of the end effector of a medical robot is a key factor in ensuring the effectiveness and safety of the task.

[0003] Currently, most methods for detecting positioning accuracy rely on external high-precision measurement equipment such as laser trackers and optical positioning. These methods not only have prominent problems such as high cost and large equipment size, but their cumbersome calculation process also makes it difficult to achieve real-time feedback on positioning accuracy.

[0004] Especially in tasks requiring extremely high positioning accuracy, such as those targeting the temporal bone region including the cochlea and semicircular canals, delayed feedback in positioning accuracy can not only significantly reduce the quality of task completion but may also cause irreversible mechanical damage to the surrounding structures of the operating area, thus severely restricting the large-scale deployment of medical robots. Summary of the Invention

[0005] This invention provides a positioning and detection method, device, system, medium, and product for medical robots, which solves the problem of feedback delay in traditional positioning and detection methods and improves the quality of medical robots in performing tasks and operational safety.

[0006] According to one embodiment of the present invention, a positioning and detection method for a medical robot is provided, the method comprising: In response to the end effector of the medical robot adjusting to the first calibration pose, a first end image containing the end effector is acquired; Acquire a second end effector image containing the end effector under the first calibration pose, wherein the second end effector image represents the reference visual information under the robot calibration scene; The first image similarity is determined based on the first end image and the second end image; Based on the first image similarity, the accuracy detection result of the end-effector positioning of the medical robot is determined.

[0007] According to another embodiment of the present invention, a positioning and detection device for a medical robot is provided, the device comprising: The first end-effector image acquisition module is used to acquire a first end-effector image containing the end-effector in response to the end-effector of the medical robot adjusting to a first calibration pose; The second end-effector image acquisition module is used to acquire a second end-effector image containing the end effector under the first calibration pose, wherein the second end-effector image represents the reference visual information under the robot calibration scenario. An image similarity determination module is used to determine a first image similarity based on the first end image and the second end image; The accuracy detection result determination module is used to determine the accuracy detection result of the end-effector positioning of the medical robot based on the similarity of the first image.

[0008] According to another embodiment of the present invention, a positioning and detection system for a medical robot is provided. The positioning and detection system includes: a medical robot, an image acquisition device, and a terminal device, wherein the medical robot and the image acquisition device are respectively communicatively connected to the terminal device. The medical robot is used to perform tasks via an end effector. The image acquisition device is used to acquire an end image including the end effector; The terminal device is used to execute the positioning and detection method of the medical robot according to any embodiment of the present invention.

[0009] According to another embodiment of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions, the computer instructions being configured to cause a processor to execute and implement the positioning and detection method of the medical robot according to any embodiment of the present invention.

[0010] According to another embodiment of the present invention, a computer program product is provided, including a computer program that, when executed by a processor, implements the positioning and detection method of a medical robot according to any embodiment of the present invention.

[0011] The technical solution of this invention, in response to the end effector of a medical robot adjusting to a first calibration pose, acquires a first end effector image including the end effector, acquires a second end effector image of the medical robot in the robot calibration scene and the first calibration pose, and determines the accuracy detection result of the end effector positioning of the medical robot based on the first image similarity between the first end effector image and the second end effector image. This eliminates the need for external high-precision measuring equipment, significantly reducing the hardware cost of positioning detection, avoiding complex calculation processes such as coordinate transformation and point cloud registration between the measuring equipment and the medical robot, greatly reducing the computational load of the positioning detection process, and solving the problem of feedback delay in traditional positioning detection methods, thereby improving the completion quality and operational safety of the medical robot in performing tasks.

[0012] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0013] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 A flowchart illustrating a positioning and detection method for a medical robot according to an embodiment of the present invention; Figure 2 A flowchart illustrating another positioning and detection method for a medical robot provided in one embodiment of the present invention; Figure 3 A flowchart illustrating another positioning and detection method for a medical robot provided in one embodiment of the present invention; Figure 4 This is a schematic diagram of a first end image and a second end image provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of a differential mask image and a Sobel gradient field provided in one embodiment of the present invention; Figure 6 This is a schematic diagram of ORB feature point matching provided in one embodiment of the present invention; Figure 7 This is a schematic diagram of the structure of a positioning and detection device for a medical robot provided in one embodiment of the present invention; Figure 8 A data flow diagram illustrating a specific example of a positioning and detection device for a medical robot provided in an embodiment of the present invention; Figure 9 This is a schematic diagram of the structure of a positioning and detection system for a medical robot provided in one embodiment of the present invention; Figure 10 This is a schematic diagram of the structure of a terminal device provided in one embodiment of the present invention. Detailed Implementation

[0015] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0016] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0017] Figure 1 This is a flowchart illustrating a positioning detection method for a medical robot according to an embodiment of the present invention. This embodiment is applicable to detecting the positioning accuracy of a medical robot. The method can be executed by a positioning detection device for the medical robot, which can be implemented in hardware and / or software and can be configured in a terminal device. Figure 1 As shown, the method includes: S110, In response to the end effector of the medical robot adjusting to the first calibration pose, a first end image including the end effector is acquired.

[0018] Specifically, the calibration pose represents the spatial position and attitude combination information of the end effector of a medical robot in a preset working coordinate system, including the three-dimensional coordinates and three-axis rotation angles of the end effector. It is the reference pose for calibrating and testing the positioning accuracy of the medical robot.

[0019] For example, the first calibration pose is pre-configured by the motion control module of the end effector or set by the user via a host computer, but is not limited to the given example.

[0020] In an optional embodiment, the method further includes: in response to detecting a reset command, obtaining the current end effector pose of the medical robot and obtaining a first calibration pose in the reset command; if the current end effector pose is different from the first calibration pose, controlling the end effector of the medical robot to adjust to the first calibration pose.

[0021] Specifically, the reset command is a control command used to trigger the end effector to return from the current arbitrary pose to the first calibrated pose, so as to provide a unified and standard attitude reference for positioning accuracy detection.

[0022] For example, the reset command may be generated in response to physical button operation, touch operation, reset voice or reset gesture, or it may be automatically generated according to a preset detection period, such as obtaining the interval between the current time and the previous detection time. When the interval is equal to the preset detection period, a reset command is generated, but it is not limited to the example given above.

[0023] Specifically, this embodiment applies to positioning detection scenarios before and during the execution of tasks by medical robots. Before the task begins, positioning accuracy detection confirms whether the robot's positioning accuracy meets the operational requirements, preventing deviations in positioning accuracy from affecting the task's effectiveness. During the task, positioning accuracy detection can be performed before critical operations and / or when the robot reaches key pose nodes, ensuring the continuity and safety of the task execution process.

[0024] Specifically, the end-effector image represents visual image data captured by an image acquisition device that includes at least the end effector of the medical robot. It is the core data carrier for subsequent image similarity calculation and positioning accuracy determination. Its pixel distribution and grayscale characteristics reflect the spatial position and attitude state of the end effector.

[0025] For example, the image acquisition device for the first end image can be a cone-beam computed tomography (CBCT) device, a medical X-ray machine, a magnetic resonance imaging device, a computed tomography device, and a depth camera, or it can be an image acquisition device adapted to the operation type of the medical robot.

[0026] In one optional embodiment, the imaging parameters of the image acquisition device meet the positioning accuracy detection conditions. Exemplary positioning accuracy detection conditions include spatial resolution meeting the millimeter-level accuracy requirements of positioning accuracy detection, imaging field of view parameters meeting the end-effector complete coverage requirements of positioning accuracy detection, single imaging time being less than the minimum time-sensitive threshold for positioning accuracy detection, parameter fluctuation values ​​under optical interference being lower than the preset fluctuation threshold for positioning accuracy detection, etc., but are not limited to the given examples.

[0027] The advantage of this setting is that it ensures that the end-effector image can clearly present the detailed features and position information of the end effector, avoiding inaccurate positioning accuracy detection due to poor imaging parameters. This ensures that the accuracy and stability of positioning accuracy detection can meet the detection needs of various operating scenarios.

[0028] In one specific embodiment, the image acquisition device is a CBCT device. The CBCT device emits a cone-shaped beam of X-rays from an X-ray source. After being attenuated by the object being measured, the projection data is received by a flat panel detector, and then a three-dimensional tomographic image is generated through a reconstruction algorithm. It possesses advantages such as sub-millimeter resolution, fast imaging speed, low radiation dose, and small size, making it suitable for medical scenarios. It can accurately and quickly capture subtle visual information of the object being measured and collaborate with medical robots to complete tasks, such as fine-grained tasks like cochlear implantation, dental implantation, and skeletal implantation.

[0029] S120. Obtain a second end effector image containing the end effector under the first calibration pose.

[0030] In this embodiment, the second end-effector image represents the reference visual information in the robot calibration scene. The robot calibration scene represents the scene in which the medical robot achieves precise registration of the end-effector coordinate system with the world coordinate system and the image coordinate system in a preset calibration environment through calibration components, such as calibration boards and calibration balls, and locks all control parameters. This scene is free from external interference factors, such as vibration and changes in illumination. Therefore, the second end-effector image has unique reference and stability.

[0031] Specifically, the imaging parameters and preprocessing procedures corresponding to the first and second end images are exactly the same. The consistency of the imaging parameters ensures the comparability of the two end images in the imaging domain and feature domain, while the consistency of the preprocessing procedures ensures the comparability of the two end images in the pixel domain and spatial domain.

[0032] For example, the preprocessing process includes, but is not limited to, pixel inversion, intensity normalization, region of interest (ROI) extraction, contrast enhancement, and filtering denoising. Pixel inversion converts the grayscale value of each pixel in the image into a complementary value within its corresponding grayscale range, highlighting target areas with lower grayscale values ​​and reducing background interference. Intensity normalization maps the grayscale intensity values ​​of image pixels to a preset grayscale range, such as [0, 255], through linear or nonlinear transformations, eliminating the influence of pixel grayscale value fluctuations under different imaging conditions. Region of interest (ROI) extraction refers to segmenting and extracting the region containing the target image from the complete image. Its principle is to filter and retain valid target pixels while removing irrelevant background pixels. Points can reduce the interference of invalid data on subsequent calculations, further improving the detection efficiency of positioning accuracy; contrast enhancement is achieved by stretching the dynamic range of pixel grayscale values ​​or adjusting the local grayscale distribution, thereby enhancing the distinguishability between the object area and the background area, and between target details and the overall contour, to overcome the imaging defect of blurred image details. For example, the contrast enhancement step is implemented using a limited contrast adaptive histogram equalization algorithm; filtering and denoising is achieved by performing convolution operations on image pixels using a preset filtering kernel to smooth grayscale abrupt changes caused by noise and restore the true features of the image. For example, the preset filtering kernel can be a median filtering kernel.

[0033] This section provides only illustrative examples of the preprocessing steps in the preprocessing workflow and does not limit them.

[0034] S130. Determine the first image similarity based on the first end image and the second end image.

[0035] In an optional embodiment, determining the first image similarity based on the first end image and the second end image includes: performing histogram matching and image registration processing on the first end image and the second end image in sequence, and determining the first image similarity based on the processed first end image and second end image.

[0036] Specifically, histogram matching is used to adjust the intensity distribution of two images to ensure they have the same mean and variance, thus eliminating interference from factors such as uneven lighting, differences in device exposure parameters, or surface reflections on the image similarity. Image registration is used to perform rigid transformations, such as translation, rotation, and scaling, on the first end image after histogram matching to align the first and second end images in pixel space, thereby eliminating geometric misalignment caused by positioning errors of the end effector.

[0037] Specifically, image similarity represents the similarity information of at least one image feature. For example, image features can be texture features, shape features, convolution features, spatial features, image embedding vectors, etc., and similarity information can be Euclidean distance, cosine similarity, Manhattan distance, Pearson correlation coefficient, KL divergence, etc., but is not limited to the examples given above.

[0038] S140. Based on the first image similarity, determine the accuracy detection result of the end-effector positioning of the medical robot.

[0039] Specifically, image similarity represents the degree of similarity between the first end-effector image and the second end-effector image at the image feature level, and accuracy detection result represents the judgment information on whether the current actual pose of the end-effector of the medical robot meets the positioning accuracy requirements relative to the first calibration pose.

[0040] The numerical range of the first image similarity is [0,1]. The closer the value is to 1, the higher the feature consistency between the two end images and the smaller the end pose deviation. Conversely, the closer the value is to 0, the greater the feature difference between the two end images and the greater the end pose deviation.

[0041] For example, the accuracy test result is at least one of the following: positioning accuracy score, positioning accuracy level, and positioning accuracy compliance conclusion.

[0042] In an optional embodiment, determining the accuracy detection result of the end-effector positioning of the medical robot based on the first image similarity includes: using the first image similarity as the positioning accuracy score of the end-effector positioning of the medical robot.

[0043] In another optional embodiment, determining the accuracy detection result of the end-effector positioning of the medical robot based on the first image similarity includes: comparing the first image similarity with a similarity threshold; if the first image similarity is less than or equal to the similarity threshold, setting the accuracy detection result of the end-effector positioning of the medical robot as "positioning accuracy not met"; if the first image similarity is greater than the similarity threshold, setting the accuracy detection result of the end-effector positioning of the medical robot as "positioning accuracy met".

[0044] In another optional embodiment, determining the accuracy detection result of the end-effector positioning of the medical robot based on the first image similarity includes: determining the positioning accuracy level of the end-effector positioning of the medical robot based on the similarity interval to which the first image similarity belongs.

[0045] The similarity interval refers to a continuous or discrete interval divided according to image similarity, which is used to map and characterize the pose offset of the end-point localization. Each similarity interval corresponds to a unique localization accuracy level. The similarity intervals do not overlap and cover the range of similarity values.

[0046] For example, the similarity interval [0.9,1] corresponds to the first precision level, the similarity interval [0.5,0.9] corresponds to the second precision level, and the similarity interval [0,0.5] corresponds to the third precision level, wherein the degree of offset of the end positioning represented by the first precision level, the second precision level, and the third precision level increases sequentially.

[0047] Based on the above embodiments, optionally, the method further includes: in response to the accuracy detection result indicating that the positioning accuracy is not up to standard, if the accuracy detection result meets a first accuracy condition, outputting a warning message indicating that the positioning accuracy is not up to standard based on the accuracy detection result; and if the accuracy detection result meets a second accuracy condition, outputting an alarm message indicating that the positioning accuracy is not up to standard based on the accuracy detection result.

[0048] Specifically, the accuracy detection result indicates that the positioning accuracy is not up to standard, meaning that the end effector pose deviation of the medical robot exceeds the preset allowable accuracy range. The first accuracy condition represents the high tolerance range of the end effector pose deviation, and the second accuracy condition represents the low tolerance range of the end effector pose deviation. For example, when the accuracy detection result is a positioning accuracy score, the first accuracy condition is [0.5, 0.9], and the second accuracy condition is [0, 0.5]. When the accuracy detection result is a positioning accuracy level, the first accuracy condition is the second accuracy level, and the second accuracy condition is the third accuracy level.

[0049] For example, the output format of warning and alarm information can be at least one of visual output, auditory output, text output, and system pop-up output. Specifically, visual output can be indicator light flashing, interactive interface highlighting, etc.; warning information can be indicator light flashing at low frequency, interactive interface highlighted in yellow; alarm information can be indicator light flashing at high frequency or remaining constantly lit, interactive interface highlighted in red or flashing. When the output format is auditory, warning information can be a low-frequency, low-volume output sound, and alarm information can be a high-frequency, high-volume alarm sound. When the output format is text output, the text style and / or text display position of warning and alarm information are different, such as text style (text color, text size, etc.). When the output format is system pop-up output, warning information can be a non-blocking pop-up, and alarm information can be a blocking pop-up.

[0050] This section only provides an example of the output format for warning and alarm messages, and does not limit them.

[0051] The technical solution of this embodiment, in response to the end effector of the medical robot adjusting to the first calibration pose, acquires a first end effector image including the end effector, acquires a second end effector image of the medical robot in the robot calibration scene and the first calibration pose, and determines the accuracy detection result of the end effector positioning of the medical robot based on the first image similarity between the first end effector image and the second end effector image. It does not rely on external high-precision measuring equipment, which significantly reduces the hardware cost of positioning detection, avoids complex calculation processes such as coordinate transformation and point cloud registration between measuring equipment and medical robot, greatly reduces the amount of calculation in the positioning detection process, solves the problem of feedback delay in traditional positioning detection methods, and thus improves the completion quality and operational safety of the medical robot in performing tasks.

[0052] Figure 2 This is a flowchart of another positioning and detection method for a medical robot provided in one embodiment of the present invention. This embodiment further refines the step of "determining the accuracy detection result of the end effector positioning of the medical robot based on the first image similarity" in the above embodiment. In this embodiment, determining the accuracy detection result of the end effector positioning of the medical robot based on the first image similarity includes: determining the end effector offset relative to the first calibration pose of the end effector based on calibration mapping data and the first image similarity; and determining the accuracy detection result of the end effector positioning of the medical robot based on the end effector offset. Figure 2 As shown, the method includes: S210, In response to the end effector of the medical robot adjusting to the first calibration pose, a first end image including the end effector is acquired.

[0053] S220. Obtain a second end effector image containing the end effector under the first calibration pose.

[0054] S230. Determine the first image similarity based on the first end image and the second end image.

[0055] S210-S230 in this embodiment are the same as those in the above embodiment. Figure 1 The S110-S130 shown are the same or similar, and will not be described again in this embodiment.

[0056] S240. Based on the calibration mapping data and the similarity of the first image, determine the end effector offset relative to the first calibration pose.

[0057] In this embodiment, the calibration mapping data represents the mapping information between image similarity and end-point calibration displacement, and is used to quantitatively convert image-level similarity information into spatial-level displacement information. Specifically, image similarity and end-point calibration displacement are negatively correlated; the greater the image similarity, the smaller the end-point calibration displacement, and vice versa.

[0058] In an optional embodiment, the method further includes: in a robot calibration scenario, acquiring an end-effector image set and an end-effector displacement sequence of the medical robot in multiple second calibration poses; determining at least two end-effector image pairs based on the end-effector image set, and determining the calibration end-effector displacement corresponding to each end-effector image pair based on the end-effector displacement sequence; determining the second image similarity corresponding to each end-effector image pair; and determining calibration mapping data based on each second image similarity and each calibration end-effector displacement.

[0059] Specifically, the first calibration pose can be one of multiple second calibration poses, suitable for high-precision detection scenarios. In one optional embodiment, the first calibration pose is the first second calibration pose, which represents the starting pose of the end effector in the robot calibration scenario. Multiple second calibration poses can also be a series of poses specifically used for calibration, independent of the first calibration pose, but both belong to the same coordinate system, suitable for rapid detection scenarios.

[0060] Specifically, the end effector image set contains end effector images of the end effector in each second calibration pose. The end effector image set may contain two end effector images repeatedly acquired in the first second calibration pose. The end effector displacement sequence contains the end effector displacement corresponding to each second calibration pose, representing the actual displacement information of the end effector from the previous second calibration pose to the current second calibration pose. For example, the end effector displacement can be acquired using devices such as a laser tracker, a coordinate measuring machine, or a high-precision vision positioning system, but is not limited to the given example.

[0061] Specifically, an end image pair can include two adjacent end images in the order of acquisition, which is suitable for continuous calibration scenarios and can reflect the continuity of end displacement changes. It can also include two end images under the first and second calibration poses, which can be used to reflect the fluctuation of image similarity and guide the setting of similarity threshold. It can also include the end image under the first and second calibration poses as well as end images under other second calibration poses besides the first and second calibration poses, which is suitable for discrete calibration scenarios.

[0062] Specifically, the calibrated end-effector displacement represents the actual end-effector displacement that occurs when the end-effector adjusts from one of the second calibration poses corresponding to the end-effector image pair to another.

[0063] The determination method for the second image similarity corresponds to the determination method for the first image similarity. When the calibration mapping data is a mapping function, the calibration mapping function is obtained by fitting each second image similarity and each calibration end displacement. The fitting method includes, but is not limited to, linear fitting or polynomial fitting. When the calibration mapping data is a mapping interval, the end displacement interval is divided according to the positioning accuracy requirements of the medical robot, such as 0-0.3mm, 0.3-0.5mm, 0.5-1.0mm, etc. The second image similarity is classified according to each calibration end displacement, and the similarity value range corresponding to each end displacement interval is statistically analyzed to obtain the calibration mapping interval.

[0064] S250. Based on the end-effector offset, determine the accuracy detection result of the end-effector positioning of the medical robot.

[0065] In an optional embodiment, determining the accuracy detection result of the end-effector positioning of the medical robot based on the end-effector offset includes: normalizing the end-effector offset and using the complement of the normalization result as the positioning accuracy score of the end-effector positioning of the medical robot.

[0066] In another optional embodiment, determining the accuracy detection result of the end-effector positioning of the medical robot based on the end-effector offset includes: comparing the end-effector offset with an offset threshold; if the end-effector offset is greater than or equal to the offset threshold, setting the accuracy detection result of the end-effector positioning of the medical robot as unsatisfactory; if the end-effector offset is less than the offset threshold, setting the accuracy detection result of the end-effector positioning of the medical robot as satisfactory.

[0067] In another optional embodiment, determining the accuracy detection result of the end-effector positioning of the medical robot based on the end-effector offset includes: determining the positioning accuracy level of the end-effector positioning of the medical robot based on the offset range to which the end-effector offset belongs.

[0068] Among them, the offset interval refers to the continuous or discrete intervals divided according to the end offset. Each offset interval corresponds to a unique positioning accuracy level, and the offset intervals do not overlap with each other.

[0069] For example, an end offset less than 0.3mm corresponds to the first accuracy level, an offset range of [0.3mm, 0.5mm] corresponds to the second accuracy level, and an end offset greater than 0.5mm corresponds to the third accuracy level. The degree of end positioning offset represented by the first accuracy level, the second accuracy level, and the third accuracy level increases sequentially.

[0070] Based on the above embodiments, the method may optionally further include: performing recognition processing on the first end image and the second end image according to the registration object to obtain a first object region and a second object region; obtaining a first region image in the first end image corresponding to the first object region, and obtaining a second region image in the second end image corresponding to the second object region; registering the first region image and the second region image to obtain an object registration degree; and determining an accuracy detection report of the end-positioning of the medical robot based on the object registration degree and the accuracy detection result.

[0071] In an alternative embodiment, the registration object is a visual positioning marker or a background reference object.

[0072] The visual positioning marker refers to a visual positioning carrier used to assist image registration and provide clear feature points. For example, the visual positioning marker can be a physical positioning component and / or a visual positioning pattern. The spatial position of the physical positioning component is relatively fixed relative to the end effector. For example, the physical positioning component can be a metal calibration plate with fixed geometric features, a fluorescent marker block, a laser positioning target, or a calibration component with coded information. The visual positioning pattern can be a checkerboard, a circular dot matrix, an Aztec code, or a custom geometric code, but is not limited to the examples given above.

[0073] The background reference object is a static background or background marker in the end-effector image, excluding the end effector and visual positioning markers. It represents objects that exist uniformly in the robot calibration scene and the robot operation scene. The background reference object may include the control table, robot base and anatomical structure, etc., but is not limited to the given example.

[0074] For example, the recognition processing can use semantic recognition algorithms, pixel threshold recognition algorithms, edge detection algorithms, or region growing algorithms, etc. The registration method can be iterative nearest point registration, feature-based SIFT / SURF registration, normalized cross-correlation registration, or iterative nearest contour registration, etc., but is not limited to the example cases given above. The appropriate registration method can be selected according to the registration object and feature complexity.

[0075] Specifically, the object registration accuracy of the visual positioning marker reflects the degree of deformation of the visual positioning marker. A high object registration accuracy indicates a high reliability of the accuracy detection result. A low object registration accuracy indicates that the first end image and the second end image are not effectively aligned. If the accuracy detection result shows that the accuracy meets the standard, it means that the accuracy detection result may be mixed with image features of non-end actuators or that the registration error has masked the actual positioning defect, resulting in an unreliable accuracy detection result. If the accuracy detection result shows that the accuracy does not meet the standard, the validity of the accuracy detection result cannot be determined.

[0076] Specifically, the object registration accuracy of the background reference object reflects the offset information of the background reference object. A high object registration accuracy indicates that the background reference object has not undergone significant offset, and the reliability of the accuracy detection result is high. A low object registration accuracy indicates that the background reference object has undergone significant offset. If the accuracy detection result shows that the accuracy meets the standard, it means that although the background reference object has been offset, the end effector still maintains accurate positioning. If the accuracy detection result shows that the accuracy does not meet the standard, it means that the offset of the background reference object is the key cause of the accuracy failure, interfering with the image features of the end effector or superimposing the offset of the background reference object on the positioning error of the end effector, ultimately causing the accuracy detection result to show that the accuracy does not meet the standard.

[0077] Based on the above embodiments, optionally, the method further includes: if the object registration degree of the visual positioning mark is less than a first registration degree threshold, controlling the medical robot to re-execute the reset command or return to the step of acquiring the first end image containing the end effector; if the object registration degree of the background reference object is less than a second registration degree threshold, outputting information to indicate that the background reference object has an offset, and in response to the positioning detection command, executing the step of acquiring the first end image containing the end effector.

[0078] The first registration threshold and the second registration threshold can be the same or different. The positioning detection command can be generated in response to the positioning detection operation input by the user. After the position adjustment of the background reference object is completed, the accuracy detection process of the end positioning is re-triggered by executing the positioning detection operation.

[0079] The advantage of this setup is that object registration is used to cross-validate the accuracy detection results, which can effectively identify misjudgments in positioning accuracy detection caused by non-positioning errors, reduce the impact of interference factors on the accuracy detection results, ensure that the accuracy detection results can accurately reflect the actual positioning performance of the end effector, and further improve the accuracy of the accuracy detection results.

[0080] The technical solution of this embodiment determines the end effector offset relative to the first calibration pose based on calibration mapping data and first image similarity. Based on the end effector offset, the accuracy detection result of the end effector positioning of the medical robot is determined. This not only retains the sensitivity of the accuracy detection result to image features, but also overcomes the defect of its ambiguous physical meaning. Compared with image similarity, the judgment standard of end effector offset is more objective and standardized, thereby significantly improving the accuracy of the accuracy detection result while ensuring the efficiency of accuracy detection.

[0081] Figure 3This is a flowchart of another positioning and detection method for a medical robot provided in one embodiment of the present invention. This embodiment further refines the step of "determining a first image similarity based on the first end image and the second end image" in the above embodiment. In this embodiment, determining the first image similarity based on the first end image and the second end image includes: determining at least one feature comparison data based on the first end image and the second end image; determining the feature similarity corresponding to each feature comparison data; and determining the first image similarity based on each feature similarity. Figure 3 As shown, the method includes: S310, In response to the end effector of the medical robot adjusting to the first calibration pose, a first end image including the end effector is acquired.

[0082] S320. Obtain a second end effector image containing the end effector under the first calibration pose.

[0083] S310-S320 in this embodiment are the same as those in the above embodiments. Figure 1 The S110-S120 shown are the same or similar, and will not be described again in this embodiment.

[0084] S330. Determine at least one feature comparison data based on the first end image and the second end image.

[0085] Figure 4 This is a schematic diagram of a first end image and a second end image provided according to an embodiment of the present invention. In this embodiment, the feature comparison data represents the comparison information of the image features corresponding to the first end image and the second end image, respectively.

[0086] In one optional embodiment, each of the feature comparison data includes at least one of a differential mask image, a gradient similarity matrix, and a feature point offset sequence. The differential mask image represents the comparison information of image pixel values ​​between the first end image and the second end image; the gradient similarity matrix represents the comparison information of the gradient fields of the first end image and the second end image; and the feature point offset sequence represents the distance information between feature point pairs in the first end image and the second end image based on local invariant feature matching.

[0087] In an optional embodiment, the feature comparison data includes a differential mask image. Determining at least one feature comparison data based on the first end image and the second end image includes: performing differential processing on the first end image and the second end image to obtain a differential mask image. Specifically, the differential mask image represents the absolute value of the pixel value difference between corresponding pixels in the first end image and the second end image, reflecting the intensity difference distribution between the first end image and the second end image.

[0088] In an optional embodiment, the feature comparison data includes a gradient similarity matrix, and determining at least one feature comparison data based on the first end image and the second end image includes: performing gradient calculations on the first end image and the second end image respectively to obtain a first gradient image and a second gradient image; and determining a gradient similarity matrix based on the first gradient image and the second gradient image.

[0089] Specifically, the matrix elements in the gradient image represent the gradient magnitude of corresponding pixels in the first and second end images, the matrix elements in the gradient similarity matrix represent the gradient similarity of corresponding pixels in the first and second end images, and the gradient field can be a Sobel gradient field, a Prewitt gradient field, or a Canny gradient field, but is not limited to the given example.

[0090] Figure 5 This is a schematic diagram of a differential mask image and a Sobel gradient field provided in an embodiment of the present invention. Specifically, the differential mask image is obtained by... Figure 4 The Sobel gradient field is obtained by subtracting the first and second end images in the image. Figure 4 The second gradient image is obtained by performing Sobel gradient calculation on the second end image in the image.

[0091] In an optional embodiment, the feature comparison data includes a feature point offset sequence. Determining at least one feature comparison data based on the first end image and the second end image includes: performing feature extraction on the first end image and the second end image respectively to obtain a first feature matrix and a second feature matrix; matching the feature points in the first feature matrix and the second feature matrix to obtain at least one feature point pair; and determining the feature point offset sequence based on the feature distance corresponding to each feature point pair.

[0092] Specifically, the matrix elements in the feature matrix represent the pixel coordinates and descriptors of the feature points. For example, feature extraction algorithms include, but are not limited to, the ORB algorithm or the SIFT algorithm.

[0093] Based on the above embodiments, optionally, determining the feature point offset sequence according to the feature distance corresponding to each feature point pair includes: for each feature point pair, deleting the feature point pair if the feature distance corresponding to the feature point pair is greater than a preset distance threshold; and constructing a feature point offset sequence according to the feature distance corresponding to the remaining at least one feature point pair.

[0094] For example, the preset distance threshold can be 0.5mm, but it is not limited to the given example.

[0095] The advantage of this setting is that it retains only feature point pairs that conform to the rigid body transformation law and eliminates mismatched feature point pairs, thereby improving the accuracy of feature similarity.

[0096] Figure 6 This is a schematic diagram illustrating ORB feature point matching according to an embodiment of the present invention. Figure 6 by Figure 4 Taking the first and second end images as examples, points A and a represent two feature points that were successfully matched based on the ORB descriptor, and points B and b represent two other feature points that were successfully matched based on the ORB descriptor. The line connecting the two matched feature points represents the registration effect of the feature points, and the length of the line represents the pixel displacement of the two feature points in the two end images. The shorter the line, the smaller the alignment deviation of the feature points and the higher the local registration accuracy. The more parallel the multiple lines are, the better the global consistency.

[0097] Based on the above embodiments, optionally, before determining the feature similarity corresponding to each feature comparison data, the method further includes: determining a quality factor based on at least one feature comparison data; and if the quality factor is less than a preset quality threshold, controlling the medical robot to re-execute the reset command or return to the step of obtaining a first end image containing the end effector.

[0098] Specifically, the quality factor is used to evaluate the reliability of the first end image, and the quality factor represents the quality level of the first end image in at least one image feature dimension.

[0099] In one specific embodiment, a quality feature value corresponding to each feature comparison data is determined, and a quality factor is determined based on each of the quality feature values. For example, each of the quality feature values ​​includes at least one of the following: noise feature value corresponding to the differential mask image, texture sharpness corresponding to the gradient similarity matrix, and matching confidence corresponding to the feature point offset sequence.

[0100] Here, the noise feature value represents the sharpness of the first end image. The larger the noise feature value, the smaller the image noise and the clearer the image. Conversely, the smaller the noise feature value, the larger the image noise and the less clear the image. For example, the difference mask image is smoothed to obtain a smooth difference matrix. The residual matrix corresponding to the difference mask image and the smooth difference matrix is ​​calculated. The matrix variance of the residual matrix is ​​used as the noise feature value corresponding to the difference mask image.

[0101] Texture sharpness represents the richness of image texture and the sharpness of edges. For example, the normalized histogram corresponding to the second gradient image is determined, the entropy value of the normalized histogram is calculated, and the ratio of the entropy value to the entropy threshold is used as the texture sharpness corresponding to the gradient similarity matrix.

[0102] In this context, matching confidence represents the stability of feature point matching. Higher matching confidence indicates consistent feature point offset patterns, while lower confidence indicates a large number of mismatches. For example, the median feature distance in the feature point offset sequence is obtained, the number of samples whose feature distances deviate from a preset threshold of the median feature distance is counted, the ratio between this number of samples and the total number of samples corresponding to the feature point offset sequence is calculated, and the complement of this ratio is used as the matching confidence corresponding to the feature point offset sequence.

[0103] For example, determining a quality factor based on each of the quality feature values ​​includes: performing a weighted summation on each of the quality feature values ​​to obtain the quality factor, or obtaining the quality judgment conditions corresponding to each of the quality feature values, and using the proportion of quality feature values ​​that satisfy the quality judgment conditions or the lowest quality feature value as the quality factor.

[0104] This embodiment introduces a quality factor mechanism, which can effectively identify problems such as image blurring and motion artifacts caused by slight background shaking, equipment vibration, or sudden changes in lighting. It ensures the accuracy and reliability of subsequent precision detection from the data source and avoids low-quality end images interfering with the determination of precision detection.

[0105] S340. Determine the feature similarity corresponding to each feature comparison data, and determine the first image similarity based on at least one feature similarity.

[0106] In an optional embodiment, the feature comparison data includes a differential mask image, and determining the feature similarity corresponding to each feature comparison data includes: counting the number of pixels in the differential mask image that satisfy the differential value range; and determining the feature similarity corresponding to the differential mask image based on the number of pixels and the number of pixels in the image.

[0107] Here, the image pixel count represents the total number of pixels in the two end images, which can be determined by the spatial resolution of the image acquisition device. Specifically, the ratio between the pixel count and the image pixel count is used as the difference region ratio, and the unit complement of the difference region ratio is used as the feature similarity corresponding to the difference mask image.

[0108] In another optional embodiment, determining the feature similarity corresponding to each feature comparison data includes: determining the difference mean and difference standard deviation corresponding to the difference mask image; determining the difference value range corresponding to the difference mask image based on the difference mean and the difference standard deviation; and determining the feature similarity corresponding to the difference mask image based on the difference mask image and the difference value range.

[0109] Taking a difference value range greater than the maximum difference threshold as an example, the maximum difference threshold is... It can be represented as: ,in, Represents the difference mean. Indicates a fixed coefficient. Indicates the difference standard deviation. .

[0110] In noisy end images, many noise points may be misidentified as change regions, while in noisy end images, some subtle real changes may be ignored. This embodiment achieves automatic adjustment of the difference value range by setting an adaptive difference value range, making the identification of difference regions more accurate and effectively suppressing the impact of noise and artifacts on feature similarity.

[0111] In another optional embodiment, the feature comparison data includes a gradient similarity matrix, and determining the feature similarity corresponding to each feature comparison data includes: calculating the mean of the gradient similarity matrix to obtain the feature similarity corresponding to the gradient similarity matrix.

[0112] In another optional embodiment, the feature comparison data includes a feature point offset sequence, and determining the feature similarity corresponding to each feature comparison data includes: calculating the mean of the feature point offset sequence, and performing normalization and unit complement calculation on the solution results to obtain the feature similarity corresponding to the feature point offset sequence.

[0113] In another optional embodiment, determining the first image similarity based on the similarity of each feature includes: obtaining the feature weights corresponding to each feature comparison data, and performing a weighted sum based on the similarity of each feature and the feature weights to obtain the first image similarity.

[0114] For example, the first image similarity Satisfy the following formula: in, , , This represents the feature similarity corresponding to the difference mask image, gradient similarity matrix, and feature point offset sequence, respectively. , , This represents the feature weights corresponding to the difference mask image, gradient similarity matrix, and feature point offset sequence, respectively. .

[0115] In one optional embodiment, the feature weight corresponding to the differential mask image is the largest. Specifically, differential features can intuitively reflect subtle changes at the pixel level and have high sensitivity to detect sub-millimeter-level end drift, collision, or loosening. This meets the accuracy detection requirements of high-precision operation scenarios, ensuring that differential features dominate the final judgment, thereby improving the accuracy of precision detection.

[0116] S350. Based on the first image similarity, determine the accuracy detection result of the end-effector positioning of the medical robot.

[0117] S350 in this embodiment is the same as in the above embodiment. Figure 1 The S140 shown is the same as or similar to that in the above embodiments. Figure 2 The S240-S250 shown are the same or similar, and will not be described again in this embodiment.

[0118] The technical solution of this embodiment determines a differential mask image, a gradient similarity matrix, and a feature point offset sequence based on the first end image and the second end image. It then determines the feature similarity corresponding to each feature comparison data and determines the first image similarity based on multiple feature similarities. The differential mask image is highly sensitive to pixel-level subtle displacements and surface grayscale changes. The gradient similarity matrix has strong recognition capabilities for deformation and rotational deviations of edge structures. The feature point offset sequence provides a more intuitive representation of overall rigid transformations and large-scale displacements. It perceives the accuracy decrease caused by positioning errors at the pixel, structural, and overall levels, solving the problem that single image features can easily lead to distortion of accuracy detection results, and further improving the accuracy and reliability of accuracy detection results.

[0119] The following are embodiments of the positioning and detection device for medical robots provided in this invention. This device and the positioning and detection method for medical robots described in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the positioning and detection device for medical robots, please refer to the content of the positioning and detection method for medical robots in the above embodiments.

[0120] Figure 7 This is a schematic diagram of the structure of a positioning and detection device for a medical robot according to an embodiment of the present invention. Figure 7 As shown, the device includes: a first end image acquisition module 410, a second end image acquisition module 420, an image similarity determination module 430, and a precision detection result determination module 440.

[0121] The first end-effector image acquisition module 410 is used to acquire a first end-effector image containing the end-effector in response to the end-effector of the medical robot being adjusted to a first calibration pose. The second end-effector image acquisition module 420 is used to acquire a second end-effector image containing the end effector under the first calibration pose, wherein the second end-effector image represents the reference visual information under the robot calibration scenario. The image similarity determination module 430 is used to determine the first image similarity based on the first end image and the second end image; The accuracy detection result determination module 440 is used to determine the accuracy detection result of the end-effector positioning of the medical robot based on the first image similarity.

[0122] The technical solution of this embodiment, in response to the end effector of the medical robot adjusting to the first calibration pose, acquires a first end effector image including the end effector, acquires a second end effector image of the medical robot in the robot calibration scene and the first calibration pose, and determines the accuracy detection result of the end effector positioning of the medical robot based on the first image similarity between the first end effector image and the second end effector image. It does not rely on external high-precision measuring equipment, which significantly reduces the hardware cost of positioning detection, avoids complex calculation processes such as coordinate transformation and point cloud registration between measuring equipment and medical robot, greatly reduces the amount of calculation in the positioning detection process, solves the problem of feedback delay in traditional positioning detection methods, and thus improves the completion quality and operational safety of the medical robot in performing tasks.

[0123] In an optional embodiment, the accuracy detection result determination module 440 is specifically used for: Based on the calibration mapping data and the similarity of the first image, the end effector offset relative to the first calibration pose is determined; Based on the end-effector offset, the accuracy detection result of the end-effector positioning of the medical robot is determined; The calibration mapping data represents the mapping information between image similarity and end-point calibration displacement.

[0124] In an optional embodiment, the device further includes: The calibration mapping data determination module is used to acquire the end-effector image set and end-effector displacement sequence of the medical robot in multiple second calibration poses in a robot calibration scenario. Based on the set of end images, at least two pairs of end images are determined, and the calibration end displacement corresponding to each pair of end images is determined based on the end displacement sequence. Determine the similarity between each end image and its corresponding second image; The calibration mapping data is determined based on the similarity of each of the second images and the displacement of each of the calibration ends.

[0125] In an optional embodiment, the image similarity determination module 430 includes: The feature comparison data determination unit is used to determine at least one feature comparison data based on the first end image and the second end image, wherein the feature comparison data characterizes the comparison information of the image features corresponding to the first end image and the second end image respectively. The image similarity determination unit is used to determine the feature similarity corresponding to each feature comparison data, and to determine the first image similarity based on the feature similarity.

[0126] In an optional embodiment, the device further includes: 1 The quality factor determination module is used to determine the quality factor based on at least one feature comparison data before determining the feature similarity corresponding to each feature comparison data. If the quality factor is less than a preset quality threshold, the medical robot is controlled to re-execute the reset command or return to the step of acquiring the first end image containing the end effector.

[0127] In an optional embodiment, the feature comparison data includes a differential mask image and an image similarity determination unit, specifically used for: Determine the mean and standard deviation of the difference corresponding to the difference mask image; The difference value range corresponding to the difference mask image is determined based on the difference mean and the difference standard deviation. Based on the difference mask image and the difference value range, the feature similarity corresponding to the difference mask image is determined.

[0128] In an optional embodiment, the device further includes: The accuracy detection report determination module is used to perform recognition processing on the first end image and the second end image according to the registration object to obtain the first object region and the second object region; Obtain a first region image in the first end image that corresponds to the first object region, and obtain a second region image in the second end image that corresponds to the second object region; The object registration score is obtained by registering the first region image and the second region image; Based on the object registration accuracy and the accuracy detection results, a precision detection report for the end-effector positioning of the medical robot is determined.

[0129] Figure 8 This is a data flow diagram illustrating a specific example of a positioning and detection device for a medical robot according to an embodiment of the present invention. Specifically, the image acquisition module acquires a second end effector image V2, which includes the end effector, acquired by the image acquisition device in a robot calibration scenario under calibration posture, and a first end effector image V1, acquired by the image acquisition device when the end effector is in the same calibration posture in a positioning and detection scenario. The preprocessing module preprocesses the first end effector image V1 and the second end effector image V2 respectively, outputting the preprocessed first end effector image V1 and the second end effector image V2. The feature calculation module then calculates the feature based on the preprocessed first end effector image V1 and the second end effector image V2. Image V1 and the second end image V2 are used to calculate feature similarity in the differential feature dimension, gradient field dimension, and ORB feature dimension, respectively. The fusion and mapping module fuses these three feature similarities to obtain a first image similarity. The end offset corresponding to the first image similarity is determined based on the calibration mapping data. The judgment and reporting module performs threshold judgments on the first image similarity and / or the end offset to obtain the accuracy detection result of the medical robot end-effector positioning. An accuracy detection report is generated based on the accuracy detection result and archived and stored by the data archiving module for traceability analysis. For example, the positioning detection device may also include a quality control module, serving as a judgment module between the feature calculation module and the fusion and mapping module. This module evaluates the quality of the first end image based on the image feature dimension corresponding to the feature calculation module, obtaining a quality factor. If the quality factor is less than a preset quality threshold, the data flow flows from the quality control module to the image acquisition module; if the quality factor is greater than or equal to the preset quality threshold, the data flow flows from the quality control module to the fusion and mapping module. Specifically, the above modules interact using a standard data interface.

[0130] The positioning and detection device for medical robots provided in this embodiment of the invention can execute the positioning and detection method for medical robots provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0131] Figure 9 This is a schematic diagram of the structure of a positioning and detection system for a medical robot according to an embodiment of the present invention, as shown below. Figure 9As shown, the positioning and detection system 500 includes a medical robot 510, an image acquisition device 520, and a terminal device 530. The medical robot 510 and the image acquisition device 520 are respectively communicatively connected to the terminal device 530.

[0132] The medical robot 510 is used to perform tasks through an end effector, and the image acquisition device 520 is used to acquire end images containing the end effector.

[0133] Figure 10 This is a schematic diagram of a terminal device provided according to one embodiment of the present invention. Terminal device 530 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0134] like Figure 10 As shown, the terminal device 530 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor 11. The processor 11 can perform various appropriate actions and processes based on the computer programs stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the terminal device 530. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0135] Multiple components in terminal device 530 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows terminal device 530 to exchange information or data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0136] Processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the positioning and detection method for a medical robot provided in the above embodiments.

[0137] In some embodiments, the positioning and detection method for a medical robot provided in the above embodiments can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the terminal device 530 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the positioning and detection method for a medical robot described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the positioning and detection method for a medical robot by any other suitable means (e.g., by means of firmware).

[0138] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication unit 19, or installed from storage unit 18, or installed from ROM 12. When the computer program is executed by processor 11, it performs the functions defined in the methods of the embodiments of the present invention.

[0139] Various embodiments of the systems and techniques described above herein can be implemented in the following systems or combinations thereof: digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard parts (ASSPs), system-on-chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0140] The computer program for implementing the positioning and detection method of the medical robot of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer program causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer program can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0141] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable storage medium. Examples of machine-readable storage media include, based on an electrical connection of at least one wire, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0142] To provide interaction with a user, the systems and techniques described herein can be implemented on a terminal device having: a display device for displaying information to the user (e.g., a cathode-ray tube (CRT) or liquid crystal display (LCD) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the terminal device. Other types of devices can also provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0143] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0144] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system. It addresses the shortcomings of traditional physical hosts and Virtual Private Server (VPS) services, such as high management difficulty and weak business scalability.

[0145] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0146] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A positioning and detection method for a medical robot, characterized in that, include: In response to the end effector of the medical robot adjusting to the first calibration pose, a first end image containing the end effector is acquired; Acquire a second end effector image containing the end effector under the first calibration pose, wherein the second end effector image represents the reference visual information under the robot calibration scene; The first image similarity is determined based on the first end image and the second end image; Based on the first image similarity, the accuracy detection result of the end-effector positioning of the medical robot is determined.

2. The method according to claim 1, characterized in that, The step of determining the accuracy detection result of the end-effector localization of the medical robot based on the first image similarity includes: Based on the calibration mapping data and the similarity of the first image, the end effector offset relative to the first calibration pose is determined; Based on the end-effector offset, the accuracy detection result of the end-effector positioning of the medical robot is determined; The calibration mapping data represents the mapping information between image similarity and end-point calibration displacement.

3. The method according to claim 2, characterized in that, The method further includes: In the robot calibration scenario, acquire the end-effector image set and end-effector displacement sequence of the medical robot in multiple second calibration poses; Based on the set of end images, at least two pairs of end images are determined, and the calibration end displacement corresponding to each pair of end images is determined based on the end displacement sequence. Determine the similarity between each end image and its corresponding second image; The calibration mapping data is determined based on the similarity of each of the second images and the displacement of each of the calibration ends.

4. The method according to claim 1, characterized in that, The step of determining the first image similarity based on the first end image and the second end image includes: Based on the first end image and the second end image, at least one feature comparison data is determined, wherein the feature comparison data characterizes the comparison information of the image features corresponding to the first end image and the second end image respectively; Determine the feature similarity corresponding to each feature comparison data, and determine the first image similarity based on the feature similarity.

5. The method according to claim 4, characterized in that, Before determining the feature similarity corresponding to each feature comparison data, the method further includes: The quality factor is determined based on at least one feature comparison data. If the quality factor is less than a preset quality threshold, the medical robot is controlled to re-execute the reset command or return to the step of acquiring the first end image containing the end effector.

6. The method according to claim 4, characterized in that, The feature comparison data includes differential mask images, and determining the feature similarity corresponding to each feature comparison data includes: Determine the mean and standard deviation of the difference corresponding to the difference mask image; The difference value range corresponding to the difference mask image is determined based on the difference mean and the difference standard deviation. Based on the difference mask image and the difference value range, the feature similarity corresponding to the difference mask image is determined.

7. The method according to claim 1, characterized in that, The method further includes: Based on the registration object, the first end image and the second end image are respectively identified and processed to obtain the first object region and the second object region; Obtain a first region image in the first end image that corresponds to the first object region, and obtain a second region image in the second end image that corresponds to the second object region; The object registration score is obtained by registering the first region image and the second region image; Based on the object registration accuracy and the accuracy detection results, a precision detection report for the end-effector positioning of the medical robot is determined.

8. A positioning and detection device for a medical robot, characterized in that, include: The first end-effector image acquisition module is used to acquire a first end-effector image containing the end-effector in response to the end-effector of the medical robot adjusting to a first calibration pose; The second end-effector image acquisition module is used to acquire a second end-effector image containing the end effector under the first calibration pose, wherein the second end-effector image represents the reference visual information under the robot calibration scenario. An image similarity determination module is used to determine a first image similarity based on the first end image and the second end image; The accuracy detection result determination module is used to determine the accuracy detection result of the end-effector positioning of the medical robot based on the similarity of the first image.

9. A positioning and detection system for a medical robot, characterized in that, The positioning and detection system includes: a medical robot, an image acquisition device, and a terminal device, wherein the medical robot and the image acquisition device are respectively communicatively connected to the terminal device; The medical robot is used to perform tasks via an end effector. The image acquisition device is used to acquire an end image including the end effector; The terminal device is used to perform the positioning and detection method of the medical robot according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the positioning and detection method of the medical robot according to any one of claims 1-7.

11. A computer program product comprising a computer program that, when executed by a processor, implements the positioning and detection method for a medical robot according to any one of claims 1-7.

Images