Medical image model construction method, structural body model construction method and planning method

Abstract

The invention relates to a construction method of a medical image model, a construction method of a structural body model and a planning method. The method comprises the steps of obtaining a medical imaging equipment image collected by a first camera; the first camera is used for shooting a complete medical imaging equipment image; constructing an aperture model of the medical image equipment according to the image of the medical image equipment, the first coordinate system conversion relation and the second coordinate system conversion relation; the first coordinate system conversion relation is a corresponding relation between a coordinate system of the first camera and a base coordinate system of the mechanical arm; the second coordinate system conversion relation is a corresponding relation between the base coordinate system and the medical image coordinate system. By adopting the method, the suitability of the aperture model of the medical imaging equipment can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of obstacle avoidance, and particularly to a method for constructing a medical image model, a method for constructing a structure model, and a method for planning an obstacle avoidance path. Background Art

[0002] With the development of medical technology, interventional surgical robots have emerged. Interventional surgical robots are usually used in conjunction with medical imaging devices. Through the interventional surgical robot, the patient and the robotic arm of the interventional surgical robot can be moved to a specified position on the medical imaging device to perform subsequent surgical procedures. Therefore, in order to ensure that the robotic arm, the medical imaging device, and the patient do not collide during the movement, it is necessary to plan an obstacle avoidance path.

[0003] However, the current method for planning an obstacle avoidance path has the problem of low adaptability to medical imaging devices. Summary of the Invention

[0004] Based on this, in view of the above technical problems, it is necessary to provide a method for constructing a medical image model, a method for constructing a structure model, and a planning method that can improve the adaptability to medical imaging devices.

[0005] In a first aspect, the present application provides a method for constructing a medical image model. The method is applied to a scanning system, and the scanning system includes a medical imaging device and a robotic arm. The method includes:

[0006] Obtain a medical imaging device image collected by a first camera; the first camera is used to capture a complete medical imaging device image;

[0007] Construct an aperture model of the medical imaging device according to the medical imaging device image, a first coordinate system conversion relationship, and a second coordinate system conversion relationship; the first coordinate system conversion relationship is the corresponding relationship between the coordinate system of the first camera and the base coordinate system of the robotic arm; the second coordinate system conversion relationship is the corresponding relationship between the base coordinate system and the medical imaging coordinate system.

[0008] In one embodiment, the medical imaging device image includes a first depth image, and the first depth image includes a first depth image outside the aperture and a first depth image inside the aperture. The first camera is disposed at the end of the robotic arm. The obtaining of the medical imaging device image collected by the first camera includes:

[0009] Obtain an image of the medical imaging device collected by the first camera at a first position to obtain the first depth image outside the aperture; the first position is within the outer region of the scanning hole of the medical imaging device;

[0010] Obtain the image of the medical imaging device collected by the first camera at the second position to obtain the first depth image inside the aperture; the second position is within the internal area of the scanning hole.

[0011] In one embodiment, constructing the aperture model of the medical imaging device according to the medical imaging device image, the first coordinate system conversion relationship, and the second coordinate system conversion relationship includes:

[0012] Construct an initial model of the medical imaging device according to the initial size of the medical imaging device and the size of the connection structure;

[0013] Construct the inner and outer aperture models of the medical imaging device according to the first depth image;

[0014] Construct the aperture model of the medical imaging device according to the first coordinate system conversion relationship, the second coordinate system conversion relationship, the initial model of the medical imaging device, and the inner and outer aperture models.

[0015] In one embodiment, constructing the aperture model of the medical imaging device according to the first coordinate system conversion relationship, the second coordinate system conversion relationship, the initial model of the medical imaging device, and the inner and outer aperture models includes:

[0016] Perform coordinate system conversion on the inner and outer aperture models according to the first coordinate system conversion relationship to obtain the first converted inner and outer aperture models;

[0017] Perform coordinate system conversion on the first converted inner and outer aperture models according to the second coordinate system conversion relationship to obtain the second converted inner and outer aperture models;

[0018] Obtain the aperture model of the medical imaging device according to the second converted inner and outer aperture models and the initial model of the medical imaging device.

[0019] In one embodiment, constructing the aperture model of the medical imaging device according to the medical imaging device image, the first coordinate system conversion relationship, and the second coordinate system conversion relationship includes:

[0020] Construct the inner and outer aperture models of the medical imaging device according to the first depth image;

[0021] Determine the aperture edge according to the inner and outer aperture models, and perform point cloud expansion according to the aperture edge to obtain the intermediate aperture model of the medical imaging device;

[0022] Construct the aperture model of the medical imaging device according to the first coordinate system conversion relationship, the second coordinate system conversion relationship, and the intermediate aperture model.

[0023] In one embodiment, the medical imaging device image includes a second depth image. Constructing the aperture model of the medical imaging device according to the medical imaging device image, the first coordinate system conversion relationship, and the second coordinate system conversion relationship includes:

[0024] Generating a point cloud image of the medical imaging device based on the second depth image;

[0025] Performing coordinate system conversion on the point cloud image according to the first coordinate system conversion relationship to obtain the point cloud image in the base coordinate system;

[0026] Performing model construction according to the second coordinate system conversion relationship and the point cloud image in the base coordinate system to obtain the aperture model of the medical imaging device.

[0027] In one embodiment, performing model construction according to the second coordinate system conversion relationship and the point cloud image in the base coordinate system to obtain the aperture model of the medical imaging device includes:

[0028] Expanding the point cloud image in the base coordinate system along the central axis of the aperture of the medical imaging device in the direction of entering the aperture to obtain the target point cloud image in the base coordinate system;

[0029] Performing model construction based on the target point cloud image to obtain the constructed point cloud model;

[0030] Performing medical imaging coordinate system conversion on the point cloud model according to the second coordinate system conversion relationship to obtain the aperture model of the medical imaging device.

[0031] In one embodiment, the method further includes:

[0032] Registering the coordinate system of the medical imaging device and the base coordinate system of the robotic arm to obtain the second coordinate system conversion relationship;

[0033] When the position of the robotic arm changes, re-registering the coordinate system of the medical imaging device and the base coordinate system of the robotic arm to obtain a new second coordinate system conversion relationship;

[0034] Determining a correction conversion relationship according to the new second coordinate system conversion relationship and the second coordinate system conversion relationship;

[0035] Correcting the aperture model of the medical imaging device according to the correction conversion relationship to obtain a new aperture model of the medical imaging device.

[0036] Second aspect, the present application provides a method for constructing a structural model, which is applied to a scanning system. The scanning system includes a medical imaging device and a robotic arm. The method includes:

[0037] Obtain a target image captured by a second camera; the target image includes a target area of the subject;

[0038] Control the bed of the medical imaging device to drive the subject to move to the scanning area of the medical imaging device;

[0039] Start the medical imaging device to scan the scanning area to obtain a scanning image of the medical imaging device; the scanning image includes a lesion area of the subject; the lesion area is a partial area within the target area;

[0040] Construct a structural model of the subject according to the target image and the scanning image.

[0041] In one embodiment, the method further includes:

[0042] Detect a binding device of the subject to obtain a detection result;

[0043] The step of obtaining a target image captured by a second camera includes:

[0044] Obtain the target image captured by the second camera when the detection result indicates successful detection.

[0045] In one embodiment, the step of detecting a binding device of the subject to obtain a detection result includes:

[0046] Detect an identifier on the binding device to determine whether the binding device includes a preset number of identifiers;

[0047] If the binding device includes a preset number of identifiers, determine a first identifier and a second identifier on the binding device, and perform detection according to the distance between the first identifier and the second identifier to obtain the detection result;

[0048] If the binding device does not include a preset number of identifiers, determine that the detection result indicates a failed detection.

[0049] Third aspect, the present application provides a method for planning an obstacle avoidance path, which is applied to a scanning system. The scanning system includes a medical imaging device and a robotic arm. The method includes:

[0050] Construct an aperture model of the medical imaging device according to the method described in any item of the first aspect;

[0051] According to the method described in any one of the second aspect, a structural model is constructed;

[0052] According to the aperture model, the structural model, and the dimensions of the robotic arm, an obstacle avoidance path of the robotic arm during the scanning of the medical imaging device is determined.

[0053] In a fourth aspect, the present application provides a device for constructing a medical imaging model, the device comprising:

[0054] A first acquisition module, configured to acquire a medical imaging device image collected by a first camera; the first camera is disposed on the bed of the medical imaging device, or the first camera is disposed at the end of the robotic arm;

[0055] A first construction module, configured to construct an aperture model of the medical imaging device according to the medical imaging device image, a first coordinate system conversion relationship, and a second coordinate system conversion relationship; the first coordinate system conversion relationship is a corresponding relationship between the coordinate system of the first camera and the base coordinate system; the second coordinate system conversion relationship is a corresponding relationship between the base coordinate system and the medical imaging coordinate system.

[0056] In a fifth aspect, the present application provides a device for constructing a structural model, the device comprising:

[0057] A second acquisition module, configured to acquire a point cloud image captured by a second camera; the point cloud image includes a target area of a captured object;

[0058] A third acquisition module, configured to acquire a scan image of the medical imaging device; the scan image includes a lesion area of the captured object; the lesion area is a partial area within the target area;

[0059] A second construction module, configured to construct a structural model of the captured object according to the point cloud image and the scan image.

[0060] In a sixth aspect, the present application provides a device for planning an obstacle avoidance path, the device comprising:

[0061] An aperture model construction module, configured to construct an aperture model of the medical imaging device according to the device described in the fourth aspect;

[0062] A structural model construction module, configured to construct a structural model according to the device described in the fifth aspect;

[0063] An obstacle avoidance path determination module, configured to determine an obstacle avoidance path of the robotic arm during the scanning of the medical imaging device according to the aperture model, the structural model, and the dimensions of the robotic arm.

[0064] In a seventh aspect, the present application further provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the method in any one of the first to third aspects of the above embodiments are implemented.

[0065] In an eighth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, and when the computer program is executed by a processor, the steps of the method in any one of the first to third aspects of the above embodiments are implemented.

[0066] In a ninth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the method in any one of the first to third aspects of the above embodiments are implemented.

[0067] For the above method for constructing a medical image model, method for constructing a structural model, and planning method, an image of a medical imaging device collected by a first camera is obtained; the first camera is used to capture a complete image of the medical imaging device; according to the medical imaging device image, a first coordinate system conversion relationship, and a second coordinate system conversion relationship, an aperture model of the medical imaging device is constructed; the first coordinate system conversion relationship is the corresponding relationship between the coordinate system of the first camera and the base coordinate system; the second coordinate system conversion relationship is the corresponding relationship between the base coordinate system and the medical imaging coordinate system. The present application can construct an aperture model of the medical imaging device in the medical imaging coordinate system according to the medical imaging device image collected by the first camera, the first coordinate system conversion relationship, and the second coordinate system conversion relationship. That is, the aperture model of the medical imaging device in the present application is generated by modeling the CT environment and does not need to be matched with an interventional surgical robot. Therefore, during the planning process of the obstacle avoidance path, the aperture model of the medical imaging device in the present application can be reused, so that the interventional surgical robot can be adapted to medical imaging devices of different types, models, and sizes, and the adaptability of the aperture model of the medical imaging device is relatively high. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments or related technologies. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0069] Figure 1 It is an application environment diagram of the method for constructing a medical image model in an embodiment;

[0070] Figure 2Schematic flowchart of a method for constructing a medical imaging model in an embodiment;

[0071] Figure 3 Schematic structural diagram of a scanning system with a first camera disposed at the end of a robotic arm in an embodiment;

[0072] Figure 4 Schematic diagram of the position of the first camera when acquiring the first depth image inside the acquisition aperture in an embodiment;

[0073] Figure 5 Schematic flowchart of the steps for constructing a first type of medical imaging model in an embodiment;

[0074] Figure 6 Schematic flowchart of the steps for constructing a second type of medical imaging model in an embodiment;

[0075] Figure 7 Schematic structural diagram of a scanning system with a first camera disposed on the bed of a medical imaging device in an embodiment;

[0076] Figure 8 Schematic diagram of the RGB field of view of the first camera in an embodiment;

[0077] Figure 9 Schematic diagram of point cloud expansion in an embodiment;

[0078] Figure 10 Schematic diagram of the position of the second calibration board in an embodiment;

[0079] Figure 11 Schematic flowchart of the steps for generating the first coordinate system conversion relationship in an embodiment;

[0080] Figure 12 Schematic diagram of the application of the second coordinate system conversion relationship in an embodiment;

[0081] Figure 13 Schematic diagram of updating the position of the model in an embodiment;

[0082] Figure 14 Schematic flowchart of a method for constructing a structure model in another embodiment;

[0083] Figure 15 Schematic diagram of a target image in an embodiment;

[0084] Figure 16 Schematic flowchart of obtaining a point cloud model in an embodiment;

[0085] Figure 17 Schematic flowchart of obtaining a scanning model in an embodiment;

[0086] Figure 18 Schematic diagram of model splicing in an embodiment;

[0087] Figure 19 Schematic diagram of a fixed restraint strap in an embodiment;

[0088] Figure 20 Schematic diagram of the detection steps in another embodiment;

[0089] Figure 21 Schematic diagram of the first identifier and the second identifier in an embodiment;

[0090] Figure 22 Structural block diagram of a medical image model construction device in an embodiment;

[0091] Figure 23 Structural block diagram of a structure model construction device in an embodiment;

[0092] Figure 24 Structural block diagram of an obstacle avoidance path planning device in an embodiment;

[0093] Figure 25 Internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0094] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0095] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs; the terms used herein are only for the purpose of describing specific embodiments and are not intended to limit this application; the terms "including" and "having" and any variations thereof in the specification and claims of this application and the above accompanying drawing descriptions are intended to cover non-exclusive inclusion.

[0096] In the description of the embodiments of this application, technical terms such as "first" and "second" are only used to distinguish different objects and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity, specific order or primary and secondary relationship of the indicated technical features. In the description of the embodiments of this application, "a plurality of" means more than two unless otherwise specifically defined.

[0097] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0098] With the development of medical technology, interventional surgical robots have emerged. Interventional surgical robots are usually used in conjunction with CT (Computed Tomography) equipment. Patients generally lie flat or supine on the bed of medical imaging equipment. The interventional surgical robot can autonomously move the patient and the robot's robotic arm to the designated puncture position on the medical imaging equipment to perform subsequent surgical procedures. Since doctors need to observe the puncture depth and puncture position in CT imaging, the robotic arm needs to enter the CT aperture of the medical imaging equipment to perform the puncture operation, and the space in the aperture area is small. Therefore, in order to ensure that the robotic arm, medical imaging equipment, and patients do not collide during the movement, high-precision sensors (such as structured light cameras) are required to accurately model obstacles in the surgical scene in order to plan obstacle avoidance paths.

[0099] However, in the related technology, interventional surgical robots are usually used in conjunction with medical imaging equipment, and the CT aperture model of the medical imaging equipment that is used with the interventional surgical robot is generally inconvenient to obtain, which leads to the low compatibility of the interventional surgical robot and the medical imaging equipment. Therefore, the current obstacle avoidance path planning method has the problem of low compatibility with medical imaging equipment.

[0100] After introducing the background technology of the time-lapse video playback method provided in the embodiment of the present application, the implementation environment involved in the time-lapse video playback method provided in the embodiment of the present application will be briefly described below. The method for constructing a medical imaging model provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. The scanning system includes a medical imaging device and a robotic arm. The medical imaging device includes a bed of the medical imaging device on which a patient can be placed. The scanning system acquires an image of the medical imaging device collected by a first camera, and constructs an aperture model of the medical imaging device according to the image of the medical imaging device, the first coordinate system conversion relationship, and the second coordinate system conversion relationship. Among them, the medical imaging device may include, but is not limited to, CT (Computed Tomography) devices, MR (Magnetic Resonance) devices, DSA (Digital Subtraction Angiography) devices, etc. Correspondingly, the medical imaging model may include, but is not limited to, the model corresponding to the CT device, the model corresponding to the MR device, the model corresponding to the DSA device, etc.

[0101] In one embodiment, as Figure 2 shown, a method for constructing a medical imaging model is provided. Taking the method applied to the Figure 1 scanning system in it as an example for illustration, it includes the following steps:

[0102] S201, acquire an image of the medical imaging device collected by a first camera; the first camera is used to capture a complete image of the medical imaging device.

[0103] Among them, the first camera is used to capture a complete image of the medical imaging device. A complete image of the medical imaging device refers to an image of the medical imaging device captured by the first camera and can ensure that the frame housing of the medical imaging device can be completely displayed in the field of view of the first camera. Optionally, the first camera can be set in a preset area on the bed or the tooling of the medical imaging device. Exemplarily, the first camera can be placed near the central axis of the CT gantry, and after the optical axis of the first camera and the central axis of the CT gantry are approximately at the same height, adjust the position of the first camera after turning on the camera. If the adjusted position of the first camera can ensure that the CT gantry housing can be completely displayed in the field of view of the first camera, then this position is within the preset area; or, the first camera can also be set at the end of the robotic arm, and the position of the robotic arm can be adjusted so that the CT gantry housing can be completely displayed in the field of view of the first camera. The first camera can be a depth camera. The depth camera can not only collect color images but also collect depth images. The image of the medical imaging device can be a collected color image, such as an RGB image; or, the image of the medical imaging device can also be a collected depth image, such as a point cloud image; or, the image of the medical imaging device can also include a color image and a depth image. Of course, the type of the image of the medical imaging device in this application is not limited.

[0104] In an embodiment of the present application, optionally, the scanning system may collect a medical imaging device image through a first camera disposed on the bed of the medical imaging device, so that the scanning system can obtain the medical imaging device image. Alternatively, the scanning system may also collect a medical imaging device image through a first camera disposed at the end of the robotic arm, so that the scanning system can obtain the medical imaging device image.

[0105] S202. Construct an aperture model of the medical imaging device according to the medical imaging device image, the first coordinate system conversion relationship, and the second coordinate system conversion relationship; the first coordinate system conversion relationship is the corresponding relationship between the coordinate system of the first camera and the base coordinate system of the robotic arm; the second coordinate system conversion relationship is the corresponding relationship between the base coordinate system and the medical imaging coordinate system.

[0106] Among them, the first coordinate system conversion relationship is the corresponding relationship between the coordinate system of the first camera and the base coordinate system of the robotic arm, and the coordinate system of the first camera is the coordinate system corresponding to the first depth camera in the first camera. The second coordinate system conversion relationship is the corresponding relationship between the base coordinate system of the robotic arm and the medical imaging coordinate system, and the medical imaging coordinate system is the coordinate system corresponding to the aperture of the medical imaging device. In the medical imaging coordinate system, the coordinate origin is located at the center of the aperture of the medical imaging device. The aperture model of the medical imaging device is a three-dimensional model of the aperture part in the medical imaging device.

[0107] In an embodiment of the present application, the scanning system may construct an aperture model of the medical imaging device according to the medical imaging device image, the first coordinate system conversion relationship, and the second coordinate system conversion relationship. Exemplarily, the scanning system may first construct an initial aperture model in the coordinate system of the first camera according to the medical imaging device image, and then perform coordinate system conversion on the initial aperture model according to the first coordinate system conversion relationship and the second coordinate system conversion relationship to obtain the aperture model of the medical imaging device in the medical imaging coordinate system. Of course, the specific manner of constructing the aperture model of the medical imaging device in the embodiment of the present application is not limited.

[0108] In the above method for constructing a medical imaging model, an image of a medical imaging device collected by a first camera is obtained; the first camera is used to capture a complete image of the medical imaging device; according to the medical imaging device image, a first coordinate system conversion relationship, and a second coordinate system conversion relationship, an aperture model of the medical imaging device is constructed; the first coordinate system conversion relationship is the corresponding relationship between the coordinate system of the first camera and the base coordinate system of the robotic arm; the second coordinate system conversion relationship is the corresponding relationship between the base coordinate system and the medical imaging coordinate system. In this application, an aperture model of the medical imaging device in the medical imaging coordinate system can be constructed based on the medical imaging device image collected by the first camera, the first coordinate system conversion relationship, and the second coordinate system conversion relationship. That is, the aperture model of the medical imaging device in this application is generated by modeling the CT environment and does not need to be matched with an interventional surgical robot. Therefore, during the planning process of the obstacle avoidance path, the aperture model of the medical imaging device in this application can be reused, enabling the interventional surgical robot to adapt to different types, models, and sizes of medical imaging devices, and the adaptability of the aperture model of the medical imaging device is relatively high.

[0109] In one embodiment, an implementation manner for obtaining an image of a medical imaging device is provided, that is, "obtaining an image of a medical imaging device collected by a first camera" in S201 above. The first depth image includes a first depth image outside the aperture and a first depth image inside the aperture. The first camera is disposed at the end of the robotic arm and includes:

[0110] Obtain an image of the medical imaging device collected by the first camera at a first position to obtain a first depth image outside the aperture; the first position is within the outer region of the scanning hole of the medical imaging device.

[0111] Obtain an image of the medical imaging device collected by the first camera at a second position to obtain a first depth image inside the aperture; the second position is within the inner region of the scanning hole.

[0112] In the embodiment of this application, as Figure 3 shown, Figure 3 FIG. is a schematic structural diagram of a scanning system in which a first camera is disposed at the end of a robotic arm in one embodiment. When the first camera is disposed at the end of the robotic arm, the medical imaging device image includes a first depth image, and the first depth image includes a first depth image outside the aperture and a first depth image inside the aperture. The scanning system can control the first camera disposed at the end of the robotic arm to move to several positions in the outer region of the scanning hole to collect an image of the medical imaging device at the first position to obtain a first depth image outside the aperture. As Figure 4 shown, Figure 4Schematic diagram of the position of the first camera when collecting the first depth image inside the acquisition aperture in an embodiment. The scanning system can also control the movement of the first camera arranged at the end of the robotic arm to several positions inside the scanning aperture area of the medical imaging device to collect images of the medical imaging device at the second position, so as to obtain the first depth image inside the aperture.

[0113] Among them, the outside of the aperture refers to the area where the outer circular edge of the scanning aperture of the medical imaging device can be observed. According to the first depth image outside the aperture, the approximate size structure of the outer circle of the scanning aperture of the medical imaging device can be determined; the inside of the aperture refers to the area near the inner circle of the scanning aperture of the medical imaging device. According to the first depth image inside the aperture, the approximate size structure of the inner circle of the scanning aperture of the medical imaging device can be determined. The first position is several positions in the area outside the scanning aperture of the medical imaging device, and the second position is several positions in the area inside the scanning aperture. Combining Figure 4 As shown, the second position can include but is not limited to position 1, position 2, etc. When the first camera reaches the second position, the end of the robotic arm can be rotated at a certain step length to achieve scanning and image acquisition at different angles. The first position and the second position can be set according to the actual measured distance of the first camera, the size of the medical imaging device, the field of view of the first camera, etc. The embodiments of the present application do not limit this.

[0114] In this embodiment, the image of the medical imaging device collected by the first camera at the first position in the area outside the scanning aperture can be obtained to obtain the first depth image outside the aperture; the image of the medical imaging device collected by the first camera at the second position in the area inside the scanning aperture can also be obtained to obtain the first depth image inside the aperture, so as to obtain a more accurate first depth image.

[0115] In an embodiment, an implementation method for constructing an aperture model of a medical imaging device is provided, that is, "construct an aperture model of the medical imaging device according to the medical imaging device image, the first coordinate system conversion relationship, and the second coordinate system conversion relationship" in S202 above. As Figure 5 shown, it includes:

[0116] S301, construct an initial model of the medical imaging device according to the initial size of the medical imaging device and the size of the connection structure.

[0117] Combining Figure 3As shown in the figure, taking a CT device as an example, the scanning system includes a medical imaging device, a robotic arm, and a host computer. The medical imaging device includes the aperture of the CT device (abbreviated as CT aperture) and the bed. The robotic arm includes a first camera and a second camera. The first camera can be a first depth camera, and the second camera can be a second depth camera. Exemplarily, the second camera can be a structured light camera. At this time, the first camera is disposed at the end of the robotic arm, and the medical imaging device image includes a first depth image.

[0118] Among them, the initial size of the medical imaging device may include but is not limited to the diameter of the aperture in the medical imaging device. It should be noted that the initial size of the medical imaging device refers to the approximate size of the medical imaging device. The connection structure may include but is not limited to a waterproof cover structure, etc. Exemplarily, the waterproof cover structure is usually a cylindrical curved surface structure. The waterproof cover structure can be transparent, or the waterproof cover structure can also be opaque. The size of the connection structure may include but is not limited to the width and height of the connection structure, etc. That is, in this embodiment, the approximate size of the medical imaging device and the size of the connection structure are known in advance. The initial model of the medical imaging device is a three-dimensional model of the connection structure in the aperture of the medical imaging device. In the embodiments of the present application, since there is a connection structure in the medical imaging device and the image information of the connection structure cannot be obtained, based on this, the scanning system can construct a model using an existing mechanical structure according to the initial size of the medical imaging device and the size of the connection structure, and construct an initial model of the medical imaging device in the medical imaging coordinate system.

[0119] S302. Construct an inner and outer model of the aperture of the medical imaging device according to the first depth image.

[0120] Among them, the medical imaging device image includes a first depth image. The first depth image is a depth image collected by the first depth camera when the first camera is disposed at the end of the robotic arm. The inner and outer model of the aperture of the medical imaging device includes a three-dimensional model of the inner and outer edges of the aperture of the medical imaging device.

[0121] In the embodiments of the present application, the scanning system can construct an inner and outer model of the aperture of the medical imaging device according to the first depth image. Exemplarily, for the outer edge of the aperture, the scanning system can construct a model of the outer edge of the aperture of the medical imaging device in the coordinate system of the first camera according to the first depth image corresponding to the outer edge of the aperture collected by the first camera; for the inner edge of the aperture, the scanning system can construct a model of the inner edge of the aperture of the medical imaging device in the coordinate system of the first camera according to the first depth image corresponding to the inner edge of the aperture collected by the first camera; by splicing the outer edge model of the aperture of the medical imaging device and the inner edge model of the aperture of the medical imaging device, the scanning system can obtain an inner and outer model of the aperture of the medical imaging device in the coordinate system of the first camera.

[0122] S303. Based on the first coordinate system conversion relationship, the second coordinate system conversion relationship, the initial model of the medical imaging device, and the inner and outer aperture models, construct the aperture model of the medical imaging device.

[0123] In the embodiment of the present application, the scanning system can construct the aperture model of the medical imaging device according to the first coordinate system conversion relationship, the second coordinate system conversion relationship, the initial model of the medical imaging device in the medical imaging coordinate system, and the inner and outer aperture models in the coordinate system of the first camera.

[0124] In one embodiment, S303 includes:

[0125] According to the first coordinate system conversion relationship, perform coordinate system conversion on the inner and outer aperture models to obtain the first converted inner and outer aperture models.

[0126] According to the second coordinate system conversion relationship, perform coordinate system conversion on the first converted inner and outer aperture models to obtain the second converted inner and outer aperture models.

[0127] According to the second converted inner and outer aperture models and the initial model of the medical imaging device, obtain the aperture model of the medical imaging device.

[0128] In the embodiment of the present application, the scanning system can convert the inner and outer aperture models in the coordinate system of the first camera to the base coordinate system of the robotic arm according to the first coordinate system conversion relationship, to obtain the first converted inner and outer aperture models in the base coordinate system of the robotic arm. Thus, the scanning system can convert the first converted inner and outer aperture models in the base coordinate system of the robotic arm to the medical imaging coordinate system according to the second coordinate system conversion relationship, to obtain the second converted inner and outer aperture models in the medical imaging coordinate system. Then, the scanning system can splice the second converted inner and outer aperture models in the medical imaging coordinate system and the initial model of the medical imaging device in the medical imaging coordinate system to obtain the aperture model of the medical imaging device.

[0129] In this embodiment, the initial model of the medical imaging device in the medical imaging coordinate system can be constructed more accurately according to the initial size of the medical imaging device and the size of the connection structure; the inner and outer aperture models of the medical imaging device in the coordinate system of the first camera can be constructed more accurately according to the first depth image; thus, according to the first coordinate system conversion relationship, the second coordinate system conversion relationship, and the inner and outer aperture models, the inner and outer aperture models in the coordinate system of the first camera can be converted to the medical imaging coordinate system, and according to the more accurate inner and outer aperture models converted to the medical imaging coordinate system and the more accurate initial model of the medical imaging device in the medical imaging coordinate system, a more accurate and complete aperture model of the medical imaging device can be obtained.

[0130] In one embodiment, an implementation method for constructing an aperture model of a medical imaging device is provided, that is, "construct an aperture model of the medical imaging device according to the medical imaging device image, the first coordinate system conversion relationship, and the second coordinate system conversion relationship" in S202 above, including:

[0131] Construct an inner and outer aperture model of the medical imaging device according to the first depth image.

[0132] Determine the aperture edge according to the inner and outer aperture model, and perform point cloud expansion according to the aperture edge to obtain an intermediate aperture model of the medical imaging device.

[0133] Construct an aperture model of the medical imaging device according to the first coordinate system conversion relationship, the second coordinate system conversion relationship, and the intermediate aperture model.

[0134] In the embodiment of the present application, if the size of the connection structure is not known in advance, at this time, the scanning system can construct an inner and outer aperture model of the medical imaging device according to the first depth image, where the inner and outer aperture model refers to the model of the medical imaging device excluding the connection structure. The specific method for constructing the inner and outer aperture model of the medical imaging device can refer to S302 and will not be elaborated here. Then, the scanning system can determine the outer aperture edge and the inner aperture edge according to the inner and outer aperture model of the medical imaging device, and perform point cloud expansion along the direction of the connection structure in the aperture according to the outer aperture edge and the inner aperture edge, so as to expand the point cloud data of the connection structure in the aperture of the medical imaging device. Furthermore, according to the point cloud data of the connection structure and the inner and outer aperture model, an intermediate aperture model of the medical imaging device is constructed, where the intermediate aperture model of the medical imaging device refers to the overall three-dimensional model of the aperture of the medical imaging device in the coordinate system of the first camera.

[0135] Then, the scanning system can convert the intermediate aperture model in the coordinate system of the first camera to the base coordinate system of the robotic arm according to the first coordinate system conversion relationship and the intermediate aperture model, and obtain a first intermediate aperture model in the base coordinate system of the robotic arm. Thus, the scanning system can convert the first intermediate aperture model in the base coordinate system of the robotic arm to the medical imaging coordinate system according to the second coordinate system conversion relationship and the first intermediate aperture model, and obtain an aperture model of the medical imaging device.

[0136] In this embodiment, when the size of the connection structure is not known in advance, the connection structure can be completely constructed by point cloud expansion according to the inner and outer aperture model, so as to obtain the overall three-dimensional model of the aperture of the complete medical imaging device including the connection structure. Furthermore, through coordinate system conversion, an aperture model of the medical imaging device in the medical imaging coordinate system can be obtained.

[0137] In one embodiment, an implementation method for obtaining the first coordinate system conversion relationship is provided. The first camera is disposed at the end of the robotic arm and includes:

[0138] Obtain a first calibration image of the first calibration board captured by the first camera; the first calibration board is disposed on the bed of the medical imaging device.

[0139] Calibrate the coordinate system of the first camera and the base coordinate system based on the first calibration image to obtain the first coordinate system conversion relationship.

[0140] In the embodiment of the present application, as shown in Figure 3 When the first camera is disposed at the end of the robotic arm, a first calibration board can be disposed on the bed of the medical imaging device. The first calibration board can be a checkerboard calibration board, and the first calibration board needs to be within the field of view of the first camera and remain stationary during the calibration process. At this time, the scanning system can control the robotic arm to move to a specified position so that the first camera captures a first calibration image of the first calibration board. Thus, the scanning system can obtain the first calibration image of the first calibration board captured by the first camera. Then, the scanning system can calibrate the coordinate system of the first camera and the base coordinate system of the robotic arm based on the first calibration image to obtain the first coordinate system conversion relationship. The calibration method can be an existing hand-eye calibration method, or it can be other calibration methods. Of course, the embodiment of the present application does not limit the calibration method.

[0141] In this embodiment, a first calibration image of the first calibration board captured by the first camera can be obtained, and the coordinate system of the first camera and the base coordinate system can be calibrated based on the first calibration image, so as to accurately obtain the first coordinate system conversion relationship.

[0142] In one embodiment, another implementation method for constructing the aperture model of the medical imaging device is provided, that is, "construct the aperture model of the medical imaging device according to the medical imaging device image, the first coordinate system conversion relationship, and the second coordinate system conversion relationship" in S202 above. As shown in Figure 6 The medical imaging device image includes a second depth image and includes:

[0143] S601, generate a point cloud image of the medical imaging device according to the second depth image.

[0144] As shown in Figure 7 shown, Figure 7Schematic diagram of a scanning system in which a first camera is disposed in a preset area on a bed or a tooling of a medical imaging device. Taking a CT device as an example, the scanning system includes a medical imaging device, a robotic arm, and a host computer. The medical imaging device includes the aperture of the CT device (abbreviated as CT aperture) and the bed. The robotic arm includes a first camera and a second camera. The first camera may be a first depth camera, and the second camera may be a second depth camera. Exemplarily, the second camera may be a structured light camera. At this time, the medical imaging device image includes a second depth image, and the second depth image may include a depth image and a color image. The first camera is disposed in a preset area on the bed or a tooling of the medical imaging device, and the first camera is connected to the host computer, as Figure 8 shown Figure 8 Schematic diagram of the RGB field of view of the first camera in one embodiment. The CT aperture is at the center of the field of view of the first camera, and it is necessary to ensure that the CT aperture is complete and clearly visible.

[0145] Among them, the medical imaging device image includes a second depth image, which is the depth image and color image collected by the first depth camera when the first camera is disposed in a preset area on the bed or a tooling of the medical imaging device. The point cloud image of the medical imaging device is an image including the point cloud information of the medical imaging device. In the embodiments of the present application, the scanning system may generate the point cloud image of the medical imaging device according to the second depth image.

[0146] In one embodiment, S601 includes:

[0147] Perform edge detection on the scanning hole of the medical imaging device in the color image collected by the first depth camera to determine the edge information of the scanning hole.

[0148] Extract the point cloud information of the scanning hole from the second depth image according to the edge information.

[0149] Construct the point cloud image of the medical imaging device according to the point cloud information.

[0150] In the embodiments of the present application, after obtaining the second depth image, the scanning system may use the Canny edge detection operator to perform edge detection on the scanning hole (i.e., the CT aperture) of the medical imaging device in the color image of the second depth image to obtain the circular edge of the scanning hole of the medical imaging device, and then use a circular template to fit the circular edge detected in the color image to determine the position of the CT aperture, so as to determine the edge information of the scanning hole according to the position of the CT aperture. Then, the scanning system may extract the point cloud information of the scanning hole from the depth image of the second depth image according to the edge information of the scanning hole. Among them, the edge information of the scanning hole includes the pixel information of the edge of the scanning hole, and the point cloud information of the scanning hole includes the point cloud coordinates of the edge of the scanning hole. It should be noted that the point cloud information of the scanning hole is in the coordinate system of the first camera.

[0151] Thus, the scanning system can construct a point cloud image of the medical imaging device based on the point cloud information. Exemplarily, for the point cloud information outside the CT aperture, since the only object in the scanning system is the examination table and the examination table is located outside the CT aperture, the point cloud information at the aperture edge (i.e., the point cloud information of the scanning hole) can be used as the basis for removing irrelevant point clouds. For example, assume that the point cloud coordinates at the CT aperture edge are: ( ) , then the average coordinates ( ) of the point cloud at the CT aperture edge can be calculated by the following formula (1):

[0152] (1)

[0153] Since the z-axis of the general first depth camera is the direction of the camera's optical center, can be set as the threshold, and the point clouds smaller than this threshold are removed. For the remaining unremoved point clouds outside the CT aperture, data processing such as filtering and noise reduction can be performed on the unremoved point clouds to obtain the point cloud information at the CT aperture edge and the point cloud information outside the CT aperture. For the point cloud information inside the CT aperture, since the robotic arm will penetrate deep into the CT aperture during the percutaneous biopsy operation, the point cloud information inside the CT aperture needs to be filled to obtain the filled point cloud information inside the CT aperture. Thus, the scanning system can construct a point cloud image of the medical imaging device based on the point cloud information at the CT aperture edge, the point cloud information outside the CT aperture, and the filled point cloud information inside the CT aperture.

[0154] S602. According to the first coordinate system conversion relationship, perform coordinate system conversion on the point cloud image to obtain the point cloud image in the base coordinate system.

[0155] In the embodiment of the present application, the scanning system can convert the point cloud image in the coordinate system of the first camera to the base coordinate system of the robotic arm according to the first coordinate system conversion relationship to obtain the point cloud image in the base coordinate system of the robotic arm.

[0156] S603. According to the second coordinate system conversion relationship and the point cloud image in the base coordinate system, perform model construction to obtain the aperture model of the medical imaging device.

[0157] In the embodiment of the present application, the scanning system can perform model construction according to the second coordinate system conversion relationship and the point cloud image in the base coordinate system of the robotic arm to obtain the aperture model of the medical imaging device.

[0158] In one of the embodiments, S603 includes:

[0159] Expand the point cloud image in the base coordinate system along the central axis of the aperture of the medical imaging device in the direction of entering the aperture to obtain the target point cloud image in the base coordinate system.

[0160] Construct a model based on the target point cloud image to obtain the constructed point cloud model.

[0161] According to the second coordinate system conversion relationship, perform medical imaging coordinate system conversion on the point cloud model to obtain the aperture model of the medical imaging device.

[0162] In the embodiments of the present application, as Figure 9 shown, Figure 9 FIG. is a schematic diagram of point cloud expansion in an embodiment. Taking a CT device as an example, the dark line is the point cloud information at the edge of the CT aperture, the right side of the dark line is the expanded point cloud image, and the white arrow is the direction of point cloud expansion. The scanning system can use the point cloud information at the edge of the CT aperture as a reference to expand the point cloud image in the base coordinate system of the robotic arm along the central axis of the aperture of the medical imaging device in the direction of entering the aperture at a certain step length to obtain the expanded point cloud image. Then, based on the point cloud image in the base coordinate system of the robotic arm and the expanded point cloud image, the target point cloud image in the base coordinate system of the robotic arm can be obtained. After that, the scanning system can construct a model based on the target point cloud image to obtain the point cloud model in the base coordinate system of the constructed robotic arm. Among them, the method of model construction can be methods such as triangulation. The embodiments of the present application do not limit the method of model construction. Thus, the scanning system can convert the point cloud model in the base coordinate system of the robotic arm to the medical imaging coordinate system according to the second coordinate system conversion relationship to obtain the aperture model of the medical imaging device in the medical imaging coordinate system.

[0163] In this embodiment, when the initial size of the medical imaging device is unknown, it is possible to generate a relatively accurate point cloud image of the medical imaging device only by taking the second depth image of the complete aperture of the medical imaging device without inserting the first camera into the aperture to take the image inside the aperture. Then, according to the first coordinate system conversion relationship, perform coordinate system conversion on the relatively accurate point cloud image to obtain the relatively accurate point cloud image in the base coordinate system. Thus, the point cloud image in the base coordinate system can be expanded along the central axis of the aperture of the medical imaging device in the direction of entering the aperture to obtain the complete target point cloud image in the base coordinate system. Furthermore, based on the second coordinate system conversion relationship and the target point cloud image, a model can be constructed to obtain the relatively accurate aperture model of the medical imaging device.

[0164] In one embodiment, an implementation method for obtaining the first coordinate system conversion relationship is provided. The first camera is set on the bed of the medical imaging device, as Figure 11 shown, including:

[0165] S1101. Obtain a second calibration image of the second calibration board captured by the second camera; the second calibration board is disposed at the end of the robotic arm.

[0166] S1102. Obtain a third calibration image of the second calibration board captured by the first camera.

[0167] In the embodiments of the present application, as shown in Figure 7 , when the first camera is disposed on the bed body of the medical imaging device, a second calibration board can be installed at the end of the robotic arm, where the second calibration board can be a checkerboard calibration board. By dragging the end of the robotic arm and adjusting the configuration of the robotic arm, the second calibration board can be positioned within the field of view of the first camera and the second camera, and the checkerboard corner points are clearly visible. At this time, as shown in Figure 10 . Figure 10 FIG. is a schematic diagram of the position of the second calibration board in one embodiment. Figure 10 The left figure of FIG. is a schematic diagram of the position of the second calibration board within the field of view of the first camera. Figure 10 The right figure of FIG. is a schematic diagram of the position of the second calibration board within the field of view of the second camera.

[0168] In the embodiments of the present application, the second camera can capture a second calibration image of the second calibration board so that the scanning system obtains the second calibration image of the second calibration board captured by the second camera. And the first camera can capture a third calibration image of the second calibration board so that the scanning system obtains the third calibration image of the second calibration board captured by the first camera. Among them, the positions of the checkerboard corner points can be clearly determined based on both the second calibration image and the third calibration image. Of course, the embodiments of the present application do not limit the order of capturing the second calibration image and the third calibration image.

[0169] S1103. Calibrate the coordinate system of the first camera and the base coordinate system according to the second calibration image and the third calibration image to obtain a first coordinate system conversion relationship.

[0170] In one of the embodiments, S1103 includes:

[0171] Perform singular value decomposition on the second calibration image and the third calibration image to obtain a first conversion relationship between the coordinate system where the second calibration image is located and the coordinate system where the third calibration image is located.

[0172] Obtain a second conversion relationship between the coordinate system of the first camera and the coordinate system where the third calibration image is located.

[0173] Obtain a third conversion relationship between the coordinate system where the second calibration image is located and the base coordinate system.

[0174] Obtain a first coordinate system conversion relationship based on the first conversion relationship, the second conversion relationship, and the third conversion relationship.

[0175] In the embodiments of the present application, the scanning system can use the known internal parameters of the second camera to convert the positions of the checkerboard corner points of the extracted second calibration image from the image coordinate system of the second camera to the RGB coordinate system of the second camera, so as to obtain the representation of the feature point set of the second calibration image in the RGB coordinate system of the second camera. After that, the scanning system can calculate the center point of the feature point set of the second calibration image in the RGB coordinate system of the second camera. Similarly, the scanning system can also use the known internal parameters of the first camera to convert the checkerboard corner points of the extracted third calibration image from the image coordinate system of the first camera to the RGB coordinate system of the first camera, so as to obtain the representation of the feature point set of the third calibration image in the RGB coordinate system of the first camera. After that, the scanning system can calculate the center point of the feature point set of the third calibration image in the RGB coordinate system of the first camera.

[0176] Thus, the scanning system can use the center point of the feature point set of the second calibration image in the RGB coordinate system of the second camera and the center point of the feature point set of the third calibration image in the RGB coordinate system of the first camera, and use the Singular Value Decomposition (SVD) method to solve the rotation matrix between the feature point set of the second calibration image and the feature point set of the third calibration image, so as to calculate the first conversion relationship between the coordinate system where the second calibration image is located (i.e., the RGB coordinate system of the second camera) and the coordinate system where the third calibration image is located (i.e., the RGB coordinate system of the first camera). After that, the scanning system can obtain the second conversion relationship between the coordinate system of the first camera and the coordinate system where the third calibration image is located. And obtain the third conversion relationship between the coordinate system where the second calibration image is located and the base coordinate system of the robotic arm. Of course, the embodiments of the present application do not limit the order of obtaining the second conversion relationship and obtaining the third conversion relationship.

[0177] After that, the scanning system can calculate the first conversion relationship. And the second conversion relationship. The product of them to obtain the corresponding relationship between the coordinate system where the second calibration image is located and the coordinate system of the first camera. Thus, the scanning system can calculate the pre-calibrated third conversion relationship. And the above corresponding relationship. The product of them to obtain the first coordinate system conversion relationship.

[0178] In this embodiment, the coordinate system of the first camera and the base coordinate system can be calibrated according to the obtained second calibration image of the second calibration board captured by the second camera and the third calibration image of the second calibration board captured by the first camera, and the first coordinate system conversion relationship can be obtained more accurately.

[0179] In one embodiment, an implementation method for obtaining the second coordinate system conversion relationship is provided. The method for constructing the medical image model further includes:

[0180] Register the coordinate system of the medical imaging device and the base coordinate system of the robotic arm to obtain the second coordinate system conversion relationship.

[0181] In the embodiments of the present application, the scanning system can register the coordinate system of the medical imaging device and the base coordinate system of the robotic arm to obtain the second coordinate system conversion relationship. Among them, as Figure 12 shown, Figure 12 FIG. is a schematic diagram of the application of the second coordinate system conversion relationship in one embodiment. The purpose of registration is to obtain the base coordinate system of the robotic arm and the second coordinate system conversion relationship between the medical imaging coordinate system . Exemplarily, a metal steel ball can be placed at the end of the robotic arm, and the end of the robotic arm can be controlled to move near the CT collimation center line. The coordinates of the center of the metal steel ball in the medical imaging coordinate system can be calculated by means of CT exposure, and the coordinates of the center of the metal steel ball in the base coordinate system of the robotic arm can be obtained according to the forward kinematics principle of the robotic arm. Repeating this process can calculate the second coordinate system conversion relationship. .

[0182] In the case where the position of the robotic arm changes, re-register the coordinate system of the medical imaging device and the base coordinate system of the robotic arm to obtain a new second coordinate system conversion relationship.

[0183] Determine the correction conversion relationship according to the new second coordinate system conversion relationship and the second coordinate system conversion relationship.

[0184] Correct the aperture model of the medical imaging device according to the correction conversion relationship to obtain a new aperture model of the medical imaging device.

[0185] In the embodiments of the present application, during actual use, according to the positioning of the operating table (i.e., the mechanical structure where the robotic arm is located), the corresponding operating table model can be added to the collision detection environment to achieve intraoperative obstacle avoidance of the robotic arm. However, the above-mentioned intraoperative obstacle avoidance process of the robotic arm has relatively high requirements for the resetability of the operating table, and is limited by factors such as the surgical site and surgical space during actual use, and the robotic arm positioning needs to be frequently changed to move the operating table to a specific position and fix it. Therefore, when there is a deviation between the actual position of the operating table and the standard left and right positions, there will be an obvious difference between the model in the original configuration and the actual model. Among them, when the operating table is on the left side of the CT aperture, it is the left position, and when the operating table is on the right side of the CT aperture, it is the right position.

[0186] Based on this, in order to reduce the potential collision risk, the scanning system can update the model position through a registration process. As Figure 13 shown, Figure 13 FIG. is a schematic diagram of updating the model position in an embodiment. After the surgical trolley is installed, the scanning system can calculate the second coordinate system conversion relationship through the registration process . During subsequent use, when the position of the robotic arm changes, it is necessary to re-register the coordinate system of the medical imaging device and the base coordinate system of the robotic arm to obtain a new second coordinate system conversion relationship . And the scanning system can be based on the new second coordinate system conversion relationship and the second coordinate system conversion relationship to determine the correction conversion relationship . Among them, the correction conversion relationship is the transformation relationship between two medical imaging coordinate systems. Since the position of the medical imaging coordinate system actually does not move, therefore, the transformation relationship between medical imaging coordinate systems actually represents the transformation relationship between the base coordinate systems of the robotic arm. Thus, the scanning system can use the transformation relationship between the base coordinate systems of the robotic arm as a compensation matrix, and use the compensation matrix to correct the aperture model of the medical imaging device to update the coordinate system of the aperture model of the medical imaging device, and obtain a new aperture model of the medical imaging device corresponding to the positioning of the new surgical trolley. It should be noted that the re-registration process does not change the size of the aperture model of the medical imaging device. The purpose of re-registration is to change the relative position of the aperture model of the medical imaging device relative to the positioning of the surgical trolley, so as to automatically update the position of the aperture model of the medical imaging device.

[0187] In this embodiment, when the position of the robotic arm changes, the coordinate system of the medical imaging device and the base coordinate system of the robotic arm are re-registered to obtain a new second coordinate system conversion relationship; according to the new second coordinate system conversion relationship and the second coordinate system conversion relationship, the correction conversion relationship is determined; according to the correction conversion relationship, the aperture model of the medical imaging device is corrected to obtain a new aperture model of the medical imaging device. In this way, only need to scan the aperture model of the medical imaging device once, and the position of the aperture model of the medical imaging device can be automatically updated without being restricted by the positioning of the surgical trolley during subsequent use.

[0188] In one embodiment, as Figure 14 shown, a method for constructing a structural model is provided. Taking the method applied to the Figure 1 scanning system as an example, it includes the following steps:

[0189] S1401. Obtain a target image captured by a second camera; the target image includes a target area of a subject.

[0190] Wherein, the target image refers to a depth image acquired by a second depth camera of the second camera. The target image includes a target area of the subject. The subject may be a patient, and the target area is the key observation area of the subject. Exemplarily, the target area may be the area of the subject from the neck to the knees.

[0191] In the embodiments of the present application, as Figure 15 shown, Figure 15 is a schematic diagram of a target image in an embodiment. The second camera can capture a target image in the depth coordinate system of the second camera, so that the scanning system can obtain the target image captured by the second camera. It should be noted that although the target image of the whole body of the subject can be obtained by multiple acquisitions and stitching of the second camera, in the actual use process, the head and calf areas of the subject are not within the working space of the robotic arm. Therefore, there is no need to model the head and calf areas of the subject, and only by capturing the target image of the target area of the subject, the target image can be obtained conveniently and quickly.

[0192] S1402. Obtain a scanned image of a medical imaging device; the scanned image includes a lesion area of the subject; the lesion area is a partial area within the target area.

[0193] Wherein, the scanned image refers to an image obtained by scanning with a medical imaging device. The scanned image includes a lesion area of the subject. The lesion area refers to an abnormal lesion area that appears inside the tissue or organ of the subject. The lesion area is a partial area within the target area. In the embodiments of the present application, the medical imaging device can scan to obtain a scanned image, so that the scanning system can obtain the scanned image of the medical imaging device.

[0194] It should be noted that S1402 is executed after S1401, that is, in the embodiments of the present application, the target image captured by the second camera is obtained first, and then the scanned image of the medical imaging device is obtained.

[0195] Since the error of CT scan modeling is only 1-2 mm, while the error of structured light modeling is generally ≥5 mm, in the related art, in the process of constructing the structural model of the subject, the subject must be moved to a specified position first, the lesion part of the subject is accurately CT scanned, then the other parts of the subject are scanned by structured light, and then the CT scanned image and the structured light scanned image are stitched together to obtain an accurate structural model of the subject. After that, the lesion part of the subject can be moved under the CT to execute the surgical procedure, and this process is relatively complex.

[0196] In the embodiments of the present application, due to the large field of view of structured light scanning, generally the field of view is 1 meter or even larger. Therefore, in the embodiments of the present application, as long as there is a target area related to the surgery within the field of view of the structured light camera, the structured light camera can be used to scan and obtain an image that can completely cover the target area related to the surgery. Thus, after the structured light camera obtains the target image, the object to be photographed can be moved into the medical imaging device, and a large-range two-dimensional positioning image can be obtained by plain scanning with the medical imaging device. The user selects the area of interest (including the lesion site) on the two-dimensional positioning image for three-dimensional spiral scanning to obtain the scanned image. It should be noted that when the above user selects the area of interest, the scanning system can automatically obtain the bed code value of the medical imaging device corresponding to the area of interest. Thus, the target image and the scanned image can be stitched based on the bed code value. Therefore, the present application can obtain the target image and the scanned image required for constructing the structural model of the object to be photographed without moving the object to be photographed to a specified position for scanning.

[0197] S1403. Construct a structural model of the object to be photographed based on the target image and the scanned image.

[0198] In the related art, the "segmented scanning + stitching" method can be used to model the body surface area of the object to be photographed. However, in the actual clinical scenario, this process requires the doctor to move the bed multiple times, increasing the surgical time and making the operation process more complex. Based on this, in the embodiments of the present application, the scanning system can model the body surface area of the object to be photographed according to the target image and the scanned image to construct a structural model of the object to be photographed. Since the target image is obtained first and then the scanned image in the present application, after a single CT scan in the present application, the structural model can be constructed without the doctor moving the bed to the preset position again, greatly shortening the surgical time and simplifying the operation process.

[0199] In the above method for constructing the structural model, obtain the target image captured by the second camera; the target image includes the target area of the object to be photographed; obtain the scanned image of the medical imaging device; the scanned image includes the lesion area of the object to be photographed; the lesion area is a partial area within the target area; construct a structural model of the object to be photographed based on the target image and the scanned image. In the present application, the target image captured by the second camera can be obtained first, and then the scanned image of the medical imaging device can be obtained. Therefore, after a single CT scan in the present application, the structural model of the object to be photographed can be constructed according to the target image and the scanned image without the doctor moving the bed to the preset position again and performing segmented scanning on the patient, greatly shortening the surgical time and simplifying the operation process.

[0200] In one embodiment, an implementation method for constructing a structural model, that is, "constructing a structural model of the photographed object according to the target image and the scanned image" in S1403 above, includes:

[0201] Constructing a point cloud model of the photographed object according to the target image.

[0202] Constructing a scanned model of the photographed object according to the scanned image.

[0203] Stitching the point cloud model and the scanned model to obtain the structural model.

[0204] In the embodiment of the present application, as Figure 16 shown, Figure 16 is a schematic flowchart of the process of obtaining a point cloud model in one embodiment. The scanning system can construct a point cloud model of the photographed object (i.e., the patient body surface model or the structured light model) in the depth coordinate system of the second camera according to the target image in the depth coordinate system of the second camera, and perform coordinate transformation on the point cloud model of the photographed object according to the first coordinate transformation relationship to obtain the point cloud model of the photographed object in the base coordinate system of the robotic arm. In addition, in the embodiment of the present application, as long as there is a surgical-related target area within the field of view of the structured light camera, the structured light camera can be used to scan and obtain an image that can completely cover the surgical-related target area.

[0205] After that, as Figure 17 shown, Figure 17 is a schematic flowchart of the process of obtaining a scanned model in one embodiment. The scanning system can construct a scanned model of the photographed object (i.e., the medical image model) according to the scanned image and the DICOM image (a general protocol used by medical institutions at home and abroad to manage and transmit medical image data and related data, Digital Imaging and Communications in Medicine). Thus, the scanning system can perform coordinate transformation on the scanned model according to the second coordinate transformation relationship to obtain the scanned model in the base coordinate system of the robotic arm. Compared with the point cloud model, the scanned model has higher accuracy. Therefore, the scanned model is suitable for accurate modeling of the lesion area. In addition, when obtaining the scanned image, it is also necessary to record the first couch value and the second couch value .

[0206] Thus, after obtaining the scanned model in the base coordinate system of the robotic arm, the scanning system can calculate the difference between and , and calculate the difference between and to update the scanned model to the position where the second camera takes a photo. As Figure 18 shown, Figure 18Schematic diagram of the process of model splicing in an embodiment. The scanning system can, according to the bed code value The first bedplate value and the second bedplate value , shear the area in the point cloud model that overlaps with the scanning model to obtain the sheared point cloud model. The scanning system can splice the sheared point cloud model and the scanning model to obtain the structural model. In addition, during the subsequent surgical process, the real-time position of the bedplate can be obtained, and the real-time position of the bedplate and the bed code value are calculated, and then the structural model is translated by a preset vector along the base coordinate system of the robotic arm according to the difference, so as to update the structural model.

[0207] In this embodiment, the point cloud model of the object to be photographed can be constructed according to the target image, and after a CT scan, the scanning model of the object to be photographed can be constructed according to the scanned image. Thus, the point cloud model and the scanning model can be spliced to obtain a more accurate structural model. This process does not require the doctor to move the bed to the preset position again, greatly shortening the operation time and simplifying the operation process.

[0208] In an embodiment, after obtaining the target image captured by the second camera, the method for constructing the above structural model further includes:

[0209] Controlling the bed of the medical imaging device to drive the object to be photographed to move to the scanning area of the medical imaging device.

[0210] Obtaining the scanned image of the medical imaging device, including:

[0211] Starting the medical imaging device to scan the scanning area to obtain the scanned image of the medical imaging device.

[0212] In the embodiment of the present application, after obtaining the target image captured by the second camera, the scanning system can control the bed of the medical imaging device to drive the object to be photographed to move to the scanning area of the medical imaging device. Thus, the scanning system can start the medical imaging device to scan the scanning area to obtain the scanned image of the medical imaging device. In this way, after obtaining the target image captured by the second camera, only one CT scan needs to be completed, and the structural model of the object to be photographed can be constructed according to the target image and the scanned image. Therefore, during the process of constructing the structural model of the object to be photographed, it is not necessary for the doctor to move the bed to the preset position again after the CT scan, greatly shortening the operation time and simplifying the operation process.

[0213] In an embodiment, the method for constructing the above structural model further includes:

[0214] Detect the binding device of the object to be photographed to obtain a detection result.

[0215] In the embodiments of the present application, after the patient lies on the bed of the medical imaging device in a supine, prone or lateral position, etc., in order to prevent the patient from moving during the scan and affecting the scan result, the doctor will use a restraint belt to fix the patient on the bed board of the bed in the medical imaging device, as Figure 19 shown. Figure 19 It is a schematic diagram of fixing the restraint belt in one embodiment. In order to avoid the surgical risks caused by the doctor not fixing the patient or the restraint being insecure before performing the operation, the scanning system can detect the binding device of the object to be photographed to obtain a detection result. Among them, the detection result is used to indicate whether the restraint degree of the patient meets the preset requirements.

[0216] In one of the embodiments, as Figure 20 shown, detecting the binding device of the object to be photographed to obtain a detection result includes:

[0217] S2001, detect the identifier on the binding device to determine whether the binding device includes a preset number of identifiers.

[0218] In the embodiments of the present application, the scanning system can detect the identifier on the binding device to determine whether the binding device includes a preset number of identifiers. In one exemplary embodiment, S2001 includes:

[0219] Obtain an image of the binding device captured by the second camera.

[0220] Identify the number of identifiers in the image of the binding device to obtain an identification result.

[0221] Determine whether the binding device includes a preset number of identifiers according to the identification result.

[0222] In the embodiments of the present application, after the patient is fixed, the bed board of the bed will move into the CT aperture. At this time, the RGB camera in the second camera can be used to capture an image of the binding device, so that the scanning system can obtain the image of the binding device captured by the second camera. Among them, the binding device can be a restraint belt. After that, the scanning system can identify the number of identifiers in the image of the binding device to obtain an identification result. If the identification codes on each single restraint belt can all be detected, it means that the binding device includes a preset number of identifiers, indicating that the restraint belt has been fixed as required; if the identification codes on each single restraint belt cannot all be detected, it means that the binding device does not include a preset number of identifiers, indicating that the restraint belt has not been fixed as required. At this time, the user needs to be prompted on the operation interface of the scanning system to adjust the fixing method of the restraint belt.

[0223] Exemplarily, the identifier can be an ArUco identification code. Thus, operations such as identification recognition and pose calculation of the ArUco identification code can be performed through the functions built in the AruCo module. Alternatively, the identifier can also be an identification object with obvious features. Thus, identification recognition can be performed through methods such as template matching and target detection. Of course, the embodiments of the present application do not limit the identifier and the way to identify the identifier.

[0224] S2002, if the binding device includes a preset number of identifiers, determine the first identifier and the second identifier on the binding device, and perform detection based on the distance between the first identifier and the second identifier to obtain a detection result.

[0225] S2003, if the binding device does not include a preset number of identifiers, determine that the detection result indicates a detection failure.

[0226] In the embodiments of the present application, if not all the identification codes on a single binding strap can be detected, it means that the binding device does not include a preset number of identifiers, indicating that the restraint strap is not fixed as required. At this time, determine that the detection result indicates a detection failure, and it is necessary to prompt the user on the operation interface of the scanning system to adjust the fixing method of the restraint strap and the surgical risk.

[0227] If all the identification codes on a single binding strap can be detected, it means that the binding device includes a preset number of identifiers, indicating that the restraint strap has been fixed as required. At this time, as Figure 21 shown, Figure 21 is a schematic diagram of the first identifier and the second identifier in an embodiment. The scanning system can determine the elastic component on the binding device, and determine the identifiers set on both sides of the elastic component as the first identifier and the second identifier. Among them, the elastic component can be an elastic band, and the first identifier and the second identifier can be two-dimensional codes. Then, the scanning system can perform detection based on the distance between the first identifier and the second identifier to obtain a detection result.

[0228] In one exemplary embodiment, performing detection based on the distance between the first identifier and the second identifier to obtain a detection result includes:

[0229] If the distance between the first identifier and the second identifier is greater than a preset threshold, determine that the detection result indicates a detection success.

[0230] If the distance between the first identifier and the second identifier is not greater than a preset threshold, determine that the detection result indicates a detection failure.

[0231] In the embodiments of the present application, the scanning system can calculate the distance between the first identifier and the second identifier. Optionally, assuming that the identifier is an ArUco identification code, then the scanning system can use the built-in function of the ArUco module in OpenCV to identify the ID of the ArUco identification code and the spatial pose transformation of the ArUco identification code corresponding to the coordinate system of the ArUco identification code relative to the second camera. Thus, the scanning system can calculate the distance between the first identifier and the second identifier according to this spatial pose transformation. Exemplarily, the built-in function of the ArUco module can be the "estimatePoseSingleMarkers" function; the ArUco identification code can be a two-dimensional code automatically generated using the dictionary in the ArUco module. The ArUco identification code can include the detection frame of the ArUco identification code, the coordinate system of the ArUco identification code, and the ID of the coordinate system of the ArUco identification code. The size and quantity of the ArUco identification code can be adjusted according to actual needs. Alternatively, the identifier can also be an identification object with obvious features. Then, the scanning system can obtain the bounding box of the identification object with obvious features and use the four corner points of the bounding box and the SolvePnP algorithm to solve the spatial pose of the identification object in the camera coordinate system, thereby obtaining the distance between the first identifier and the second identifier.

[0232] After that, the scanning system can determine whether the distance between the first identifier and the second identifier is greater than a preset threshold. Among them, the preset threshold can be determined according to the material of the elastic band, for example, 50 mm. Of course, the specific value of the preset threshold in the embodiments of the present application is not limited. If the distance between the first identifier and the second identifier is greater than the preset threshold, it means that the elastic band is tightened. At this time, it can be determined that the detection result indicates a successful detection. If the distance between the first identifier and the second identifier is not greater than the preset threshold, it means that the elastic band is not tightened. At this time, it can be determined that the detection result indicates a failed detection, and the user is prompted to adjust the binding degree of the binding band on the operation interface of the scanning system to prompt the user to tighten the binding band. In addition, in order to prevent the calculated distance between the first identifier and the second identifier from jumping near the preset threshold, a first-order lag filter and a sliding window can be used to filter the detection result of the distance detection to make the detection result more stable.

[0233] In this embodiment, obtaining the target image captured by the second camera includes:

[0234] When the detection result indicates a successful detection, obtain the target image captured by the second camera.

[0235] In the embodiment of the present application, when the detection result indicates successful detection, that is, when the scanning system detects the restraint belt and the restraint degree of the restraint belt meets the requirements, the camera field of view of the second camera can cover most of the patient's body surface area (i.e., the target area including the object to be photographed). At this time, the photographing function of the second camera can be triggered to obtain the point cloud data of the patient's body surface, so that the scanning system can obtain the target image photographed by the second camera. After that, the PCL point cloud processing library can be used to perform processing such as noise reduction, segmentation, filtering, and smoothing on the target image, and a point cloud model of the patient's body surface can be constructed.

[0236] In this embodiment, when the detection result indicates successful detection, that is, when the scanning system detects the restraint belt and the restraint degree of the restraint belt meets the requirements, the camera field of view of the second camera can cover most of the patient's body surface area (i.e., the target area including the object to be photographed). Therefore, in the present application, it is not necessary to move the second camera to a specific position to capture the target image of the target area. Instead, only when it is determined that the restraint belt is tightly fastened, the target image of the target area can be obtained, and then the point cloud model can be constructed according to the target image of the target area, greatly shortening the operation time and simplifying the operation process.

[0237] In one embodiment, a method for planning an obstacle avoidance path is provided, characterized in that taking the scanning system in Figure 1 as an example for illustration, the method includes the following steps:

[0238] According to the method in any of the above embodiments, an aperture model of the medical imaging device is constructed.

[0239] According to the method in any of the above embodiments, a structure model is constructed.

[0240] According to the aperture model, the structure model, and the dimensions of the robotic arm, the obstacle avoidance path of the robotic arm during the scanning process of the medical imaging device is determined.

[0241] In the embodiment of the present application, the scanning system can construct an aperture model and a structure model of the medical imaging device, and can determine the obstacle avoidance path of the robotic arm during the scanning process of the medical imaging device according to the aperture model, the structure model, and the dimensions of the robotic arm by using the existing path planning method. Among them, the specific methods for constructing the aperture model and the structure model of the medical imaging device can refer to the above embodiments and will not be elaborated here.

[0242] In the above-described method for planning the obstacle avoidance path, an aperture model of a medical imaging device with relatively high adaptability can be obtained, and a structural model can be constructed more quickly and conveniently. Therefore, according to the aperture model of the medical imaging device with relatively high adaptability, the structural model, and the dimensions of the robotic arm, the obstacle avoidance path of the robotic arm with relatively high adaptability during the scanning process of the medical imaging device can be determined more quickly and conveniently.

[0243] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are sequentially shown according to the indications of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same moment, but can be executed at different moments. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0244] Based on the same inventive concept, an embodiment of the present application further provides a medical imaging model construction device for implementing the above-described medical imaging model construction method. The solution provided by this device for solving problems is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the medical imaging model construction device provided below can refer to the limitations on the medical imaging model construction method in the above text, and will not be repeated here.

[0245] In an exemplary embodiment, as Figure 22 shown, a medical imaging model construction device is provided, including: a first acquisition module 21 and a first construction module 22, where:

[0246] The first acquisition module 21 is configured to acquire a medical imaging device image collected by a first camera; the first camera is used to capture a complete medical imaging device image.

[0247] The first construction module 22 is configured to construct an aperture model of the medical imaging device according to the medical imaging device image, a first coordinate system conversion relationship, and a second coordinate system conversion relationship; the first coordinate system conversion relationship is the corresponding relationship between the coordinate system of the first camera and the base coordinate system of the robotic arm; the second coordinate system conversion relationship is the corresponding relationship between the base coordinate system and the medical imaging coordinate system.

[0248] In one of the embodiments, the first depth image includes a first depth image outside the aperture and a first depth image inside the aperture. The first camera is disposed at the end of the robotic arm. The first acquisition module 21 includes:

[0249] An outer-aperture first depth image acquisition unit, configured to acquire an image of a medical imaging device collected by a first camera at a first position, so as to obtain an outer-aperture first depth image; the first position is within an outer region of a scanning aperture of the medical imaging device;

[0250] An inner-aperture first depth image acquisition unit, configured to acquire an image of a medical imaging device collected by a first camera at a second position, so as to obtain an inner-aperture first depth image; the second position is within an inner region of the scanning aperture.

[0251] In one embodiment, the medical imaging device image includes a first depth image, and the first construction module 22 includes:

[0252] An initial model construction unit of the medical imaging device, configured to construct an initial model of the medical imaging device according to the initial size of the medical imaging device and the size of the connection structure;

[0253] An inner-outer aperture model construction unit, configured to construct an inner-outer aperture model of the medical imaging device according to the first depth image;

[0254] A first construction unit, configured to construct an aperture model of the medical imaging device according to a first coordinate system conversion relationship, a second coordinate system conversion relationship, the initial model of the medical imaging device, and the inner-outer aperture model.

[0255] In one embodiment, the first construction unit includes:

[0256] A first conversion inner-outer aperture model generation subunit, configured to perform coordinate system conversion on the inner-outer aperture model according to the first coordinate system conversion relationship, so as to obtain a first conversion inner-outer aperture model;

[0257] A second conversion inner-outer aperture model generation subunit, configured to perform coordinate system conversion on the first conversion inner-outer aperture model according to the second coordinate system conversion relationship, so as to obtain a second conversion inner-outer aperture model;

[0258] A first construction subunit, configured to obtain an aperture model of the medical imaging device according to the second conversion inner-outer aperture model and the initial model of the medical imaging device.

[0259] In one embodiment, the first construction module 22 includes:

[0260] An inner-outer aperture model construction unit, configured to construct an inner-outer aperture model of the medical imaging device according to the first depth image;

[0261] An intermediate aperture model construction unit, configured to determine an aperture edge according to the inner-outer aperture model, and perform point cloud expansion according to the aperture edge, so as to obtain an intermediate aperture model of the medical imaging device;

[0262] An aperture model construction unit, configured to construct an aperture model of a medical imaging device according to a first coordinate system conversion relationship, a second coordinate system conversion relationship, and an intermediate aperture model.

[0263] In one embodiment, the medical imaging device image includes a second depth image, and the first construction module 22 includes:

[0264] A point cloud image generation unit, configured to generate a point cloud image of the medical imaging device according to the second depth image;

[0265] A point cloud image conversion unit, configured to perform coordinate system conversion on the point cloud image according to the first coordinate system conversion relationship to obtain a point cloud image in the base coordinate system;

[0266] A model construction unit, configured to perform model construction according to the second coordinate system conversion relationship and the point cloud image in the base coordinate system to obtain an aperture model of the medical imaging device.

[0267] In one embodiment, the model construction unit includes:

[0268] A point cloud expansion sub-unit, configured to expand the point cloud image in the base coordinate system along the central axis of the aperture of the medical imaging device in the direction of entering the aperture to obtain a target point cloud image in the base coordinate system;

[0269] A model construction sub-unit, configured to perform model construction according to the target point cloud image to obtain a constructed point cloud model;

[0270] A model conversion sub-unit, configured to perform medical imaging coordinate system conversion on the point cloud model according to the second coordinate system conversion relationship to obtain an aperture model of the medical imaging device.

[0271] In one embodiment, the point cloud image generation unit includes:

[0272] An edge information determination sub-unit, configured to perform edge detection on the scanning hole of the medical imaging device in the color image to determine the edge information of the scanning hole;

[0273] A point cloud information extraction sub-unit of the scanning hole, configured to extract the point cloud information of the scanning hole from the second depth image according to the edge information;

[0274] A point cloud image generation sub-unit, configured to construct a point cloud image of the medical imaging device according to the point cloud information.

[0275] In one embodiment, the first camera is disposed at the end of the robotic arm, and the medical imaging model construction device further includes:

[0276] The first calibration image acquisition module is used to acquire a first calibration image of a first calibration board captured by a first camera; the first calibration board is arranged on the bed body of the medical imaging device;

[0277] The first generation module is used to calibrate the coordinate system of the first camera and the base coordinate system according to the first calibration image, so as to obtain a first coordinate system conversion relationship.

[0278] In one embodiment, the first camera is arranged on the bed body of the medical imaging device, and the medical imaging model construction device further includes:

[0279] The second calibration image acquisition module is used to acquire a second calibration image of a second calibration board captured by a second camera; the second calibration board is arranged at the end of the robotic arm;

[0280] The third calibration image acquisition module is used to acquire a third calibration image of the second calibration board captured by the first camera;

[0281] The second generation module is used to calibrate the coordinate system of the first camera and the base coordinate system according to the second calibration image and the third calibration image, so as to obtain a first coordinate system conversion relationship.

[0282] In one embodiment, the second generation module includes:

[0283] The singular value decomposition unit is used to perform singular value decomposition on the second calibration image and the third calibration image to obtain a first conversion relationship between the coordinate system where the second calibration image is located and the coordinate system where the third calibration image is located;

[0284] The second conversion relationship acquisition unit is used to acquire a second conversion relationship between the coordinate system of the first camera and the coordinate system where the third calibration image is located;

[0285] The third conversion relationship acquisition unit is used to acquire a third conversion relationship between the coordinate system where the second calibration image is located and the base coordinate system;

[0286] The second generation unit is used to obtain a first coordinate system conversion relationship according to the first conversion relationship, the second conversion relationship and the third conversion relationship.

[0287] In one embodiment, the medical imaging model construction device further includes:

[0288] The registration module is used to register the coordinate system of the medical imaging device and the base coordinate system of the robotic arm to obtain a second coordinate system conversion relationship.

[0289] The re-registration module is used to re-register the coordinate system of the medical imaging device and the base coordinate system of the robotic arm when the position of the robotic arm changes, so as to obtain a new second coordinate system conversion relationship;

[0290] A correction transformation relationship determination module, configured to determine a correction transformation relationship according to a new second coordinate system transformation relationship and a second coordinate system transformation relationship;

[0291] A correction module, configured to correct the aperture model of a medical imaging device according to the correction transformation relationship to obtain a new aperture model of the medical imaging device.

[0292] In an exemplary embodiment, as Figure 23 shown, a structure model construction device is provided, including: a second acquisition module 31, a third acquisition module 32, and a second construction module 33, where:

[0293] The second acquisition module 31 is configured to acquire a point cloud image captured by a second camera; the point cloud image includes a target area of a captured object.

[0294] The third acquisition module 32 is configured to acquire a scan image of a medical imaging device; the scan image includes a lesion area of a captured object; the lesion area is a partial area within the target area.

[0295] The second construction module 33 is configured to construct a structure model of a captured object according to the point cloud image and the scan image.

[0296] In one embodiment, the second construction module 33 includes:

[0297] A point cloud model construction unit, configured to construct a point cloud model of a captured object according to a target image;

[0298] A scan model construction unit, configured to construct a scan model of a captured object according to the scan image;

[0299] A splicing unit, configured to splice the point cloud model and the scan model to obtain a structure model.

[0300] In one embodiment, after acquiring the target image captured by the second camera, the structure model construction device further includes:

[0301] A moving module, configured to control the bed body of the medical imaging device to drive the captured object to move to the scan area of the medical imaging device;

[0302] The third acquisition module 32 includes:

[0303] A scan unit, configured to start the medical imaging device to scan the scan area to obtain a scan image of the medical imaging device.

[0304] In one embodiment, the structure model construction device further includes:

[0305] A detection module, configured to detect a binding device of a captured object to obtain a detection result;

[0306] The second acquisition module 31 includes:

[0307] A second acquisition unit, configured to obtain a target image captured by a second camera when the detection result indicates successful detection.

[0308] In one embodiment, the detection module includes:

[0309] A first detection unit, configured to detect an identifier on a bound device and determine whether the bound device includes a preset number of identifiers;

[0310] A second detection unit, configured to, if the bound device includes a preset number of identifiers, determine a first identifier and a second identifier on the bound device, and perform detection based on the distance between the first identifier and the second identifier to obtain a detection result;

[0311] A third detection unit, configured to, if the bound device does not include a preset number of identifiers, determine that the detection result indicates detection failure.

[0312] In one embodiment, the first detection unit includes:

[0313] An image acquisition subunit of the bound device, configured to obtain an image of the bound device captured by a second camera;

[0314] An identifier quantity recognition subunit, configured to recognize the quantity of identifiers in the image of the bound device to obtain a recognition result;

[0315] A first detection subunit, configured to determine whether the bound device includes a preset number of identifiers according to the recognition result.

[0316] In one embodiment, the second detection unit includes:

[0317] A second detection subunit, configured to determine an elastic member on the bound device, and determine the identifiers provided on both sides of the elastic member as the first identifier and the second identifier.

[0318] In one embodiment, the second detection unit includes:

[0319] A detection success subunit, configured to, if the distance between the first identifier and the second identifier is greater than a preset threshold, determine that the detection result indicates successful detection;

[0320] A detection failure subunit, configured to, if the distance between the first identifier and the second identifier is not greater than a preset threshold, determine that the detection result indicates detection failure.

[0321] In an exemplary embodiment, such as Figure 24As shown, a device for planning an obstacle avoidance path is provided, including: an aperture model construction module 41, a structure model construction module 42, and an obstacle avoidance path determination module 43, where:

[0322] The aperture model construction module 41 is configured to construct an aperture model of a medical imaging device according to the above device.

[0323] The structure model construction module 42 is configured to construct a structure model according to the above device.

[0324] The obstacle avoidance path determination module 43 is configured to determine an obstacle avoidance path of the robotic arm during the scanning of the medical imaging device according to the aperture model, the structure model, and the size of the robotic arm.

[0325] Each module in the above device for constructing a medical imaging model can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0326] In an exemplary embodiment, a computer device is provided. The computer device can be a terminal, and its internal structure diagram can be as Figure 25 shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. The computer program, when executed by the processor, implements a method for constructing a medical imaging model. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0327] Those skilled in the art can understand,Figure 25 The structure shown is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0328] In one embodiment, a computer device is further provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0329] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0330] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0331] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0332] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0333] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered to be within the scope described in this specification.

[0334] The above embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for constructing a medical imaging model, characterized in that, The method is applied to a scanning system, which includes a medical imaging device and a robotic arm. The method includes: Obtaining an image of the medical imaging device collected by a first camera; the first camera is used to capture a complete image of the medical imaging device; Constructing an aperture model of the medical imaging device according to the medical imaging device image, a first coordinate system conversion relationship, and a second coordinate system conversion relationship; the first coordinate system conversion relationship is the corresponding relationship between the coordinate system of the first camera and the base coordinate system of the robotic arm; the second coordinate system conversion relationship is the corresponding relationship between the base coordinate system and the medical imaging coordinate system.

2. The method according to claim 1, characterized in that, The medical imaging device image includes a first depth image, and the first depth image includes a first depth image outside the aperture and a first depth image inside the aperture. The first camera is arranged at the end of the robotic arm. The obtaining of the image of the medical imaging device collected by the first camera includes: Obtaining the image of the medical imaging device collected by the first camera at a first position to obtain the first depth image outside the aperture; the first position is within the outer region of the scanning hole of the medical imaging device; Obtaining the image of the medical imaging device collected by the first camera at a second position to obtain the first depth image inside the aperture; the second position is within the inner region of the scanning hole.

3. The method according to claim 2, characterized in that, The constructing of the aperture model of the medical imaging device according to the medical imaging device image, the first coordinate system conversion relationship, and the second coordinate system conversion relationship includes: Constructing an initial model of the medical imaging device according to the initial size of the medical imaging device and the size of the connection structure; Constructing an inner and outer aperture model of the medical imaging device according to the first depth image; Constructing the aperture model of the medical imaging device according to the first coordinate system conversion relationship, the second coordinate system conversion relationship, the initial model of the medical imaging device, and the inner and outer aperture model.

4. The method according to claim 3, characterized in that, The constructing of the aperture model of the medical imaging device according to the first coordinate system conversion relationship, the second coordinate system conversion relationship, the initial model of the medical imaging device, and the inner and outer aperture model includes: Performing coordinate system conversion on the inner and outer aperture model according to the first coordinate system conversion relationship to obtain a first converted inner and outer aperture model; Performing coordinate system conversion on the first converted inner and outer aperture model according to the second coordinate system conversion relationship to obtain a second converted inner and outer aperture model; Obtaining the aperture model of the medical imaging device according to the second converted inner and outer aperture model and the initial model of the medical imaging device.

5. The method according to claim 2, characterized in that, The constructing of the aperture model of the medical imaging device according to the medical imaging device image, the first coordinate system conversion relationship, and the second coordinate system conversion relationship includes: Constructing an inner and outer aperture model of the medical imaging device according to the first depth image; Determining the aperture edge according to the inner and outer aperture model, and performing point cloud expansion according to the aperture edge to obtain an intermediate aperture model of the medical imaging device; Construct the aperture model of the medical imaging device according to the first coordinate system conversion relationship, the second coordinate system conversion relationship, and the intermediate aperture model.

6. The method according to claim 1, characterized in that, The medical imaging device image includes a second depth image. Constructing the aperture model of the medical imaging device according to the medical imaging device image, the first coordinate system conversion relationship, and the second coordinate system conversion relationship includes: Generate a point cloud image of the medical imaging device according to the second depth image; Perform coordinate system conversion on the point cloud image according to the first coordinate system conversion relationship to obtain the point cloud image in the base coordinate system; Perform model construction according to the second coordinate system conversion relationship and the point cloud image in the base coordinate system to obtain the aperture model of the medical imaging device.

7. The method according to claim 6, characterized in that, Performing model construction according to the second coordinate system conversion relationship and the point cloud image in the base coordinate system to obtain the aperture model of the medical imaging device includes: Expand the point cloud image in the base coordinate system along the central axis of the aperture of the medical imaging device in the direction of entering the aperture to obtain the target point cloud image in the base coordinate system; Perform model construction according to the target point cloud image to obtain the constructed point cloud model; Perform medical imaging coordinate system conversion on the point cloud model according to the second coordinate system conversion relationship to obtain the aperture model of the medical imaging device.

8. The method according to claim 1, characterized in that, The method further includes: Register the coordinate system of the medical imaging device and the base coordinate system of the robotic arm to obtain the second coordinate system conversion relationship; When the position of the robotic arm changes, re-register the coordinate system of the medical imaging device and the base coordinate system of the robotic arm to obtain a new second coordinate system conversion relationship; Determine a correction conversion relationship according to the new second coordinate system conversion relationship and the second coordinate system conversion relationship; Correct the aperture model of the medical imaging device according to the correction conversion relationship to obtain a new aperture model of the medical imaging device.

9. A method for constructing a structural model, characterized in that,The method is applied to a scanning system, the scanning system includes a medical imaging device and a robotic arm, and the method includes: Obtain a target image captured by a second camera; the target image includes a target area of a subject; Control the bed of the medical imaging device to drive the subject to move to the scanning area of the medical imaging device; Start the medical imaging device to scan the scanning area to obtain a scanning image of the medical imaging device; the scanning image includes a lesion area of the subject; the lesion area is a partial area within the target area; Construct a structural model of the subject according to the target image and the scanning image.

10. The method according to claim 9, characterized in that, The method further includes: Detect a binding device of the subject to obtain a detection result; The obtaining the target image captured by the second camera includes: Obtain the target image captured by the second camera when the detection result indicates successful detection.

11. The method according to claim 10, characterized in that, The detecting the binding device of the subject to obtain a detection result includes: Detect the identifiers on the binding device to determine whether the binding device includes a preset number of identifiers; If the binding device includes a preset number of identifiers, determine the first identifier and the second identifier on the binding device, and perform detection based on the distance between the first identifier and the second identifier to obtain the detection result; If the binding device does not include a preset number of identifiers, determine that the detection result indicates a detection failure.

12. A method for planning an obstacle avoidance path, characterized in that, The method is applied to a scanning system, and the scanning system includes a medical imaging device and a robotic arm. The method includes: Construct an aperture model of the medical imaging device according to the method described in any one of claims 1-8; Construct a structure model according to the method described in any one of claims 9-11; Determine an obstacle avoidance path of the robotic arm during the scanning process of the medical imaging device according to the aperture model, the structure model, and the dimensions of the robotic arm.

13. An apparatus for constructing a medical image model, characterized in that, The device includes: A first acquisition module, configured to acquire a medical imaging device image collected by a first camera; the first camera is used to capture a complete medical imaging device image; A first construction module, configured to construct an aperture model of the medical imaging device according to the medical imaging device image, a first coordinate system conversion relationship, and a second coordinate system conversion relationship; the first coordinate system conversion relationship is the corresponding relationship between the coordinate system of the first camera and the base coordinate system of the robotic arm; the second coordinate system conversion relationship is the corresponding relationship between the base coordinate system and the medical imaging coordinate system.

14. An apparatus for constructing a structure model, characterized in that, The device includes: A second acquisition module, configured to acquire a target image captured by a second camera; the target image includes a target area of a subject; A movement module, configured to control the bed of the medical imaging device to drive the subject to move to the scanning area of the medical imaging device; A third acquisition module, configured to start the medical imaging device to scan the scanning area to obtain a scanning image of the medical imaging device; the scanning image includes a lesion area of the subject; the lesion area is a partial area within the target area; A second construction module, configured to construct a structure model of the subject according to the target image and the scanning image.

15. An apparatus for planning an obstacle avoidance path, characterized in that, The device includes: An aperture model construction module, configured to construct an aperture model of the medical imaging device according to the device described in claim 13; A structure model construction module, configured to construct a structure model according to the device described in claim 14; An obstacle avoidance path determination module, configured to determine an obstacle avoidance path of the robotic arm during the scanning process of the medical imaging device according to the aperture model, the structure model, and the dimensions of the robotic arm.

Images