Object three-dimensional model data acquisition method and device and image processing equipment

Abstract

The invention is suitable for the technical field of medical treatment, and provides a three-dimensional model data acquisition method and device of an object and image processing equipment. The method for acquiring the three-dimensional model data of the object comprises the following steps: acquiring N groups of images obtained by respectively shooting the object at N different shooting heights, each group of images comprising a plurality of images shot at different shooting angles of view, and N being a positive integer greater than 1; determining N first point cloud models in one-to-one correspondence with the N shooting heights according to the N groups of images; and determining first three-dimensional model data of the object according to the intersection of the N first point cloud models. According to the embodiment of the invention, the detail reproduction degree of the three-dimensional model can be improved.

Description

Technical Field

[0001] The present application belongs to the field of medical technology, and in particular relates to a method, device and image processing equipment for acquiring three-dimensional model data of an object. Background Art

[0002] A 3D model is a polygonal representation of an object, and is currently used in medical, gaming, transportation and other technical fields. Commonly used 3D reconstruction is to select image data from five specific views (i.e., top view, front view, left view, back view, and right view) for 3D reconstruction. For objects with fine and complex structures, the degree of detail reproduction of the reconstructed 3D model is relatively low. Summary of the invention

[0003] The embodiments of the present application provide a method, device and image processing equipment for acquiring three-dimensional model data of an object, which can improve the degree of detail reproduction of the three-dimensional model.

[0004] A first aspect of an embodiment of the present application provides a method for acquiring three-dimensional model data of an object, comprising: acquiring N groups of images by photographing the object at N different shooting heights, each group of images including multiple images taken at different shooting angles, wherein N is a positive integer greater than 1; determining, based on the N groups of images, N first point cloud models corresponding one-to-one to the N shooting heights; and determining, based on the intersection of the N first point cloud models, first three-dimensional model data of the object.

[0005] In some embodiments of the first aspect, each group of images in the N groups of images includes M images taken from M different shooting perspectives, where M is a positive integer greater than 1; in the operation of determining N first point cloud models corresponding one-to-one to the N shooting heights based on the N groups of images, the operation of determining the first point cloud model corresponding to the i-th shooting height, 1≤i≤N, includes: determining M visual cones corresponding one-to-one to the M different shooting perspectives based on the M images corresponding to the i-th shooting height; and determining the first point cloud model of the i-th shooting height based on the intersection of the M visual cones.

[0006] In some embodiments of the first aspect, each of the N groups of images includes M images taken from M different shooting angles, where M is a positive integer greater than 1;

[0007] In the operation of determining N first point cloud models corresponding one-to-one to the N shooting heights based on the N groups of images, the operation of determining the first point cloud model corresponding to the i-th shooting height, 1≤i≤N, includes: determining a disparity map of the i-th shooting height based on the M images corresponding to the i-th shooting height; and determining the first point cloud model of the i-th shooting height based on the disparity map.

[0008] In some implementations of the first aspect, in the operation of acquiring N groups of images obtained by photographing the object at N different photographing heights, and determining N first point cloud models corresponding to the N photographing heights according to the N groups of images, the operation of determining the first point cloud model corresponding to the j-th photographing height, 1≤j≤N, includes: acquiring the first point cloud model corresponding to the j-th photographing height from M 0 M taken from different shooting angles 0 images, M 0 >1; according to the M corresponding to the j-th shooting height 0 The first point cloud model to be verified at the j-th shooting height is determined; the first point cloud model to be verified is verified for data accuracy; if the data accuracy verification fails, the n-th point cloud model to be verified is verified for data accuracy, until the data accuracy verification of the M-th point cloud model to be verified is successful, and the point cloud model with successful data accuracy verification is used as the first point cloud model at the j-th shooting height, wherein the n-th point cloud model to be verified is the M-th point cloud model corresponding to the j-th shooting height. 0 +n-1 images, the M 0 +n-1 images corresponding to the M 0 Different shooting angles and M 0 There are n-1 shooting angles with different shooting angles, where n and M are both positive integers greater than 1; n≤M.

[0009] In some embodiments of the first aspect, data accuracy verification is performed on the pth point cloud model to be verified, p≥1, including: obtaining the p+1th point cloud model to be verified; and determining that the data accuracy verification has failed when the data error between the pth point cloud model to be verified and the p+1th point cloud model to be verified is greater than or equal to a preset threshold.

[0010] In some implementations of the first aspect, the point corresponding to the newly added shooting angle of view is located at a midpoint between any two preset points corresponding to the shooting angles on the guide rail.

[0011] In some embodiments of the first aspect, the data error includes: a volume difference between the p-th point cloud model to be verified and the p+1-th point cloud model to be verified, and / or a difference in the number of point clouds between the p-th point cloud model to be verified and the p+1-th point cloud model to be verified.

[0012] In some embodiments of the first aspect, in the operation of determining M visual cones corresponding one-to-one to the M different shooting angles according to the M images corresponding to the i-th shooting height, determining the visual cone corresponding to the k-th shooting angle, 1≤k≤M, includes: extracting a first contour line of the object in the image of the k-th shooting angle; if the object has an occluded area, fitting the contour line in the occluded area according to the first contour line to obtain a second contour line of the object; and determining the visual cone corresponding to the k-th shooting angle based on the second contour line.

[0013] In some embodiments of the first aspect, after determining the first three-dimensional model data of the object based on the intersection of the N first point cloud models, it also includes: performing data accuracy verification on the first three-dimensional model data; if the data accuracy verification fails, obtaining a new image obtained by photographing the object at a new shooting height, and updating the first three-dimensional model data with the new image.

[0014] In some implementations of the first aspect, the newly added shooting height is half of the sum of any two shot heights.

[0015] In some embodiments of the first aspect, at least one camera is controlled to move to N groups of preset points on the guide rail, and the N groups of images obtained by the at least one camera respectively photographing the object at the N groups of preset points are obtained, wherein the N groups of preset points correspond one-to-one to N different shooting heights; or, the N groups of images obtained by the N groups of cameras respectively photographing the object are obtained, and the N groups of cameras are fixedly set at N groups of preset points, wherein the N groups of preset points correspond one-to-one to N different shooting heights.

[0016] In some embodiments of the first aspect, determining the first three-dimensional model data of the object based on the intersection of the N first point cloud models includes: determining the offset between the reference coordinate system of each shooting height and the reference height based on the height difference between each shooting height and the reference height; adjusting the corresponding first point cloud model according to the offset corresponding to each shooting height to convert the N first point cloud models to the reference coordinate system; and obtaining the intersection of the N first point cloud models in the reference coordinate system to obtain the first three-dimensional model data.

[0017] In some embodiments of the first aspect, the object is a surgical instrument; and the first three-dimensional model data is called for surgical navigation of a surgery using the surgical instrument.

[0018] A second aspect of an embodiment of the present application provides a device for acquiring three-dimensional model data of an object, comprising: an image acquisition unit, used to acquire N groups of images by photographing the object at N different shooting heights, each group of images including multiple images taken at different shooting angles, wherein N is a positive integer greater than 1; a point cloud model determination unit, used to determine, based on the N groups of images, N first point cloud models corresponding one-to-one to the N shooting heights; and a three-dimensional model data determination unit, used to determine first three-dimensional model data of the object based on the intersection of the N first point cloud models.

[0019] A third aspect of an embodiment of the present application provides an image processing device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method for acquiring three-dimensional model data of the object when executing the computer program.

[0020] A fourth aspect of an embodiment of the present application provides an image processing system, comprising an image processing device and a camera, wherein the camera is used to acquire images, and the image processing device is used to execute the steps of the above-mentioned method for acquiring three-dimensional model data of the object.

[0021] A fifth aspect of an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps of the method for acquiring three-dimensional model data of the above-mentioned object are implemented.

[0022] A sixth aspect of an embodiment of the present application provides a computer program product. When the computer program product runs on an image processing device, the image processing device executes the steps of the above-mentioned method for acquiring three-dimensional model data of the object.

[0023] In an embodiment of the present application, N groups of images are obtained by photographing the object at N different shooting heights, N first point cloud models corresponding to the N shooting heights are determined, and the first three-dimensional model data of the object is determined based on the intersection between the N first point cloud models. Since each group of images includes multiple images taken at different shooting angles, and N is a positive integer greater than 1, it helps to reduce the problem of the accuracy of the three-dimensional model data being affected by blind spots in the field of view at a specific shooting height or a specific shooting angle, thereby improving the degree of detail reproduction of the three-dimensional model. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0025] Figure 1 is a schematic diagram of the structure of an image processing system provided in an embodiment of the present application;

[0026] Figure 2 is a schematic diagram of a specific structure of an image processing system provided in an embodiment of the present application;

[0027] Figure 3 is a schematic diagram of a specific structure of an image processing system using multiple cameras provided in an embodiment of the present application;

[0028] Figure 4 It is a schematic diagram of the implementation flow of a method for acquiring three-dimensional model data of an object provided in an embodiment of the present application;

[0029] Figure 5 This is a schematic diagram of a first specific implementation flow of step S402 provided in an embodiment of the present application;

[0030] Figure 6 It is a schematic diagram of a second specific implementation flow of step S402 provided in an embodiment of the present application;

[0031] Figure 7 It is a structural schematic diagram of a device for acquiring three-dimensional model data of an object provided in an embodiment of the present application;

[0032] Figure 8 It is a structural schematic diagram of the image processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work are protected by the present application.

[0034] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, wholes, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or combinations thereof.

[0035] In the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0036] References to "one embodiment" or "some embodiments" etc. described in the specification of this application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0037] In the related art, three-dimensional reconstruction is mostly performed by fixing image data of five specific views (i.e., top view, front view, left view, rear view, and right view) for three-dimensional reconstruction. For objects with finer and more complex structures, the degree of detail reproduction of the reconstructed three-dimensional model is relatively low.

[0038] In view of this, the present application proposes a method for acquiring three-dimensional model data of an object, which can acquire three-dimensional model data of an object based on images of different shooting heights and different shooting angles, thereby improving the degree of detail reproduction of the three-dimensional model.

[0039] In order to illustrate the technical solution of the present application, a specific embodiment is provided below for illustration.

[0040] Please refer to Figure 1 , Figure 1 An image processing system provided by the present application is shown. The image processing system may include a camera and an image processing device.

[0041] Among them, the camera can be used to collect images of objects. In the implementation of the present application, the camera can be a charge coupled device (CCD) camera, a complementary metal oxide semiconductor (CMOS) camera, or other types of optical cameras. In order to facilitate the subsequent acquisition of point cloud data, the camera can also be a depth camera. This application does not impose any restrictions on this.

[0042] The image processing device can obtain the 3D model data of the object based on the image captured by the camera. It is understandable that the 3D model data of the object can be used for 3D reconstruction to obtain the 3D model of the object.

[0043] In the implementation manner of the present application, the three-dimensional model data of the object can be applied in different fields. For example, when the object is a surgical instrument, the staff can call the three-dimensional model data during surgical navigation or instrument maintenance to complete the three-dimensional reconstruction of the surgical instrument. For another example, when the object is a vehicle, the staff can call the three-dimensional model data during traffic management or automobile testing to complete the three-dimensional reconstruction of the vehicle. This application does not limit the use scenarios and methods of the three-dimensional model data.

[0044] In some embodiments, the image processing system may include a cylindrical box, which may be used to fix the object. Meanwhile, the camera may be disposed on the cylindrical box. Specifically, in some embodiments, a fixed bracket may be disposed inside the cylindrical box, which may be used to fix the object to be photographed. To facilitate subsequent image processing, the cylindrical box and the fixed bracket may be designed with special colors so that the outline of the object can be extracted based on the color.

[0045] In the implementation of the present application, the image processing device needs to photograph the object at N (N is a positive integer greater than 1) shooting heights to obtain N groups of images, and each group of images includes multiple images taken at different shooting angles. Figure 2 The camera can be fixed on a circular rail on the inner wall of the cylindrical box that can be raised and lowered (i.e., the shooting height can be adjusted). Several preset points can be evenly distributed on the rail, allowing the camera to move around the rail and shoot images at the preset points. Each preset point can be used as the position point where the camera is located when shooting an image.

[0046] In the implementation of the present application, the number of cameras may be one or more. For example, Figure 3 The case where the number of cameras is 4 is shown. It can be understood that, relative to the use of a single camera, if the number of cameras is k (k is a positive integer greater than 1), the efficiency of the camera in acquiring images will be increased by k times. Accordingly, the image processing device can control at least one camera to move to N groups of preset points on the guide rail. Among them, the N groups of preset points correspond to N different shooting heights one by one, and each group of preset points corresponds to a shooting height, so that the image processing device can obtain N groups of images obtained by at least one camera shooting the object at N groups of preset points.

[0047] As a special example, the number of cameras can be two, and the two cameras can form a binocular camera. In this case, the image processing device can control the binocular cameras to move synchronously on the guide rail. The so-called synchronous movement means that the relative position relationship between the binocular cameras is maintained during movement, so that the process of repeated calibration of the binocular cameras can be reduced.

[0048] As another special example, the number of cameras may be M. Where M>1, indicating the number of shooting angles required for a single shooting angle. The M cameras may be arranged at the same height of the inner wall of the cylindrical box. In this case, the guide rail only needs to support movement in the height direction to obtain N sets of images.

[0049] In some other embodiments of the present application, N groups of preset points can be fixedly set on the inner wall of the cylindrical box, and N groups of cameras can be fixed on the N groups of preset points. Similarly, the N groups of preset points correspond to N different shooting heights one by one. For example, N×M cameras can be set at N×M preset points. At this time, the image processing device can obtain N groups of images obtained by N groups of cameras respectively photographing the object, so as to obtain N groups of images in a single shot. Moreover, at this time, the inner wall of the cylindrical box may not need guide rails, motors, or other devices for controlling the movement of cameras or guide rails.

[0050] Please refer to Figure 4 , Figure 4 A schematic diagram of the implementation flow of a method for acquiring three-dimensional model data of an object provided in an embodiment of the present application is shown. The method can be applied to an image processing device and is suitable for situations where the degree of detail reproduction of the three-dimensional model needs to be improved.

[0051] Among them, the above-mentioned image processing device can be a smart device such as a computer or a mobile phone, or it can be a device specifically used for image processing in a medical system, such as an image processing device in an endoscope system, and this application does not impose any restrictions on this.

[0052] Specifically, the method for acquiring the three-dimensional model data of the above object may include the following steps S401 to S403.

[0053] Step S401 , obtaining N groups of images by photographing the object at N different photographing heights.

[0054] Wherein, N is a positive integer greater than 1, indicating the total number of shooting angles. Each set of images includes multiple images shot at different shooting angles.

[0055] It should be understood that the process of obtaining N sets of images can refer to Figures 1 to 3 The relevant description of this application will not be repeated here.

[0056] Step S402: Determine N first point cloud models corresponding to the N shooting heights one by one according to the N groups of images.

[0057] Specifically, based on a group of images captured at a single shooting angle, a first point cloud model corresponding to the shooting angle can be determined using multiple images captured at different shooting angles included in the group. N groups of images are processed separately to obtain N first point cloud models corresponding to N shooting heights.

[0058] Step S403: determining first three-dimensional model data of the object according to the intersection of the N first point cloud models.

[0059] In an implementation manner of the present application, the first three-dimensional model data of the object can be determined by finding the intersection of N first point cloud models, so that the first three-dimensional model data can be combined with point cloud information at different shooting heights, and the point cloud information at each shooting perspective refers to the image information at different shooting perspectives. Therefore, the first three-dimensional model data can reflect the contours of the object observed at different shooting heights and different shooting perspectives.

[0060] In an embodiment of the present application, N groups of images are obtained by photographing the object at N different shooting heights, N first point cloud models corresponding to the N shooting heights are determined, and the first three-dimensional model data of the object is determined based on the intersection between the N first point cloud models. Since each group of images includes multiple images taken at different shooting angles, and N is a positive integer greater than 1, it helps to reduce the problem of the accuracy of the three-dimensional model data being affected by blind spots in the field of view at a specific shooting height or a specific shooting angle, thereby improving the degree of detail reproduction of the three-dimensional model.

[0061] In step S402, the image processing device may use different image processing algorithms to obtain a first point cloud model.

[0062] Specifically, Figure 5 As shown, in some embodiments of the present application, in the step S402, the operation of determining the first point cloud model corresponding to the i-th shooting height (1≤i≤N, i.e., any shooting height) may include the following steps S501 to S502.

[0063] Step S501, determining M viewing cones corresponding to M different shooting angles one by one according to M images corresponding to the i-th shooting height.

[0064] Among them, the viewing cone represents the cone formed by the visible space of the camera in the shooting angle, that is, the three-dimensional cone formed by the preset points and the object contour.

[0065] Specifically, in step S501, determining the viewing cone corresponding to the k-th shooting angle (1≤k≤M, i.e., any shooting angle) may include: extracting the first contour line of the object in the image of the k-th shooting angle. If the object does not have an obstructed area, determining the viewing cone corresponding to the k-th shooting angle based on the first contour line. If the object has an obstructed area, fitting the contour line in the obstructed area according to the first contour line to obtain the second contour line of the object, and determining the viewing cone corresponding to the k-th shooting angle based on the second contour line.

[0066] Among them, contour extraction can be achieved by threshold segmentation, deep learning, etc., which is not limited in this application. In the aforementioned image processing system, since the background and the fixed bracket of the cylindrical box are designed with special colors, the image can be cut out based on the special color to obtain the first contour of the object.

[0067] After obtaining the first contour line, it is possible to detect whether the object is obscured based on the integrity of the first contour line. If there is an obscured area of ​​the object, the contour line in the obscured area can be fitted according to the first contour line to obtain a complete second contour line. The contour line fitting method can adopt a line fitting method such as global fitting and local fitting. Exemplarily, in some embodiments of the present application, the contour line in the obscured area can be fitted by quadratic Bezier curve interpolation. The Bezier curve formula can be expressed as:

[0068] B(t)=(1-t 2 ) 0 +2t(1-t)P 1 +t 2 P 2 , t∈[0,1];

[0069] Among them, P 0 , P 1 , P 2 Represent the starting point, control point and end point respectively. The three points may include two points on the first contour line distributed on the front side of the occluded area, and one point on the first contour line distributed on the back side of the occluded area; or, may include one point on the first contour line distributed on the front side of the occluded area and two points on the first contour line distributed on the back side of the occluded area. Preferably, the three points may be the three extreme points on the first contour line that are closest to the occluded area.

[0070] Step S502: determining a first point cloud model at the i-th shooting height according to the intersection of the M viewing cones.

[0071] In the implementation of the present application, each viewing cone can be represented as a set of points of the object, and the first point cloud model of the i-th shooting height can be obtained by intersecting the M viewing cones. Similarly, the first point cloud model can also be represented as a set of points of the object.

[0072] Understandably, Figure 5 The method shown can be implemented based on one camera or multiple cameras, and therefore has high scene adaptability.

[0073] like Figure 6 As shown, in some other embodiments of the present application, in the step S402, the operation of determining the first point cloud model corresponding to the i-th shooting height (1≤i≤N, i.e., any shooting height) may include the following steps S601 to S602.

[0074] Step S601 : determining a disparity map of the ith shooting height according to M images corresponding to the ith shooting height.

[0075] In the implementation of the present application, M images can be collected by the aforementioned binocular camera, and stereo matching can be performed based on the M images to calculate the disparity to obtain a disparity map. The disparity value of the pixel at the corresponding position of the binocular camera can be stored at each position in the disparity map.

[0076] Step S602: determining a first point cloud model at the i-th shooting height based on the disparity map.

[0077] After obtaining the disparity map, it can be converted into a depth map based on the camera parameters. The depth value of the point at the corresponding position can be saved at each position in the depth map. The first point cloud data of the object can be obtained based on the depth value. Similarly, the first contour line of the object can be extracted based on the image. If there is no obscured area of ​​the object, the first point cloud model of the i-th shooting height is determined based on the depth value of the first contour line point. If there is an obscured area of ​​the object, the contour line in the obscured area is fitted according to the first contour line to obtain the second contour line of the object, and the first point cloud model of the i-th shooting height is determined based on the depth value of the point on the second contour line.

[0078] In the implementation of the present application, the first point cloud model corresponding to the shooting height is determined at each shooting angle, and N first point cloud models can be obtained. At this time, in step S402, in order to obtain the first three-dimensional model data, the N first point cloud models can be converted to the same coordinate system to obtain the intersection and obtain the first three-dimensional model data of the object.

[0079] Specifically, step S402 may include: determining the offset between the reference coordinate system of each shooting height and the reference height according to the height difference between each shooting height and the reference height. Then, adjusting the corresponding first point cloud models according to the offset corresponding to each shooting height, so as to convert the N first point cloud models to the reference coordinate system. Then, the intersection between the N first point cloud models may be obtained in the reference coordinate system to obtain the first three-dimensional model data.

[0080] Assuming that the height difference between the shooting height of the plane where the guide rail is located and the reference plane is δz, then the angle between the preset point on the guide rail and the reference plane is α=actan(δz / r), r is the radius of the guide rail, and the camera on the guide rail needs to be offset by α to adjust the shooting angle to the center of the box. Similarly, the first point cloud model generated on the guide rail needs to adjust the offset α in the opposite direction so that the coordinate system of the first point cloud model is consistent with the reference coordinate system. In this way, multiple groups of first point cloud models can be transformed into the same coordinate system through simple geometric transformation, eliminating the need for repeated calibration. In addition, in some embodiments, since the first point cloud models at different shooting heights can be processed separately, the image processing device can use parallel computing to obtain the first point cloud models at different shooting heights, which helps to further improve computing efficiency.

[0081] In the implementation of the present application, the process of obtaining the intersection can be implemented by a surface intersection algorithm. In the process of surface intersection, the first three-dimensional model data can be described as a visual shell. The mathematical description of the visual shell is defined as follows:

[0082]

[0083] Where N is the number of shooting heights, C i represents the camera position, V is the viewing frustum formed by the preset point position and the object contour, and S is the result of the intersection of the N first point cloud models, which is approximately the visible shell of the object.

[0084] It can be understood that when the total number N×M of preset points is greater, a more refined visible shell can be obtained. In order to ensure the accuracy of the first three-dimensional model data, the total number of preset points in the present application is preferably greater than or equal to 5.

[0085] In order to obtain a more appropriate value of N×M, the present application also provides a method for adaptively obtaining the value of N×M.

[0086] Specifically, when determining the first point cloud model at j shooting heights (1≤j≤N, i.e., any shooting height), the image processing device may obtain the first point cloud model at the j-th shooting height from M 0 M taken from different shooting angles 0 images. Among them, M0 is the default total number of shooting angles, M 0 >1. According to the M corresponding to the jth shooting height 0 The first point cloud model to be verified at the jth shooting height can be determined by using the image. The method for determining the point cloud model to be verified can refer to Figure 5 or Figure 6 The description of the corresponding embodiments is not repeated in this application.

[0087] Subsequently, the image processing device can perform data accuracy verification on the first point cloud model to be verified to analyze whether the first point cloud model to be verified meets the data accuracy requirements. If the data accuracy verification is successful, it means that the M 0 If the point cloud model angles obtained from images with different shooting angles meet the accuracy requirements, the point cloud model to be verified that has successfully verified the data accuracy can be used as the first point cloud model at the i-th shooting height. At this time, there is no need to add new images.

[0088] If the data accuracy verification fails, the data accuracy verification is performed on the nth point cloud model to be verified until the data accuracy verification of the Mth point cloud model to be verified succeeds. The point cloud model that succeeds in the data accuracy verification is used as the first point cloud model of the jth shooting height.

[0089] Among them, the nth point cloud model to be verified is M corresponding to the jth shooting height 0 +n-1 images, the M 0 +n-1 images correspond to M 0 Different shooting angles and M 0 n-1 shooting angles with different shooting angles. n and M are both positive integers greater than 1; n≤M. In addition, the value of n gradually increases from 2. Each time n is increased by 1, a data accuracy check can be performed until the data accuracy check of the Mth point cloud model to be checked is successful, and the value of n is M.

[0090] In other words, if the data accuracy verification of the first point cloud model to be verified fails, a new shooting angle can be added and the corresponding new image can be obtained, that is, the Mth image at the jth shooting height can be obtained. 0 +1 shooting angle corresponding to the Mth 0 +1 image, based on the M corresponding to the jth shooting height 0+1 image, obtain the second point cloud model to be verified, and perform data accuracy verification on the second point cloud model to be verified. If the data accuracy verification of the second point cloud model to be verified is successful, the point cloud model with successful data accuracy verification can be used as the first point cloud model of the jth shooting height. At this time, M=2. If the data accuracy verification fails, you can add a new shooting angle and obtain the corresponding new image again. According to the M corresponding to the jth shooting height, 0 +2 images, obtain the third point cloud model to be verified, and perform data accuracy verification on the third point cloud model to be verified. And so on, continue to add one to the value of n and perform data accuracy verification until there is a point cloud model with successful data accuracy verification, and then the point cloud model with successful data accuracy verification can be used as the first point cloud model at the jth shooting height.

[0091] In the implementation manner of the present application, the method for performing data accuracy verification can be selected according to the actual situation. In some implementation manners of the present application, when performing data accuracy verification on the pth (p≥1) point cloud model to be verified, the p+1th point cloud model to be verified can be obtained, and when the data error between the pth point cloud model to be verified and the p+1th point cloud model to be verified is greater than or equal to the preset threshold, it indicates that the newly added image is of great help in improving the accuracy of the point cloud model, and it is determined that the data accuracy verification has failed.

[0092] For example, when performing data accuracy verification on the first point cloud model to be verified, the second point cloud model to be verified can be obtained, and when the data error between the first point cloud model to be verified and the second point cloud model to be verified is greater than or equal to a preset threshold, it can be determined that the data accuracy verification has failed.

[0093] Among them, the data error may include but is not limited to: the volume difference between the pth point cloud model to be verified and the p+1th point cloud model to be verified, and / or the difference in the number of point clouds between the pth point cloud model to be verified and the p+1th point cloud model to be verified.

[0094] When the data error between the pth point cloud model to be verified and the p+1th point cloud model to be verified is less than the preset threshold, it means that the newly added image is of little help in improving the accuracy of the point cloud model, and it can be determined that the data accuracy verification is successful.

[0095] It should be noted that, since the p+1th point cloud model has been obtained when the data accuracy of the pth point cloud model to be verified is verified, if the pth point cloud model to be verified is verified successfully, it can be confirmed that both the pth point cloud model to be verified and the p+1th point cloud model to be verified are verified successfully, and any one of the pth point cloud model to be verified and the p+1th point cloud model to be verified can be used as the first point cloud model.

[0096] Similarly, in the selection of N, the image processing device can obtain 0 The N obtained by shooting the object at different shooting heights 0 group of images. Among them, N 0 The default total number of shooting heights, N 0 >1. According to N 0 group of images, can be determined with N 0 There are N shooting heights that correspond to each other. 0 The first point cloud model, according to N 0 The intersection between the first point cloud models is used to determine the first 3D model data to be verified of the object. The specific method for determining the 3D model data to be verified can refer to the description of the method for determining the first 3D model data in the previous text, which will not be described in detail in this application.

[0097] Then, the data accuracy of the first 3D model data to be verified can be verified. If the data accuracy verification is successful, it means that the N 0 N corresponding to the shooting height 0 If the three-dimensional model data obtained by combining the images meets the accuracy requirements, the three-dimensional model data that has successfully undergone data accuracy verification can be used as the first three-dimensional model data, and there is no need to add new images at a new shooting height.

[0098] In the case of data accuracy failure, it means that the N 0 N corresponding to the shooting height 0 If the 3D model data obtained by the N group of images does not meet the accuracy requirements, additional images obtained by photographing the object at an additional shooting height can be obtained to obtain the image data according to N 0 Group images and newly added images (i.e. N 0 +1 group of images), determine the second 3D model data to be verified of the object, and perform data accuracy verification on the second 3D model data to be verified. Similarly, if the data accuracy verification succeeds, the 3D model data with successful data accuracy verification can be used as the first 3D model data. If the data accuracy verification fails, continue to obtain additional images at the newly added height and re-determine N 0 +n shooting height point cloud model (n is the total number of newly added heights), determine the nth 3D model data to be verified of the object, and perform data accuracy verification on the nth 3D model data to be verified, until there is 3D model data with successful data accuracy verification, then the 3D model data with successful data accuracy verification is used as the first 3D model data of the object.

[0099] Similarly, the method for performing data accuracy verification can be selected according to actual conditions. In some embodiments of the present application, the qth (q≥1) three-dimensional model data to be verified can be obtained. When the data error between the qth three-dimensional model data to be verified and the q+1th three-dimensional model data to be verified is less than a preset threshold, the data accuracy verification is determined to be successful. When the data error between the qth three-dimensional model data to be verified and the q+1th three-dimensional model data to be verified is greater than or equal to a preset threshold, the data accuracy verification is determined to have failed.

[0100] The data error may be a volume difference between the qth 3D model data to be verified and the q+1th 3D model data to be verified, and / or a difference in the number of point clouds.

[0101] It should be noted that the newly added shooting height and the newly added shooting angle can be selected according to the actual situation. For example, a newly added shooting angle can be selected between any two preset points that have been shot, and a newly added shooting height can be selected between any two shooting heights that have been shot. For example, the point corresponding to the newly added shooting angle can be located at the midpoint of the preset points corresponding to any two shooting angles on the guide rail. The newly added shooting height is half of the sum of any two shooting heights. This application does not impose any restrictions on this.

[0102] In this way, in a default set of [M 0 , N 0 ] to collect images. Then, at each shooting height, images of new shooting angles are selectively added through data accuracy verification until the data accuracy verification is successful, thereby ensuring the accuracy of the first point cloud model obtained at each shooting height. For the shooting height, images of new shooting heights can be selectively added through data accuracy verification until the data accuracy verification is successful, thereby ensuring the accuracy of the final first three-dimensional model data. In addition, this method can realize the adaptive selection of shooting angles and shooting heights. For objects with different structures, it can be [M 0 , N 0 ], gradually add images based on the image quality, and adaptively adjust the final values ​​of N and M to obtain detailed three-dimensional model data of the object.

[0103] In other implementations of the present application, N×M images may be directly captured and then accuracy verification may be performed.

[0104] Specifically, after determining the first point cloud model at the i-th shooting height, the image processing device may perform data accuracy verification on the first point cloud model, and then verify whether the first point cloud model obtained based on the M images meets the accuracy requirements. In the event that the data accuracy verification fails, a newly added image corresponding to the M+1th image at the i-th shooting height may be obtained to update the first point cloud model based on the M+1th image corresponding to the i-th shooting height.

[0105] Specifically, the first point cloud model obtained according to the M images may be intersected with the view frustum obtained from the M+1th image to obtain an updated first point cloud model.

[0106] In other words, this method is to consider whether to add more images to optimize the point cloud model through data accuracy verification after M images have been taken and the first point cloud model is obtained based on the M images. It should be understood that the method for determining the point cloud model and the method for data accuracy verification can refer to the previous description, and this application will not elaborate on this.

[0107] Similarly, after determining the first three-dimensional model data of the object, the image processing device can also perform data accuracy verification on the first three-dimensional model data, and if the data accuracy verification fails, obtain a new image obtained by photographing the object at a new shooting height, so as to update the first three-dimensional model data based on the new image.

[0108] Specifically, the first point cloud model obtained according to the newly added image may be intersected with the first three-dimensional model data obtained according to the N groups of images to obtain updated first three-dimensional model data.

[0109] In other words, this method is to consider whether to add more images to optimize the 3D model data through data accuracy verification after N groups of images have been taken and the first 3D model data has been obtained based on the N groups of images. It should be understood that the method for determining the 3D model data and the method for data accuracy verification can refer to the above description, and this application will not elaborate on this.

[0110] In this way, the accuracy of the first three-dimensional model data can also be guaranteed.

[0111] After the first three-dimensional model data is obtained, the first three-dimensional model data may be stored in a memory of an image processing device or in a cloud server or other devices, waiting to be called when three-dimensional reconstruction of the object is performed.

[0112] As an example, when the object is a surgical instrument, the first three-dimensional model data can be used to be called when performing surgical navigation for an operation using the surgical instrument. Specifically, during an operation using the surgical instrument, the doctor can input the identification of the operation (such as the name of the operation, the identification number) or the identification of the surgical instrument (such as the name of the instrument, the identification number). The relevant equipment can determine the surgical instrument that needs to be three-dimensionally reconstructed based on the identification. If the object in step S401 is a surgical instrument that needs to be three-dimensionally reconstructed, the first three-dimensional model data obtained in the previous text can be obtained and three-dimensionally reconstructed. The three-dimensional model obtained by three-dimensional reconstruction can be used for surgical navigation. In this way, compared with the method of using line segments or point-shaped identifications to represent surgical instruments in related technologies, the actual situation of the operation can be better simulated in surgical navigation.

[0113] The implementation method of the present application can support one or more cameras to obtain three-dimensional model data, and has high universality. By placing the camera in a cylindrical box, the calibration process can be omitted, and the portability of user use can be improved. In the process of obtaining three-dimensional model data, algorithms with good real-time performance such as visual shells can be used, and the algorithm can optimize the point cloud model results at different heights, which can improve the accuracy of three-dimensional reconstruction.

[0114] It should be noted that, for the sake of simplicity of description, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present application is not limited to the described order of actions, because according to the present application, certain steps can be performed in other orders.

[0115] like Figure 7 The figure is a schematic structural diagram of a device 700 for acquiring three-dimensional model data of an object provided in an embodiment of the present application. The device 700 for acquiring three-dimensional model data of an object is configured on an image processing device.

[0116] Specifically, the object three-dimensional model data acquisition device 700 may include:

[0117] The image acquisition unit 701 is used to acquire N groups of images by photographing the object at N different shooting heights, each group of images includes multiple images taken at different shooting angles, where N is a positive integer greater than 1;

[0118] A point cloud model determining unit 702 is used to determine N first point cloud models corresponding to the N shooting heights one by one according to the N groups of images;

[0119] The three-dimensional model data determining unit 703 is used to determine the first three-dimensional model data of the object according to the intersection between the N first point cloud models.

[0120] In some embodiments of the present application, each group of N groups of images includes M images taken from M different shooting perspectives, where M is a positive integer greater than 1; the above-mentioned point cloud model determination unit 702 can be specifically used to: determine M viewing cones corresponding one-to-one to the M different shooting perspectives according to the M images corresponding to the i-th shooting height; and determine the first point cloud model of the i-th shooting height according to the intersection between the M viewing cones.

[0121] In some embodiments of the present application, each group of N groups of images includes M images taken from M different shooting perspectives, where M is a positive integer greater than 1; the above-mentioned point cloud model determination unit 702 can be specifically used to: determine the disparity map of the i-th shooting height according to the M images corresponding to the i-th shooting height; based on the disparity map, determine the first point cloud model of the i-th shooting height.

[0122] In some embodiments of the present application, the above-mentioned image acquisition unit 701 and point cloud model determination unit 702 may be specifically used to: obtain the image from M at the jth shooting height 0 M taken from different shooting angles 0 images, M 0 >1; according to the M corresponding to the j-th shooting height 0 The first point cloud model to be verified at the j-th shooting height is determined; the first point cloud model to be verified is verified for data accuracy; if the data accuracy verification fails, the n-th point cloud model to be verified is verified for data accuracy, until the data accuracy verification of the M-th point cloud model to be verified is successful, and the point cloud model with successful data accuracy verification is used as the first point cloud model at the j-th shooting height, wherein the n-th point cloud model to be verified is the M-th point cloud model corresponding to the j-th shooting height. 0 +n-1 images, the M 0 +n-1 images corresponding to the M 0 Different shooting angles and M 0 There are n-1 shooting angles with different shooting angles, where n and M are both positive integers greater than 1; n≤M.

[0123] In some embodiments of the present application, the three-dimensional model data acquisition device 700 of the above-mentioned object may include a verification unit, which is used to: obtain the p+1th point cloud model to be verified; when the data error between the pth point cloud model to be verified and the p+1th point cloud model to be verified is greater than or equal to a preset threshold, determine that the data accuracy verification has failed.

[0124] In some embodiments of the present application, the above-mentioned data error includes: the volume difference between the p-th point cloud model to be verified and the p+1-th point cloud model to be verified, and / or the difference in the number of point clouds between the p-th point cloud model to be verified and the p+1-th point cloud model to be verified.

[0125] In some embodiments of the present application, the above-mentioned point cloud model determination unit 702 can be specifically used to: extract the first contour line of the object in the image of the k-th shooting angle; if there is an occluded area of ​​the object, fit the contour line in the occluded area according to the first contour line to obtain the second contour line of the object; based on the second contour line, determine the viewing cone corresponding to the k-th shooting angle.

[0126] In some embodiments of the present application, the above-mentioned three-dimensional model data determination unit 703 can be specifically used to: perform data accuracy verification on the first three-dimensional model data; if the data accuracy verification fails, obtain a new image obtained by photographing the object at a new shooting height, so as to update the first three-dimensional model data according to the new image.

[0127] In some embodiments of the present application, the image acquisition unit 701 can be specifically used to: control at least one camera to move to N groups of preset points on the guide rail, and acquire the N groups of images obtained by the at least one camera respectively photographing the object at the N groups of preset points, wherein the N groups of preset points correspond one-to-one to N different shooting heights; or, acquire the N groups of images obtained by the N groups of cameras respectively photographing the object, and the N groups of cameras are fixedly set at N groups of preset points, wherein the N groups of preset points correspond one-to-one to N different shooting heights.

[0128] In some embodiments of the present application, the above-mentioned three-dimensional model data determination unit 703 can be specifically used to: determine the offset between each shooting height and the reference coordinate system of the reference height according to the height difference between each shooting height and the reference height; adjust the corresponding first point cloud model according to the offset corresponding to each shooting height, so as to convert the N first point cloud models into the reference coordinate system; obtain the intersection between the N first point cloud models in the reference coordinate system to obtain the first three-dimensional model data.

[0129] In some embodiments of the present application, the above-mentioned object is a surgical instrument; and the first three-dimensional model data is called for surgical navigation of a surgery using the surgical instrument.

[0130] It should be noted that, for the convenience and simplicity of description, the specific working process of the above-mentioned object three-dimensional model data acquisition device 700 can refer to Figures 1 to 6 The corresponding process of the method will not be repeated here.

[0131] like Figure 8 FIG. 8 is a schematic diagram of an image processing device 8 provided in an embodiment of the present application. Specifically, the image processing device 8 may include: a processor 80, a memory 81, and a computer program 82 stored in the memory 81 and executable on the processor 80, such as a program for acquiring 3D model data of an object. When the processor 80 executes the computer program 82, the steps in the above-mentioned method for acquiring 3D model data of each object are implemented, such as Figure 4 Alternatively, when the processor 80 executes the computer program 82, the functions of each module / unit in the above-mentioned device embodiments are realized, for example, Figure 7 The functions of the image acquisition unit 701, the point cloud model determination unit 702 and the three-dimensional model data determination unit 703 are shown.

[0132] The computer program may be divided into one or more modules / units, which are stored in the memory 81 and executed by the processor 80 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, which are used to describe the execution process of the computer program in the image processing device.

[0133] For example, the computer program can be divided into: an image acquisition unit, a point cloud model determination unit, and a three-dimensional model data determination unit. The specific functions of each unit are as follows: an image acquisition unit, used to acquire N groups of images obtained by photographing the object at N different shooting heights, each group of images includes multiple images taken at different shooting angles, where N is a positive integer greater than 1; a point cloud model determination unit, used to determine N first point cloud models corresponding to the N shooting heights one by one according to the N groups of images; a three-dimensional model data determination unit, used to determine the first three-dimensional model data of the object according to the intersection between the N first point cloud models.

[0134] The image processing device 8 may include, but is not limited to, a processor 80 and a memory 81. Those skilled in the art will appreciate that Figure 8 It is only an example of an image processing device and does not constitute a limitation of the image processing device. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the image processing device may also include input and output devices, network access devices, buses, etc.

[0135] The processor 80 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0136] The memory 81 may be an internal storage unit of the image processing device, such as a hard disk or memory of the image processing device. The memory 81 may also be an external storage device of the image processing device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the image processing device. Further, the memory 81 may also include both an internal storage unit and an external storage device of the image processing device. The memory 81 is used to store the computer program and other programs and data required by the image processing device. The memory 81 may also be used to temporarily store data that has been output or is to be output.

[0137] It should be noted that, for the convenience and brevity of description, the structure of the above-mentioned image processing device 8 can also refer to the specific description of the structure in the method embodiment, and will not be repeated here.

[0138] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0139] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0140] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0141] In the embodiments provided in the present application, it should be understood that the disclosed devices / equipment and methods can be implemented in other ways. For example, the device / equipment embodiments described above are merely schematic, for example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0142] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0143] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0144] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0145] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for acquiring three-dimensional model data of an object, It is characterized in that include: Obtain N groups of images by photographing the object at N different shooting heights, each group of images includes multiple images taken at different shooting angles, where N is a positive integer greater than 1; Determine, according to the N groups of images, N first point cloud models corresponding one-to-one to the N shooting heights; The first three-dimensional model data of the object is determined according to the intersection of the N first point cloud models.

2. The method for acquiring three-dimensional model data of an object according to claim 1, It is characterized in that Each of the N groups of images includes M images taken from M different shooting angles, where M is a positive integer greater than 1; In the operation of determining N first point cloud models corresponding one-to-one to the N shooting heights according to the N groups of images, the operation of determining the first point cloud model corresponding to the i-th shooting height, 1≤i≤N, includes: Determine, according to the M images corresponding to the i-th shooting height, M viewing cones corresponding one-to-one to the M different shooting angles; The first point cloud model of the i-th shooting height is determined according to the intersection of the M viewing cones.

3. The method for acquiring three-dimensional model data of an object according to claim 1, It is characterized in that Each of the N groups of images includes M images taken from M different shooting angles, where M is a positive integer greater than 1; In the operation of determining N first point cloud models corresponding one-to-one to the N shooting heights according to the N groups of images, the operation of determining the first point cloud model corresponding to the i-th shooting height, 1≤i≤N, includes: Determine a disparity map of the i-th shooting height according to the M images corresponding to the i-th shooting height; Based on the disparity map, the first point cloud model of the i-th shooting height is determined.

4. The method for acquiring three-dimensional model data of an object according to claim 1, It is characterized in that In the operation of obtaining N groups of images by photographing the object at N different photographing heights, and determining N first point cloud models corresponding to the N photographing heights one by one according to the N groups of images, the operation of determining the first point cloud model corresponding to the j-th photographing height, 1≤j≤N, includes: Get the j-th shooting height from M 0 M taken from different shooting angles 0 images, M 0 >1; According to the M corresponding to the j-th shooting height 0 images, determining the first point cloud model to be verified at the j-th shooting height; Performing data accuracy verification on the first point cloud model to be verified; In the case where the data accuracy check fails, the data accuracy check is performed on the nth point cloud model to be checked until the data accuracy check of the Mth point cloud model to be checked succeeds, and the point cloud model with successful data accuracy check is used as the first point cloud model of the jth shooting height, wherein the nth point cloud model to be checked is the Mth point cloud model corresponding to the jth shooting height. 0 +n-1 images, the M 0 +n-1 images corresponding to the M 0 Different shooting angles and M 0 There are n-1 shooting angles with different shooting angles, where n and M are both positive integers greater than 1; n≤M.

5. The method for acquiring three-dimensional model data of an object as claimed in claim 4, It is characterized in that Perform data accuracy verification on the pth point cloud model to be verified, p≥1, including: Get the p+1th point cloud model to be verified; When the data error between the pth point cloud model to be verified and the p+1th point cloud model to be verified is greater than or equal to a preset threshold, it is determined that the data accuracy verification has failed.

6. The method for acquiring three-dimensional model data of an object as claimed in claim 5, It is characterized in that The data errors include: The volume difference between the p-th point cloud model to be verified and the p+1-th point cloud model to be verified, and / or the point cloud quantity difference between the p-th point cloud model to be verified and the p+1-th point cloud model to be verified.

7. The method for acquiring three-dimensional model data of an object as claimed in claim 2, It is characterized in that In the operation of determining M viewing cones corresponding to the M different shooting angles one by one according to the M images corresponding to the i-th shooting height, determining the viewing cone corresponding to the k-th shooting angle, 1≤k≤M, includes: Extracting a first contour line of the object in the image at the k-th shooting angle; If the object has an obstructed area, fitting the contour line in the obstructed area according to the first contour line to obtain a second contour line of the object; Based on the second contour line, a viewing cone corresponding to the kth shooting viewing angle is determined.

8. The method for acquiring three-dimensional model data of an object according to any one of claims 1 to 7, It is characterized in that After determining the first three-dimensional model data of the object according to the intersection of the N first point cloud models, the method further includes: Performing data accuracy verification on the first three-dimensional model data; In the case where the data accuracy check fails, a new image obtained by photographing the object at a new photographing height is obtained, so as to update the first three-dimensional model data according to the new image.

9. The method for acquiring three-dimensional model data of an object according to any one of claims 1 to 7, It is characterized in that The step of acquiring N groups of images obtained by photographing the object at N different photographing heights includes: Controlling at least one camera to move to N groups of preset points on the guide rail, and obtaining the N groups of images obtained by the at least one camera photographing the object at the N groups of preset points, wherein the N groups of preset points correspond one to one to N different shooting heights; Alternatively, the N groups of images obtained by respectively photographing the object with N groups of cameras are obtained, and the N groups of cameras are fixedly arranged at N groups of preset points, wherein the N groups of preset points correspond one-to-one to N different shooting heights.

10. The method for acquiring three-dimensional model data of an object according to any one of claims 1 to 7, It is characterized in that Determining first three-dimensional model data of the object according to the intersection of the N first point cloud models includes: Determine the offset between each shooting height and the reference coordinate system of the reference height according to the height difference between each shooting height and the reference height; According to the offset corresponding to each shooting height, the corresponding first point cloud models are adjusted respectively, so as to convert the N first point cloud models into a reference coordinate system; The intersection of the N first point cloud models is obtained in the reference coordinate system to obtain the first three-dimensional model data.

11. The method for acquiring three-dimensional model data of an object according to any one of claims 1 to 7, It is characterized in that The object is a surgical instrument; the first three-dimensional model data is called for surgical navigation of a surgery using the surgical instrument.

12. A device for acquiring three-dimensional model data of an object, It is characterized in that include: An image acquisition unit, used to acquire N groups of images by photographing the object at N different shooting heights, each group of images including a plurality of images photographed at different shooting angles, wherein N is a positive integer greater than 1; A point cloud model determination unit, configured to determine N first point cloud models corresponding one-to-one to the N shooting heights according to the N groups of images; The three-dimensional model data determining unit is used to determine the first three-dimensional model data of the object according to the intersection between the N first point cloud models.

13. An image processing device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, It is characterized in that When the processor executes the computer program, the steps of the method for acquiring three-dimensional model data of an object as claimed in any one of claims 1 to 11 are implemented.

Images