Human body part three-dimensional scanning device and scanning method thereof

Abstract

The invention discloses a device for carrying out three-dimensional scanning on human body parts by using double depth cameras. The device comprises a supporting assembly, a scanning assembly with at least two cameras and a data processing module. The invention also discloses a human body part three-dimensional scanning method, which comprises the following steps of: acquiring photos in an overlapped imaging area of the two cameras by using the standard image calibration plate and a calibration algorithm, calculating to obtain an external parameter rotation translation matrix between the cameras, and mapping images of the two cameras to the same coordinate system for splicing by using the matrix, three-dimensional data of the surfaces of the feet and the shanks of the human body can be quickly obtained, so that the defects of long operation time, intolerance of a patient and low point cloud data precision caused by poor scanning stability and limited detection range of a handheld single-depth camera used clinically are overcome; rapid real-time three-dimensional imaging splicing of human foot and shank surface data can be realized under the condition that image corner point features are not extracted for matching.

Description

Technical Field

[0001] The present application relates to medical scanning imaging, and more particularly to a device and a scanning method for performing three-dimensional scanning of human body parts using a dual-depth camera. Background Art

[0002] 3D scanning imaging technology is an advanced method that integrates optical, mechanical, electronic and computer technologies. It can accurately capture the spatial coordinates, color or spectrum of an object and generate a 3D model. This technology plays an important role in many fields such as 3D printing, intelligent manufacturing and digital medicine, especially in the process of human body modeling and personalized orthosis manufacturing, where 3D scanning imaging technology plays a vital role.

[0003] However, current handheld 3D scanners face some challenges in clinical applications. The main problems include insufficient scanning stability, low accuracy of data acquisition, limited detection range, and the need for multiple repeated scans to ensure that all necessary surface features are covered, which not only increases the operation time but also puts a greater burden on patients. In addition, the patient's involuntary movements may cause distortion of the posture data, which in turn affects the accuracy of the point cloud data.

[0004] In order to overcome these limitations and improve the stability and detection range of 3D imaging scanning, a solution needs to be proposed to improve the efficiency of the scanning process and the accuracy of the data, so as to greatly enhance the application effect of 3D scanning imaging technology in the clinical medical field, especially in the design and manufacturing of personalized orthotics, to provide patients with more comfortable and accurate treatment plans, and promote technological progress and market development of related industries. Summary of the invention

[0005] In one aspect, a device for three-dimensional scanning of a human body part using a dual-depth camera is described, comprising a support structure, a scanning structure of at least two cameras, and a data processing module. The scanning mechanism comprises at least two cameras and a rotating frame that rotates at a large angle outside the human body part. The two cameras are arranged at a certain angle and have overlapping scanning areas, and maintain relative positions unchanged during the scanning process.

[0006] In another aspect, a multi-depth camera reconstruction and stitching method for three-dimensional scanning of human body parts is described, in which surface spatial features of human body parts are collected by multi-depth cameras with fixed relative positions, and the external parameter rotation and translation matrix between the cameras is calculated, and the images of the two cameras are mapped to the same coordinate system for stitching using the rotation and translation matrix to form a real-time reconstruction mode. In some embodiments, the reconstruction and stitching method also includes an offline reconstruction mode, in which the image data during scanning is stored using a recording component, and the stored image data is read using a playback component, and then three-dimensional reconstruction and stitching are performed.

[0007] The present application can achieve at least one of the following beneficial effects or a combination thereof:

[0008] (1) Improved scanning stability during 3D scanning of human body parts and expanded the detection range;

[0009] (2) The scanning component of the present application performs scanning in a stable driving manner. At the same time, the three-dimensional scanning method of human body parts of the present application can realize fast real-time three-dimensional imaging and splicing of human foot and calf surface data without extracting image corner point features for matching, thereby reducing the time required for scanning;

[0010] (3) The support position is reasonable, which reduces the burden on patients to cooperate with the scan, avoids unnecessary movements during scanning, reduces the probability of posture data distortion, and improves the accuracy of the scan data and the precision of the point cloud data;

[0011] (4) The computer controls the scanning and performs data processing to achieve three-dimensional imaging stitching and reconstruction. At the same time, the three-dimensional scanning and reconstruction program has two modes: real-time reconstruction and offline reconstruction. It can flexibly cope with computers with different processing performance and improve mobility. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a three-dimensional schematic diagram of a human body part three-dimensional scanning device according to an embodiment of the present application.

[0013] Figure 2 Schematic diagram of the connection between the electric turntable, stepper motor driver and controller according to an embodiment of the present application.

[0014] Figure 3 It is a schematic diagram of the splicing principle of the three-dimensional scanning method of human body parts according to an embodiment of the present application.

[0015] Figure 4 This is a diagram of the computer program interface in real-time reconstruction mode.

[0016] Figure 5 This is a diagram of the computer program interface in offline reconstruction mode.

[0017] Figure 6 This is the original three-dimensional image of the human foot and lower leg after reconstruction.

[0018] Figure 7 This is a cropped 3D image of the human foot and calf.

[0019] Among them, 100, actuator; 110, scanning component; 111, support assembly; 1111, first upright support; 11111, support seat; 1112, support frame; 11121, support plane; 11122, connecting frame; 11123, locking component; 1113, second upright support; 112, scanning assembly; 1121, scanning frame; 11211, first scanning part; 11212, second scanning part; 11213, connecting part; 11214, counterweight; 1122, scanning module; 11221, first depth camera; 11222, second depth camera; 113, chassis; 120, data processing module; 200, control mechanism; 210, controller; 300, driving mechanism; 310, stepping motor; 320, stepping motor driver; 330, turntable; 340, switching power supply.

[0020] P represents a point in the world coordinate system. is the coordinate system of point P in the left camera (X l ,Y l ,Z l ), is the coordinate system of point P in the right camera (X r ,Y r ,Z r ), is the rotation and translation matrix from the world coordinate system to the left camera coordinate system, is the rotation and translation matrix from the world coordinate system to the right camera coordinate system, C l represents the imaging center of the left camera, C r Indicates the imaging center of the right camera.

[0021] exist Figure 4 4a is a color image of the left camera in real-time reconstruction mode, 4b is a depth image of the left camera in real-time reconstruction mode, 4c is a color image of the right camera in real-time reconstruction mode, 4d is a depth image of the right camera in real-time reconstruction mode, and 4e is a reconstructed image displayed synchronously with the color image and the depth image in real-time reconstruction mode.

[0022] exist Figure 5 5a is the color image of the left camera in the offline reconstruction mode, 5b is the depth image of the left camera in the offline reconstruction mode, 5c is the color image of the right camera in the offline reconstruction mode, 5d is the depth image of the right camera in the offline reconstruction mode, and 5e is the reconstructed image displayed asynchronously with the color image and the depth image in the offline reconstruction mode. DETAILED DESCRIPTION

[0023] The present application is further described below in conjunction with the accompanying drawings and embodiments.

[0024] The following will clearly and completely describe the concept, specific structure and technical effects of the present application in combination with the embodiments and drawings, so as to fully understand the purpose, characteristics and effects of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of them. Based on the embodiments of the present application, other embodiments obtained by technicians in this field without creative work are all within the scope of protection of the present application. In addition, all the connection / connection relationships involved in the patent do not refer to the direct connection of components, but refer to the formation of a better connection structure by adding or reducing connection accessories according to the specific implementation situation. The various technical features in the invention can be combined interactively without conflicting with each other.

[0025] Exemplary human body part three-dimensional scanning device and scanning method thereof

[0026] like Figures 1 to 7 As shown, a three-dimensional scanning device for human body parts according to an embodiment of the present application is illustrated, which includes an actuator 100, a control mechanism 200 and a driving mechanism 300. The actuator 100 further includes a scanning component 110 and a data processing module 120. The scanning component 110 performs three-dimensional scanning on human body parts and transmits the scanned image data to the data processing module 120 for processing. In this process, the driving mechanism 300 is used to drive the three-dimensional scanning of the scanning component 110. The control mechanism 200 is used to control the driving mechanism 300 and the scanning component 110 coupled to the driving mechanism 300 to perform activities such as movement and scanning, and realize the data processing of the data processing module 120 and adjust the rotation action and amplitude of the driving mechanism 300 through circuits or signal transmission methods. Among them, the human body parts in the following examples are a human foot and calf as an embodiment.

[0027] The scanning component 110 further includes a support component 111 and a scanning component 112. The support component 111 is used to frame the scanning space area for the person to be scanned and provide support for the human body part to be scanned and the scanning component 112. The scanning component 112 performs a large-scale scan on the periphery of the human body part to be scanned, and its scanning range covers all spatial features of the outer surface of the human body part, so that the scanning component 112 can scan and collect all image data of the human body part. (Here, considering the limitations of the scanning range and resolution of the scanning component 112 and the debugging error of the movement of the scanning component 112, in the actual working process, the scanning component 112 may have some missing image data when collecting image data of the human body part. In this regard, as long as the actually collected image data can reconstruct the external contour of the human body part, further technical improvements can be made based on the scanning results according to actual needs, which should be understood by ordinary technicians in this field.) The image data obtained after the dual-depth camera scan is completed is transmitted to the data processing module 120.

[0028] In a specific example of the present application, the support assembly 111 further includes a first upright support member 1111, a second upright support member 1113, and a support frame 1112 extending at an angle from the top side of the first upright support member 1111, wherein the first upright support member 1111 and the second upright support member 1113 are both extended upward in parallel from the bottom of the support assembly 111, i.e., the chassis 113, and are arranged opposite to each other. In some optional embodiments, the first upright support member 1111 is two upright extension legs. A support seat 11111 is formed on the top side of the first upright support member 1111, and the top surface of the support seat 11111 is 90° to the upright direction of the first upright support member 1111, i.e., a horizontal plane is formed, which is used to support one end of the human body part to be scanned upward from the bottom side of the joint of the human body part (e.g., the hip, elbow, axillary bottom or popliteal fossa). The support frame 1112 has a support plane 11121 to support the other end or part of the human body part to be scanned, so as to provide another support site for the joint of the human body part. The second upright support member 1113 provides support for the scanning component 112. The support frame 1112 is located between the first upright support member 1111 and the second upright support member 1113, and the support plane 11121 is arranged on the extended end side of the support frame 1112, that is, the support frame 1112 connects the support plane 11121 and the first upright support member 1111. The upper space where the first upright support member 1111 is connected to the support frame 1112 and extends is the framed scanning space. The human body part to be scanned is supported by the first upright support member 1111 and the support frame 1112, so that the spatial features of the outer surface of the human body part are all located in the framed scanning space, so as to facilitate the scanning component 112 to rotate around the periphery of the human body part for three-dimensional scanning. In some optional embodiments, the support frame 1112 is two parallel frames without any structural connection between the two frames, and is only connected by the support plane 11121 at the extended end side, so as to facilitate the scanning component 110 to better and completely scan the human body part to be measured. In other words, in some optional embodiments, the scanning space is defined or framed by the support seat 11111 on the top side of the first upright support member 1111 and the support plane 11121 to place or fix the two end sides of the human body part (such as elbows, armpits, etc.), so as to facilitate the comfortable and stable placement and scanning of the long limbs, so as to further imitate the natural state of the human body for clear scanning. Among them, the bottom side of the first upright support member 1111 is horizontally formed into a chassis 113 and connected to the second upright support member 1113. At the same time, the extension direction of the support frame 1112 forms an extension surface that is at a certain angle to the first upright support member 1111, and the support frame 1112 and the first upright support member 1111 are rotatably connected to meet the different situations of different users or patients and conform to ergonomic design, ensuring that the human body parts can be stretched normally according to the posture when standing or sitting during scanning, thereby improving the accuracy of the scanning results.

[0029] In a specific example of the present application, the first upright support member 1111 is implemented as an upright double-columnar structure and a support seat 11111 is provided on the top side. The support frame 1112 is implemented to be rotatably connected to the support seat 11111. In some optional embodiments, the height of the first upright support member 1111 is adjustable, that is, a telescopic setting. The support frame 1112 further includes a support plane 11121 and a connecting frame body 11122 provided on the side of the support plane 11121 and extending to the first upright support member 1111. The two ends of the connecting frame body 11122 are respectively hinged on the support seat 11111 and the support plane 11121 and can rotate respectively. The length between the support seat 11111 and the plate body is the framed scanning space length. When the human body part is in a drooping state, the height range between the lowest point of the support plane 11121 and the highest point of the support seat 11111 is the framed scanning space height; when the human body part is placed in a nearly horizontal state, the height range between the lowest point of the support plane 11121 and the highest point of the plate body is the framed scanning space height. Among them, at least one locking component 11123 may be further included between the two connecting frames 11122 respectively located on the support plane 11121 to prevent the displacement in the scanning space positioning caused by the azimuth offset or insufficient holding force between the multiple structural members included in the support frame 1112 from causing unnecessary movement of the human body part, resulting in distortion of the posture data. More specifically, the connecting frame 11122 can be implemented as a telescopic frame with variable length, which is used to adjust the scanning space defined between the support plane 11121 and the support seat 11111, and provide flexible support conditions for the joints of human body parts of different lengths, thereby achieving scanning stability of different human body parts.

[0030] In order to further clearly demonstrate and understand the advantages and principles of the above design, taking the human calf or human forearm as an example, the foot or hand of the person to be scanned is suspended in the air and lifted up at a certain angle, and the top of the support seat 11111 supports the popliteal fossa or elbow joint of the human thigh to assist the person to be scanned in maintaining the lifted state of the human calf or human forearm, and the support plane 11121 contacts the sole or palm of the person to be scanned, and the connecting frame 11122 rotates a certain angle at the rotating connection between the connecting frame 11122 and the support seat 11111 as the human calf or human forearm is lifted up, so that the human calf or human forearm maintains a stable lifted state in the framed scanning space. The advantage of this design is that protective gear needs to be worn during general orthopedic rehabilitation, but because the joints and limbs of patients are different and the injuries are complicated, targeted scanning and protective gear design are required. Therefore, combining the patient's limb condition and restoring the patient's normal stretching state become key requirements. The advantage of the above design is that it can be adjusted according to the different limb needs and postures of different patients, restore the patient's comfortable posture and perform scanning modeling.

[0031] Specifically, the support assembly 111 provides at least two support points for the human body part through the support seat 11111 and the support plane 11121, so as to reduce the endurance required for the person to be scanned to maintain the position of the human body part during the scanning process, and at the same time, can more accurately simulate the posture of the human body in an actual standing or sitting environment. Among them, at least one of the support points is located at at least one joint of the human body part, which can strengthen the maintenance of the patient's posture during scanning and reduce the impact of long-term movement maintenance on the scanning of the outer contour line of the human body part.

[0032] In order to further ensure the stability of the human body parts being supported during the scanning process, a telescopic component is provided between the bottom of the connecting frame 11122 and the side of the supporting seat 11111 to prevent the connecting frame from being displaced in a direction perpendicular to the rotation plane during the rotation process. In some specific embodiments, the telescopic component is implemented as an inner rod and an outer rod that are mutually sleeved, and a height control component is provided between the inner rod and the outer rod to improve the telescopic stability of the telescopic component. In some optional embodiments, the height control component is a slide rail and a slider sliding in the slide rail, and the slide rail and the slider are respectively provided on the contact surface of the inner rod and the outer rod. In some specific embodiments, the telescopic component is implemented as a telescopic cylinder, which is electrically connected to a stepping motor 310. During actual operation, the stepping motor driver 320 is controlled by a computer control program to drive the stepping motor 310 to power on or off the telescopic cylinder and control the telescopic amount of the telescopic cylinder.

[0033] In a specific example of the present application, the first upright support member 1111 and the support frame 1112 are both implemented as upright columnar structures, wherein the top side of the support frame 1112 is hinged with a support plane 11121. Taking the human foot and calf as an example, the foot of the person to be scanned is suspended in the air and raised up at a certain angle. The top of the support seat 11111 and the support plane 11121 are respectively supported at the popliteal fossa of the human thigh and the human foot surface, so that the human foot and calf maintain a stable upward state, which is used to simulate the position of the human foot and calf in the upward state in an actual normal standing or sitting situation. Among them, the top of the support seat 11111 and the support plane 11121 are adjusted in distance by the extension and contraction of at least one connecting frame 11122, so as to adapt to human body parts of different lengths.

[0034] More specifically, the top of the support base 11111 has a soft cushion for improving the comfort of the supported human body parts, reducing the endurance required by the person to be scanned to maintain the state of the human body parts to be scanned, or relieving the tense state of the limbs.

[0035] In some optional embodiments, the support frame 1112 may further include a posture calibration component, such as a contact pad. The posture calibration component is used to contact the obvious bone of the human body or the area with less impact on the overall contour and perform body posture calibration, such as the knee or heel of the lower leg of the human body, to further restore the ideal body posture.

[0036] In some optional embodiments of the present application, the supporting surface of the support frame 1112 is made of a transparent material, which is used to support human body parts while facilitating three-dimensional scanning.

[0037] like Figure 1 As shown, the scanning assembly 112 includes a scanning frame 1121 rotatably mounted on the second upright support 1113 and a scanning module 1122 disposed on the scanning frame 1121. The scanning module 1122 performs a three-dimensional scan of the human body part during the circumferential movement of the scanning frame 1121 around the periphery of the human body part. The scanning assembly 112 preferably rotates in a vertical direction, that is, rotates on a plane perpendicular to the horizontal plane.

[0038] Specifically, the scanning frame 1121 includes a scanning frame 1121 rotatably mounted on a second upright support 1113, and the scanning frame 1121 is rotatably mounted on a side wall of the second upright support 1113 facing the first upright support 1111. In order to avoid scanning errors caused by relative displacement between the support assembly 111 and the scanning assembly 112, the support assembly 111 and the scanning assembly 112 are directly fixed or relative positions are determined through other structures. In some examples, the first upright support 1111 is implemented as a support seat 11111 having a chassis 113, and the second upright support 1113 is implemented as an upright columnar base 11131 provided on the chassis 113. More specifically, the bottom end of the support seat 11111 and the bottom end of the base 11131 can slide on the top of the chassis 113 and be fixed in position by a locking member to achieve adjustment and fixation of the relative position between the support seat 11111 and the base 11131, thereby adapting to human body parts of different lengths.

[0039] In a specific example of the present application, the second upright support member 1113 is implemented as a vertical support rod, the first upright support member 1111 is parallel to the vertical support rod of the second upright support member 1113 and is roughly similar in height, and the scanning component 112 is disposed on the top side of the second upright support member 1113 and is rotatably connected. In some embodiments, the scanning frame 1121, which is specifically implemented as the scanning component 112, is rotatably connected to the top side of the vertical support rod and is designed as a rotating long rod. The rotation connection between the scanning frame 1121 and the second upright support member 1113 is located on the vertical support rod and faces the first upright support member 1111. The rotating long rod rotates around the scanning space defined between the above-mentioned support plane 11121 and the support seat 11111 with the rotation connection as the rotation point.

[0040] like Figure 1-Figure 3 As shown, the scanning module 1122 includes at least two depth cameras, namely a first depth camera 11221 and a second depth camera 11222, the first depth camera 11221 and the second depth camera 11222 are respectively located at the peripheral positions of the human body parts at different angles, by simulating the principle of human binocular vision, and further preferably capturing image information of the same object from different angles, that is, image data of the outer surface of the human body part, and then calculating the parallax or phase difference between the image data captured by the dual depth cameras through a data processing module 120 to obtain depth information, so as to obtain spatial feature data information of the outer surface of the human body part. Preferably, the first depth camera 11221 and the second depth camera 11222 are implemented as depth cameras of the same model. The first depth camera 11221 and the second depth camera 11222 can be used in combination with at least one or more depth acquisition methods such as visible light, structured light, TOF imaging or spectral imaging.

[0041] In a specific embodiment of the present application, the scanning frame 1121 is implemented as a rotating long rod that is bent in multiple sections and extends toward the support assembly 111 or the first upright support member 1111, including a first scanning portion 11211, a second scanning portion 11212 and a connecting portion 11213 that are connected in sequence, wherein the connecting portion 11213 is arranged parallel to the second upright support member 1113 and is rotatably connected, preferably vertically rotatably connected. In some optional embodiments, the first scanning portion 11211 is connected at an angle from the connecting portion 11213 and extends toward the support assembly 111, wherein the connection bending angle between the first scanning portion 11211 and the connecting portion 11213 is between 5° and 35°. In some optional embodiments, the second scanning portion 11212 is connected at an angle from the first scanning portion 11211 and extends toward the support assembly 111, wherein the angle between the first scanning portion 11211 and the second scanning portion 11212 is in the range of 90° to 150°. Further, the first scanning unit 11211 and the second scanning unit 11212 are provided with a scanning module 1122 on one side of the support frame 1112. In some optional embodiments, the scanning module 1122 is implemented as a plurality of depth cameras, wherein the first depth camera 11221 is provided on the first scanning unit 11211, and the second depth camera 11222 is provided on the second scanning unit 11212, and they are respectively facing the support assembly 111, and at least one rotation connection of the scanning frame 1121 is provided on the connection portion 11213. The rotation connection is provided on the connection portion 11213. In some preferred embodiments, the first depth camera 11221 is arranged on a side of the first scanning unit 11211 close to the second upright support 1113, and the second depth camera 11222 is arranged at an end of the second scanning unit 11212 away from the second upright support 1113, so as to form a larger binocular depth camera setting baseline spacing, and a relatively far baseline spacing can improve the acquisition accuracy. In order to simulate the principle of human binocular vision, the first depth camera 11221 and the second depth camera 11222 need to be located at the peripheral position of the human body parts at different angles, that is, the first scanning unit 11211 and the second scanning unit 11212 need to be located at the peripheral position of the human body parts at different angles. More specifically, the first scanning unit 11211 and the second scanning unit 11212 form a certain angle, so that the rotational scanning of the dual depth camera can satisfy the requirement of covering the entire outer surface of the human body parts while having enough overlapping matching points and maintaining a certain depth resolution. In some specific examples, the angle formed by the first scanning unit 11211 and the second scanning unit 11212 (which can also be regarded as the first depth camera 11221 and the second depth camera 11222) ranges from 90° to 150°.In some other examples, the angle formed by the first scanning unit 11211 and the second scanning unit 11212 is adjustable, that is, the first scanning unit 11211 and the second scanning unit 11212 are hinged, and more specifically, the first scanning unit 11211 and the second scanning unit 11212 are adjustably connected by adjusting the hinge.

[0042] In order to improve the stability of the rotation scanning process, a turntable 330 is provided at the rotation connection of the connection part 11213 and the vertical support rod. In actual use, the computer control program controls the stepper motor driver 320 to drive the stepper motor 310 to power the turntable 330. The turntable 330 drives the rotating long rod to rotate around the scanning space at a relatively uniform speed, thereby improving the accuracy and stability of the depth information acquisition during the scanning process. Since the image data scanned by the dual-depth camera needs to be located at a part of the overlapping matching points, the angle setting of the dual-depth camera should not be too large, preferably 90° to 150°. When rotating, the local weight difference on both sides of the turntable 330 is large, and there is a situation of unstable rotation. In order to further improve the stability of the rotation of the rotating long rod, the other end of the connection part 11213 away from the first scanning part 11211 is provided with a counterweight 11214 to reduce the weight difference of the scanning frame 1121 on both sides of the turntable 330 when the turntable 330 rotates, thereby improving the stability of the rotation of the rotating long rod. The turntable 330 may be implemented as an electric hollow rotating platform.

[0043] Furthermore, since the human body is basically long and narrow, the use of a single depth camera during the scanning process has the problem of small acquisition coverage and difficulty in establishing a complete or high-precision three-dimensional model. In some optional embodiments, when actively scanning point cloud acquisition, poor control of speed or acquisition angle may cause some acquired images to be distorted or some images to be lost.

[0044] In some optional embodiments, the second scanning unit 11212 extends to one side of the length extension direction of the connecting frame body 11122, that is, toward the circumferential side of the support frame 1112, and at least one second depth camera 11222 is arranged on the second scanning unit 11212 and faces the connecting frame body 11122 for collecting circumferential depth information of the human body trunk, and at least one first depth camera 11221 is arranged on the first scanning unit 11211 and faces the support frame 1112 for collecting depth information of at least one end side or another torso part of the human body, that is, toward the end side of the support frame 1112. In some preferred embodiments, the first depth camera 11221 faces the supporting plane 11121, and the first depth camera 11221 is arranged on a side close to the connecting portion 11213, for focusing more on image acquisition on one side and improving the acquisition accuracy of an end area of ​​a limb, such as the foot area. The angle between the two depth cameras is preferably 90°-150° and they respectively collect different areas of the torso to be measured, which can effectively take into account both point cloud collection and modeling accuracy. At the same time, preferably, the support frame 1112 is designed as two connecting frames 11122 extending in parallel, and the support plane 11121 has a transparent material body to reduce occlusion during scanning.

[0045] In some specific examples, the scanning frame 1121 includes a first rotating long rod and a second rotating long rod, and the structures of the first rotating long rod and the second rotating long rod are the same as the structural composition of the above-mentioned rotating long rod, and the angular positions of the first rotating long rod and the second rotating long rod are different, that is, the dual depth camera on the first rotating long rod or the dual depth camera on the second rotating long rod are both located on the same length plane of the scanning space, and the dual depth camera on the first rotating long rod and the dual depth camera on the second rotating long rod are located on different length planes of the scanning space. More specifically, when the first rotating long rod and the second rotating long rod are implemented so that their relative positions remain unchanged during the rotational scanning process, the first rotating long rod and the second rotating long rod can be implemented so that they are both located on one side of the scanning space or are relatively arranged on both sides of the scanning space, and need to circle the outside of the human body part; when the first rotating long rod and the second rotating long rod are implemented so that their relative positions change in real time during the rotational scanning process, the first rotating long rod and the second rotating long rod are driven by different drivers and the rotation is controlled by different controllers 210. It is not necessary to rotate one circle during scanning. As long as the regional image data scanned by the arc motion of the depth camera on the first rotating long rod and the second rotating long rod can be spliced ​​into the complete outer surface spatial features of the human body part, the rotation angle and number of circles are not limited here and are not elaborated. In addition, when the first rotating long rod and the second rotating long rod are implemented with different lengths, as long as the circumferential segment area image data scanned by the depth cameras on the first rotating long rod and the second rotating long rod can be spliced ​​into the complete outer surface spatial features of the human body part, the rotation angle and number of weeks are not limited here and will not be elaborated.

[0046] In other specific examples, the rotating frame includes more than two rotating long rods. As long as the angular positions of different rotating long rods are different, the relative positions of the dual depth cameras located on the same rotating long rod remain unchanged during rotation, and the areas scanned by all depth cameras can be spliced ​​into the complete outer surface spatial features of the human body part, the composition, length, position distribution design and movement mode of the multiple rotating long rods are not restricted.

[0047] In other examples of the present application, the second upright support member 1113 is implemented as a bottom rotating seat, which is arranged below the scanning space, and the scanning frame 1121 is implemented as an arc-shaped rotating rod. The first depth camera 11221 and the second depth camera 11222 are both arranged on the inner curved wall of the arc-shaped rotating rod and maintain a certain distance. The arc-shaped rotating rod is movably connected to the top of the bottom rotating seat at a certain point or a certain section so that the arc-shaped rotating rod moves along the circumferential trajectory of the scanning space, so that the scanning range of the dual depth camera covers the outer surface of the human body part.

[0048] In some specific examples, a roller is provided on the top of the bottom rotating seat, and the roller is slidably connected to the outer curved wall of the arc-shaped rotating rod. The roller rolls and drives the arc-shaped rotating rod to move along the outer circumferential trajectory of the scanning space to complete the scanning. In other specific examples, the top of the bottom rotating seat and the arc-shaped rotating rod are driven by a mutually meshing gear structure, that is, a gear is provided on the top of the bottom rotating seat, and a curved rack meshing with the gear is provided on the outer curved wall of the arc-shaped rotating rod. The roller or the gear is driven by the driving mechanism 300 to rotate, and the opening and closing of the rotation and the rotation speed are controlled by the computer control program.

[0049] In some specific examples, the arc-shaped rotating rod is provided with a plurality of scanning modules with dual-depth cameras, and during the movement of the arc-shaped rotating rod along the circumferential trajectory of the scanning space, the plurality of scanning modules simultaneously scan until the outer surface of the human body part is scanned. In this process, as long as the regional image data scanned by the plurality of scanning modules can be spliced ​​into the complete outer surface spatial features of the human body part, the composition, spacing, and position distribution design of the plurality of scanning modules are not limited.

[0050] In a specific example of the present application, the data processing module 120 is implemented as a computer. Preferably, the processor of the computer is an Intel Core i5 or a processor with better performance, and the GPU of the computer is an Nvidia GTX 950 or an AMD graphics card equivalent thereto, or a GPU and an AMD graphics card with better performance, so as to ensure that the acquired image data can be calculated and reconstructed in real time after the three-dimensional scanning is completed.

[0051] In a specific example of the present application, the driving mechanism 300 is implemented as a stepper motor 310 and a stepper motor driver 320, and the control mechanism 200 is implemented as a controller 210 and a switching power supply 340, and the controller 210 and the switching power supply 340 are electrically connected to the stepper motor 310 and the stepper motor driver 320, and the controller 210 is used to receive computer serial port signal control. In some optional embodiments, the control mechanism 200 realizes scanning motion control, camera image data tracking and acquisition, and real-time reconstruction of point cloud data three-dimensional scanning through the program of the built-in or external data processing module 120.

[0052] like Figures 1 to 7 As shown, a three-dimensional scanning method of a human body part according to an embodiment of the present application is explained, comprising the following steps:

[0053] S1, confirm that the relative positions of the first depth camera 11221 and the second depth camera 11222 are fixed, the person to be scanned places the torso of the human body to be scanned on the support assembly 111, ensure that both ends of the torso of the human body to be scanned are stably supported and remain motionless, and then control the dual depth cameras to move around the outer circumference of the human body part through the control program of the computer, and perform three-dimensional scanning at the same time;

[0054] S2, such as Figure 3 As shown, a standard image calibration plate is used to collect photos in the overlapping imaging area of ​​two cameras using a calibration algorithm, and the external parameter rotation and translation matrix (R, T) between the cameras is calculated;

[0055] The specific algorithm is as follows:

[0056] 1) Assume there is a point P in space, in the world coordinate system (X w ,Y w ,Z w ) is make is the coordinate system of point P in the left camera (X l ,Y l ,Z l ), is the coordinate system of point P in the right camera (X r ,Y r ,Z r ), is the rotation and translation matrix from the world coordinate system to the left camera coordinate system, is the rotation and translation matrix from the world coordinate system to the right camera coordinate system. The following two equations are obtained:

[0057] and

[0058] 2) Order For the rotation and translation matrix from the left camera coordinate system to the right camera coordinate system, the following equation can be obtained:

[0059]

[0060] 3) The above three formulas can be further simplified to eliminate get:

[0061] and

[0062] S3. Use the rotation and translation matrix (R, T) to map the images of the two cameras to the same coordinate system for splicing, and realize the real-time reconstruction of the 3D scanning data. Assume that the point of the left camera is Pc l (x l ,y l ,z l ), the point of the right camera is Pc r (x r ,y r ,z r ), the coordinate transformation relationship from the left camera to the right camera is:

[0063]

[0064] Where R is the calibrated 3*3 rotation matrix, and T is the calibrated 3*1 translation matrix.

[0065] Preferably, two recorders are set at the data output end of the dual depth camera to respectively store the color image and depth image data transmitted by the first depth camera 11221 and the second depth camera 11222. After the scanning is completed, the stored data is read through the computer's playback device, and then S2 and S3 are performed to reconstruct and stitch, thereby realizing offline reconstruction of the three-dimensional scanning data.

[0066] The present application uses a computer to control an electric hollow rotating platform to drive at least two depth cameras set at a certain angle on a scanning frame 1121, namely a first depth camera 11221 and a second depth camera 11222, to perform a panoramic scan of the outer surface of a human trunk (such as a human foot and calf). The external parameters between the above two cameras with fixed relative positions on the scanning frame 1121 are obtained by calibration means, and the rotation matrix R and translation matrix T between the cameras are calculated. The imaging point clouds of the two cameras are mapped to the same coordinate system using R and T for reconstruction and splicing, and the surface three-dimensional data of the human trunk is quickly obtained, thereby improving the pain points of poor scanning stability and long operation time caused by limited detection range when using a single handheld depth camera in clinical practice, and solving the shortcomings of patient intolerance and low accuracy of point cloud data. With a relatively low cost and solution architecture, fast real-time three-dimensional imaging and splicing of surface data of human foot and calf can be achieved without extracting image corner point features for matching, and ultimately used to realize three-dimensional information acquisition and for the design of subsequent diagnosis and treatment protective gear.

[0067] In some embodiments, the three-dimensional scanning and reconstruction program of the present application has two modes: real-time reconstruction and offline reconstruction. Real-time reconstruction is adopted when using a high-performance desktop computer, while the offline reconstruction mode can scan first and then reconstruct to avoid tracking failure caused by frame loss, which meets the needs of computers with low configuration but easy to carry, such as laptops, thereby increasing the mobility of the device of the present invention.

[0068] The above is a specific description of the preferred implementation of the present application, but the invention is not limited to the embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A three-dimensional scanning device for human body parts, characterized in that: include: A support assembly including a first upright support member and a second upright support member disposed upright; at least one support frame disposed above the first upright support member; at least one scanning assembly rotatably disposed on the second upright support; as well as At least two scanning modules are fixedly arranged at an angle on the scanning component and facing the supporting frame, and the two scanning modules have overlapping scanning areas.

2. The three-dimensional scanning device for human body parts according to claim 1, characterized in that: The first upright support member includes a support base horizontally arranged above the first upright support member; the support frame is arranged below the support base and extends toward the second upright support member at an angle.

3. The three-dimensional scanning device for human body parts according to claim 2, characterized in that: The support frame is rotatably disposed on the first upright support member and faces the second upright support member, and a support plane is disposed at an angle on the extended end side of the support frame.

4. The three-dimensional scanning device for human body parts according to claim 3, characterized in that: The scanning assembly further includes a scanning frame having a connecting portion, a first scanning portion, and a second scanning portion arranged in sequence, wherein the first scanning portion is bent at an angle from the connecting portion and extends toward the upright support member, and the second scanning portion is bent at an angle from the first scanning portion and further extends toward the upright support member.

5. The three-dimensional scanning device for human body parts according to claim 4, characterized in that: The scanning assembly further includes a scanning frame, which has a connecting portion, a first scanning portion, and a second scanning portion arranged in sequence, wherein the first scanning portion is bent at an angle from the connecting portion and extends toward the upright support member, and the second scanning portion is bent at an angle from the first scanning portion and further extends toward the upright support member.

6. The three-dimensional scanning device for human body parts according to claim 5, characterized in that: The support frame is arranged between the first upright support member and the second upright support member; the connection bending angle between the first scanning part and the connecting part is between 5°-35°; the angle formed by the first scanning part and the second scanning part ranges from 90° to 150°.

7. The three-dimensional scanning device for human body parts according to claim 6, characterized in that: The scanning module is implemented as a first depth camera and a second depth camera, wherein the first depth camera is disposed on a side of the first scanning portion close to the second upright support, and the second depth camera is disposed on an end of the second scanning portion away from the second upright support.

8. A method for three-dimensional scanning of human body parts, comprising the following steps: S1. Confirm that the relative positions of all cameras are fixed, place a human body part on the support assembly and ensure that both ends of the human body part are supported and the whole body remains stationary, and then drive a scanning component with at least two cameras to perform a panoramic three-dimensional scan around the human body part; S2. Using a standard image calibration plate and a calibration algorithm, collect photos in the overlapping imaging area of ​​the two cameras, and calculate and obtain the extrinsic rotation and translation matrix between the cameras; S3. Use the rotation and translation matrix to map the images of the two cameras to the same coordinate system for splicing, and complete the real-time reconstruction of the three-dimensional scanning data.

9. The three-dimensional scanning method of a human body part according to claim 8, wherein: The input ends of the two cameras and the scanning component are connected to the computer, and the scanning process is controlled by the control program of the computer and the image data obtained by the scanning is received and processed.

10. The three-dimensional scanning method of a human body part according to claim 8, wherein: Connect the camera's output to the recorder and reader in sequence, and connect the reader to the computer's serial port; While the two cameras are acquiring scan data, the scan data is transmitted to the recorder for storage; Before performing data calculation, the control program of the computer first reads the data stored in the recorder, and then performs steps S2 and S3 as claimed in claim 8 to complete the real-time reconstruction of the three-dimensional scanning data.

Images