3D scanning systems for medical applications

Abstract

The present invention discloses a 3D scanning system for the medical field, which belongs to the field of 3D scanning technology and aims to solve the problems of dynamic changes, low data acquisition efficiency and high complexity of post-processing encountered in the 3D scanning process. The system comprises the following steps: projecting a light pattern onto a target object through a projector of a 3D scanner, capturing the reflection intensity with a camera and automatically adjusting the brightness and contrast of the projector; capturing a reflected image of the light pattern and obtaining an acquisition distance, and automatically adjusting the camera shutter speed to optimize data capture after ensuring that scanning rules are met; pre-processing the reflected data, filtering invalid data, converting it into a point cloud and performing posture transformation, perspective merging, density balancing and outlier removal; generating a 3D model based on the processed point cloud data, and completing target recognition and annotation through grid optimization; storing and managing the reflected data and processing results to achieve efficient and accurate 3D scanning and modeling.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional scanning technology, and more particularly to a three-dimensional scanning system used in the medical field. Background Art

[0002] The application of 3D scanning systems in the medical field is becoming increasingly widespread, particularly in prosthetics and rehabilitation, demonstrating tremendous potential and value. With the continuous advancement of technology, traditional prosthetic fabrication methods have gradually exposed their limitations, such as long production cycles, high costs, and poor fit. The introduction of 3D scanning technology has provided new ideas and methods to address these issues. In recent years, with the development of industrial automation and computer-aided design (CAD), the application of 3D scanning technology in the manufacture of medical rehabilitation devices has gained increasing attention. In particular, in the customization of prosthetics and orthotics, 3D scanning technology can provide precise patient data, enabling the creation of personalized products that better meet patient needs. Through non-contact measurement, 3D scanning technology can quickly acquire precise 3D data of a patient's residual limb or body, reducing errors associated with manual measurement and improving modeling accuracy. This technology not only simplifies the prosthetic fabrication process but also generates digital models, facilitating subsequent modification and re-creation. Furthermore, 3D scanned models are editable, allowing for further digital design and fabrication. The human body is complex and sophisticated. 3D scanning can obtain extremely detailed and rich data in a relatively short period of time, allowing us to tailor medical products such as prostheses, implants, rehabilitation braces, and orthotic braces for patients, achieving precise matching and meeting their personalized needs.

[0003] Chinese patent application CN118161194B discloses a handheld probe 3D scanning imaging system and method, comprising a hardware timing control module, a binocular optical tracking module, an inertial measurement unit (IMU), a handheld detection module, and a data acquisition module. Leveraging the compatibility of frame rates between different imaging modalities and different 3D tracking systems, the position information of the handheld detection module collected by the binocular optical tracking module is matched with the photoacoustic image, and the position information of the handheld detection module collected by the IMU is matched with the ultrasound image. Furthermore, the timing of all modules is aligned through the hardware timing control module, thereby achieving photoacoustic and ultrasound multimodal handheld probe scanning imaging with both high global accuracy and high instantaneous frame rate.

[0004] While existing technologies achieve mutual calibration of three-dimensional coordinates by combining two positioning methods, balancing global and instantaneous accuracy, they still fail to address the dynamic changes encountered during complex surface scanning, low data acquisition efficiency, and high complexity in subsequent data processing. To overcome these limitations, the present invention proposes a three-dimensional scanning system for use in the medical field. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the purpose of the present invention is to provide a three-dimensional scanning system for the medical field, which solves the problems of dynamic changes, low data acquisition efficiency and high complexity of subsequent data processing encountered in the scanning process of complex surfaces. By intelligently optimizing light pattern projection, dynamically adjusting equipment parameters, real-time tracking and collecting data, automatically removing redundant information and accurately reconstructing three-dimensional models, the quality and reliability of the scanning data are ensured.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A three-dimensional scanning system for the medical field, comprising a signal transmitting module, a data receiving module, a data processing module, a data modeling module, and a data storage module;

[0008] The signal transmission module is used to project a light pattern onto a target object through a projector in the 3D scanner, collect the reflection intensity of the target object through a camera in the 3D scanner, and automatically adjust the brightness and contrast of the projector according to the reflection intensity;

[0009] The data receiving module is used to capture the reflected image of the light pattern through the camera in the 3D scanner, and obtain the acquisition distance based on the reflected image. When the acquisition distance meets the scanning rules, the camera shutter speed is automatically adjusted according to the reflection properties of the target object light pattern to capture the reflected image, and the capture optimization is achieved by dynamically tracking the reflection data.

[0010] The data processing module pre-processes the received reflection data, filters invalid or erroneous reflection data, converts it into point cloud data, calculates the 3D scanner pose transformation, merges point clouds from different viewpoints, performs density balancing and area division, and removes outliers and details to optimize the point cloud data.

[0011] The data modeling module is used to generate a three-dimensional model of the target object based on the processed point cloud data, optimize the three-dimensional model through mesh smoothing and mesh layout, and perform target recognition and labeling.

[0012] Specifically, the specific steps of automatic adjustment of the projector include:

[0013] Initialize the projector parameters and use the camera in the 3D scanner to capture the reflection image of the target object to obtain the reflection intensity of the target object. The reflection intensity is calculated as follows:

[0014]

[0015] in, Pixel The reflection intensity at is the pixel point in the reflected image The brightness at To project light patterns on pixels The brightness at

[0016] Evaluate the reflected image by the reflection intensity The mean and standard deviation of are used to calculate the overall brightness and contrast of the target object surface and automatically adjust the projector parameters:

[0017]

[0018]

[0019] in, is the adjusted projector brightness, is the initial projector brightness, is the mean brightness of the reflected image, is an adjustable non-negative proportional coefficient of brightness, is the mean brightness of the target reflection image, is the adjusted projector contrast, is the initial projector contrast, is the discrete degree of the reflected image, is an adjustable non-negative contrast scaling factor, is the contrast of the target reflected image;

[0020] According to the adjusted projector brightness and contrast, the target object is projected, and the projected image is obtained and automatically adjusted until it converges to the error range, that is:

[0021]

[0022]

[0023] in, and is the convergence threshold.

[0024] Specifically, the steps of obtaining the acquisition distance and evaluating whether the acquisition distance meets the scanning rules include:

[0025] Acquire a reflected image, extract key features of the light pattern in the reflected image, perform light pattern offset analysis based on the key features of the light pattern in the reflected image, and acquire an offset amount of the reflected image;

[0026] Using triangulation, the acquisition distance from the target object is calculated based on the shape of the projector's projected light pattern and the changes in the reflected pattern captured by the camera, that is:

[0027]

[0028]

[0029] in, For the The acquisition distance measured by the light pattern, is the focal length of the camera, It is The light pattern shifts, is the number of light patterns per unit length, is the acquisition distance, is the number of light patterns used to calculate the acquisition distance;

[0030] Configure the distance threshold. If the collection distance is within the distance threshold range, that is:

[0031]

[0032] The first display is performed through a three-dimensional scanner, wherein: is the distance threshold, is the acquisition distance, It is a non-negative distance parameter used to control the allowable range of the acquisition distance. If the acquisition distance is less than the difference between the distance threshold and the distance parameter, the second display is performed through the 3D scanner, otherwise the third display is performed.

[0033] Specifically, the automatic adjustment of the camera shutter speed includes:

[0034] The reflection intensity and acquisition distance are obtained based on the reflection image, and the shutter speed is dynamically adjusted in combination with the camera ISO setting, aperture, and exposure value, namely:

[0035]

[0036] in, It is The shutter speed of the secondary camera, is a non-negative constant, It is Secondary projected light intensity, It is 1st reflected light intensity, It is 1 collection distance, is the atmospheric attenuation coefficient, ISO is the ISO setting of the camera, is the exposure value, is the aperture value.

[0037] Specifically, the steps for dynamically tracking reflection data include:

[0038] By utilizing the position sensor of the 3D scanner, the position coordinates of the 3D scanner are recorded in real time to generate the motion trajectory of the 3D scanner;

[0039] When the reflected image is received, the current 3D scanner coordinates are compared with each coordinate point in the 3D scanner motion trajectory to determine whether the position coordinates of the 3D scanner are within the motion trajectory;

[0040] If the current three-dimensional scanner position is in the trajectory, a similarity measurement is performed between the currently captured reflection image and the reflection image at the adjacent position in the trajectory;

[0041] Configure a similarity threshold. If the similarity measurement result of the reflected image exceeds the similarity threshold, a signal warning will be issued to inform the operator that there is redundant data in the current scan. If the similarity measurement result of the reflected image does not exceed the similarity threshold, the currently captured image is considered to be new valid data and the reflected image acquisition will continue.

[0042] Based on the recorded reflection images and scanning paths, a real-time thermal map is generated to show the distribution of scanned and unscanned areas. The color of the thermal map is represented by the number of scans and the quality index of the reflection image.

[0043] Specifically, the specific steps of performing data preprocessing on the received reflection data include:

[0044] Based on the reflection images captured by the camera, reflection data of each reflection image is obtained, where the reflection data includes the reflection intensity and acquisition distance of each pixel point obtained based on the reflection image;

[0045] Configure the intensity range threshold and distance range threshold to perform preliminary data filtering on the reflection data, and remove pixels in the reflection image whose reflection intensity and acquisition distance do not meet the intensity range threshold and distance range threshold, that is:

[0046]

[0047] Among them, if the pixel coordinates are Filter coefficient When the value is 1, it means that the pixel point is retained. If the pixel coordinates are Filter coefficient When the value is 0, it means that the pixel is removed. The pixel coordinates are The reflection intensity, The pixel coordinates are The collection distance, [ R min , R max ] is the intensity range, [ d min , d max ] is the distance range;

[0048] Convert to point cloud data, including: according to the reflection data filtered by the preliminary data, convert the reflection image into point cloud data, and convert the three-dimensional coordinates of all pixel coordinate points filtered by the preliminary data and reflection intensity As point cloud data output, each point cloud data point is represented as:

[0049]

[0050] in, It is The three-dimensional coordinates of valid point cloud data, is the reflection intensity at that point.

[0051] Specifically, the specific steps of performing data preprocessing on the received reflection data also include:

[0052] According to the motion trajectory of the 3D scanner during the capture process, the 3D scanner posture at each reflection data acquisition is calculated, and the 3D scanner posture is a transformation matrix represented by a rotation vector and a translation vector;

[0053] Based on the last reflection data acquisition and the current 3D scanner pose, the transformation matrix from the last acquisition coordinate system to the current coordinate system is calculated. According to the transformation matrix, the point cloud data is transformed into the global coordinate system. The transformation matrix is expressed as:

[0054]

[0055] in, It is from The coordinate system of the first acquisition to the The transformation matrix of the coordinate system at the time of acquisition, It is The inverse transformation matrix of the 3D scanner posture at the time of acquisition, It is The 3D scanner pose transformation matrix during the acquisition.

[0056] Specifically, the steps for merging point clouds from different viewpoints and performing density balancing and area division include:

[0057] Merge point cloud data from different perspectives to construct a complete global point cloud data. Calculate the density of each point cloud by the number of neighboring point clouds of each point cloud, that is:

[0058]

[0059] in, It is Point Cloud The density, is the neighborhood volume used to calculate density, is the neighborhood radius used to calculate density, is an indicator function, if If true, the value is 1, otherwise the value is 0. It is the first in the global point cloud data point clouds, It is a point cloud and the distance between them;

[0060] Based on the density of the point cloud, the global point cloud data is divided into sparse areas, normal areas and dense areas. The density of different types of areas is adaptively balanced. For sparse areas, a distance-based weighted interpolation method is used to generate new point cloud data based on the point cloud data with a closer distance. For dense areas, a voxel grid downsampling method is used to reduce the point cloud density.

[0061] The spatial position analysis of the global point cloud data after adaptive density equalization is performed, the clustering algorithm is used to remove the deviated outliers, and the edge preservation algorithm is used to remove details.

[0062] Specifically, the steps of 3D model generation and optimization include:

[0063] Perform 3D mesh reconstruction on the point cloud data converted by the data processing module, convert the discrete point cloud data into a triangular mesh, and use a constraint algorithm to maximize the minimum angle of each triangular mesh;

[0064] Calculate the normal vector of each triangle mesh vertex. For each vertex, its normal vector is the weighted average of the normal vectors of the adjacent triangle facets where the vertex is located. By calculating the difference angle between the normal vector of each vertex and the normal vector of the adjacent triangle, the local curvature of the triangle mesh vertex is estimated, that is:

[0065]

[0066] in, It is The local curvature of the vertex, It is The total area of the adjacent triangles of the vertices, Used to indicate the The set of adjacent triangles of vertices, It is The normal vector of the vertex and the The difference angle between the normal vectors of adjacent triangles.

[0067] Specifically, the specific steps of 3D model generation and optimization also include:

[0068] Adaptively smooth the triangle mesh vertices and adjust the smoothing coefficient using the local curvature of the mesh vertices, that is:

[0069]

[0070] in, It is The smoothing coefficient of each vertex, It is The local curvature of the vertex, Is a non-negative constant that controls the influence of curvature; for each triangle mesh vertex, its position is updated according to the coordinates of other vertices in its neighborhood and the local curvature. The update formula is:

[0071]

[0072] in, It is the updated The position of the vertices, It is The original positions of the vertices, It is The set of other vertices in the neighborhood of a vertex, The first The position of the vertices;

[0073] Use optimized topology detection algorithms to detect mesh self-intersections and identify overlapping areas. If the mesh has self-intersections or unreasonable connections, perform topology repair.

[0074] The target recognition and labeling includes: using a deep learning model to automatically label the target area in the three-dimensional model, and providing an interactive interface for manual labeling through a visual interface.

[0075] Beneficial effects of the present invention:

[0076] 1. By automatically adjusting projector parameters to ensure optimal reflected image quality, the system provides more accurate data input, helps generate high-precision 3D models, reduces human error, and improves diagnostic accuracy. It also automatically adjusts shutter speed based on the reflective properties of the target object and the acquisition distance, ensuring high-quality reflected images under different lighting conditions, further improving data accuracy.

[0077] 2. The position sensor records the position coordinates of the 3D scanner in real time and generates a motion trajectory, which helps to accurately track changes in the reflected image, achieve dynamic adjustment and optimization, configure similarity thresholds, measure the similarity of the captured reflected images, identify and remove redundant data, reduce unnecessary repeated scans, and improve data collection efficiency. Based on the recorded reflected images and scanning paths, a real-time heat map is generated to intuitively display the distribution of scanned and unscanned areas, helping operators better plan the scanning path and avoid missed or repeated scans.

[0078] 3. Preliminary filtering of the reflection data is performed using intensity range thresholds and distance range thresholds to remove unqualified data points. The reflection image is then converted into point cloud data to ensure data quality for subsequent processing. The pose of each data acquisition is calculated based on the motion trajectory of the 3D scanner, and the point cloud data is transformed into a global coordinate system to construct a complete global point cloud data, providing accurate basic data for 3D reconstruction. The point cloud data of different regions is processed using an adaptive density equalization algorithm to remove deviated outliers and details, optimize the structure of the point cloud data, and improve the smoothness and consistency of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0079] Figure 1 This is a schematic structural diagram of a three-dimensional scanning system for medical use according to the present invention;

[0080] Figure 2 A flowchart of specific steps for automatic adjustment of the projector of the present invention;

[0081] Figure 3 A flowchart of the specific steps of acquiring the acquisition distance and evaluating whether the acquisition distance meets the scanning rules of the present invention;

[0082] Figure 4 A flowchart of the specific steps of dynamic tracking of reflection data of the present invention;

[0083] Figure 5 is a flowchart of the specific steps in the data processing module of the present invention;

[0084] Figure 6 Flowchart of the specific steps for generating and optimizing the three-dimensional model of the present invention. DETAILED DESCRIPTION

[0085] See also Figure 1 ,This embodiment introduces a three-dimensional scanning system for the medical field, including a signal transmitting module, a data receiving module, a data processing module, a data modeling module and a data storage module;

[0086] The signal transmission module is used to project a light pattern onto a target object through a projector in the 3D scanner, collect the reflection intensity of the target object through a camera in the 3D scanner, and automatically adjust the brightness and contrast of the projector according to the reflection intensity;

[0087] The data receiving module is used to capture the reflected image of the light pattern through the camera in the 3D scanner, and obtain the acquisition distance based on the reflected image. When the acquisition distance meets the scanning rules, the camera shutter speed is automatically adjusted according to the reflection properties of the target object light pattern to capture the reflected image, and the capture optimization is achieved by dynamically tracking the reflection data.

[0088] The data processing module pre-processes the received reflection data, filters invalid or erroneous reflection data, converts it into point cloud data, calculates the 3D scanner pose transformation, merges point clouds from different viewpoints, performs density balancing and area division, and removes outliers and details to optimize the point cloud data.

[0089] The data modeling module is used to generate a three-dimensional model of the target object based on the processed point cloud data, optimize the three-dimensional model through mesh smoothing and mesh layout, and perform target recognition and labeling.

[0090] The data storage module is used to store reflection data and its processing results. It optimizes storage by compressing reflection data and provides cloud storage options to achieve data sharing.

[0091] In this embodiment, a projector projects a light pattern onto the surface of a target object. The light pattern can include stripes or a grid. Target objects have different materials and reflective properties, and the projected light pattern produces different effects depending on their reflective properties. To ensure accurate data acquisition, the signal transmission module automatically adjusts the projector's brightness and contrast based on the target object's surface reflective properties, preventing excessive or insufficient light from affecting data acquisition and ensuring that the projected light pattern remains clearly visible even in complex environments. Automatic brightness and contrast adjustment ensures a clear and stable reflection pattern on all target surfaces, significantly improving the quality and stability of data acquisition. A camera captures the light pattern reflected from the target object's surface. The camera's resolution and frame rate play a key role in the system. Based on the target object's reflective properties, the data receiving module automatically adjusts the camera's shutter speed to optimize the capture of the reflected image. When the acquisition distance meets the scanning criteria—that is, the distance between the target object and the 3D scanner is within a valid range—the shutter speed is dynamically adjusted to ensure accurate capture of varying reflection intensities, further improving scanning accuracy and efficiency, thereby achieving high-quality 3D data acquisition. After receiving the data, the data processing module preprocesses the reflection data, including removing noise, repairing missing data, and filtering out invalid or erroneous reflection information. Using efficient algorithms, the data processing module optimizes the scanning mesh, enhancing the detail and accuracy of the scan results. Furthermore, the data processing module integrates and registers the reflection data, ensuring that reflection data captured from different angles is correctly merged to generate a unified 3D dataset. This significantly improves the quality of the scan data, enabling more accurate subsequent modeling and analysis, and better capturing the details of the target object. The processed reflection data is then transmitted to the data modeling module, where a 3D model is constructed through feature selection and feature removal. During this process, feature points in the reflection data are precisely screened, removing excess noise and irrelevant features, and constructing high-quality 3D point cloud data. Furthermore, mesh smoothing and layout optimization effectively repair defects caused by missing data or uneven reflections, resulting in a smoother and more accurate 3D model. Target areas can be automatically identified and annotated, providing decision support for physicians. To ensure data security and shareability, the data storage module stores the reflection images and their processed results in various ways. Reflected images are compressed and optimized for storage, effectively saving storage space while maintaining data integrity and availability. Cloud storage options allow users to access, share, and back up scan data at any time, allowing for subsequent analysis when needed. Cloud storage also enables convenient collaboration among medical teams across regions, enhancing the system's remote diagnosis and treatment capabilities.

[0092] See also Figure 2 Preferably, the specific steps of automatically adjusting the projector include:

[0093] Initialize the projector parameters, including light pattern type, light source mode, projector brightness, and projector contrast. Light pattern types include stripes, checkerboard, and dot patterns; light source modes include standard white light and laser projection. By initializing the parameters, ensure that the projector is in a known state at the beginning, laying the foundation for subsequent automatic adjustment.

[0094] Use the camera to collect the reflection data of the target object to obtain the reflection intensity of the target object. The calculation formula of the reflection intensity is as follows:

[0095]

[0096] in, Pixel The reflection intensity at is the pixel point in the reflected image The brightness at To project light patterns on pixels The brightness at the location where the target object is located is obtained; the surface reflection of the target object under actual lighting conditions is obtained to provide data support for subsequent evaluation and adjustment.

[0097] Evaluate the reflected image by the reflection intensity The mean and standard deviation of are used to calculate the overall brightness and contrast of the target object surface to evaluate the projection effect, namely:

[0098]

[0099]

[0100] in, is the mean brightness of the reflected image, which is used to represent the average brightness of the entire reflected image. is the discrete degree of the reflected image, which is used to evaluate the contrast of the image. is the number of sampling points, For the The pixel where the sampling point is located By calculating the brightness mean and standard deviation, we can quantitatively understand the quality of the current projection effect and provide a basis for automatic adjustment.

[0101] Automatically adjust projector parameters based on the brightness and contrast of the reflected image:

[0102]

[0103]

[0104] in, is the adjusted projector brightness, is the initial projector brightness, is the mean brightness of the reflected image, It is an adjustable non-negative proportional coefficient of brightness, which is used to control the sensitivity of brightness adjustment. is the average brightness of the target reflection image, set according to specific needs, is the adjusted projector contrast, is the initial projector contrast, is the discrete degree of the reflected image, It is an adjustable non-negative contrast ratio coefficient used to control the sensitivity of contrast adjustment. It is the contrast of the target reflected image, which is set according to specific needs; the projector parameters are dynamically adjusted to adapt to different lighting environments and visual needs, and to improve the stability and consistency of the projection effect.

[0105] According to the adjusted projector brightness and contrast, the target object is projected, and the projected image is obtained and automatically adjusted until it converges to the error range, that is:

[0106]

[0107]

[0108] in, is the adjusted projector brightness, is the adjusted projector contrast, is the mean brightness of the target reflection image, is the contrast of the target reflected image, and The convergence threshold is set. Through continuous monitoring and adjustment, the projection effect is gradually optimized to ensure that the final result meets the expected visual effect requirements.

[0109] See also Figure 3 Preferably, the specific steps of obtaining the acquisition distance and evaluating whether the acquisition distance meets the scanning rules include:

[0110] Acquire a reflected image and extract key features of the light pattern in the reflected image, including the position of the light spot, brightness distribution, and fringe offset. The light spot position refers to the position of the projected pattern on the target surface, and the brightness distribution of the light spot reflects the reflection properties between the target surface and the projector. If the projected light pattern contains stripes or grids, the deformation degree of the stripes can be used to estimate the changes on the target object surface.

[0111] Based on the key features of the light pattern in the reflected image, the light pattern offset analysis is performed to obtain the offset of the reflected image. If the light pattern type is stripes, the center position of each stripe is obtained through peak detection, and the displacement between adjacent stripes is extracted to obtain the offset of the reflected image, that is:

[0112]

[0113]

[0114] in, It is The center position of the stripes Hedi The center position of the stripes The displacement between is the pitch of the light pattern projected by the projector, It is The displacement of the light pattern reflects the change of the light pattern on the target surface and can provide quantitative information about the deformation of the target surface.

[0115] Using triangulation, the acquisition distance from the target object is calculated based on the shape of the projector's projected light pattern and the changes in the reflected pattern captured by the camera, that is:

[0116]

[0117]

[0118] in, For the The acquisition distance measured by the fringes, is the focal length of the camera, It is The stripe displacement, is the number of fringes per unit length, is the acquisition distance, It is the number of fringes used to calculate the acquisition distance; by averaging multiple point measurements, the error influence of a single point can be reduced.

[0119] Configure the distance threshold. If the collection distance is within the distance threshold range, that is:

[0120]

[0121] The first display is performed through a three-dimensional scanner, wherein: is the distance threshold, is the acquisition distance, is a non-negative distance parameter used to control the allowable range of the acquisition distance. If the acquisition distance is less than the difference between the distance threshold and the distance parameter, the second display is performed through the 3D scanner, otherwise the third display is performed; the first display, second display and third display can be displayed by configuring green, yellow and red display lights in the 3D scanner.

[0122] Preferably, the automatic adjustment of the camera shutter speed specifically includes:

[0123] The reflection intensity and acquisition distance are obtained based on the reflection image, and the shutter speed is dynamically adjusted in combination with the camera ISO setting, aperture, and exposure value to achieve optimal exposure, namely:

[0124]

[0125] in, It is The shutter speed of the secondary camera, is a non-negative constant related to the hardware characteristics of the camera. It is Secondary projected light intensity, It is Secondary reflected light intensity, It is Secondary collection distance, is the atmospheric attenuation coefficient, which reflects the degree of absorption or scattering of light by the air. ISO is the ISO setting of the camera, which controls the sensitivity. is the exposure value, which is used to express the logarithmic value of light brightness. The aperture value is adjusted in real time through feedback control, optimizing exposure, reducing blur, improving image clarity, and effectively responding to environmental changes, enabling more stable and accurate image acquisition in dynamic scenes.

[0126] See also Figure 4 Preferably, the specific steps of dynamically tracking the reflection data include:

[0127] By utilizing the position sensor of the 3D scanner, the position coordinates of the 3D scanner are recorded in real time, and mathematical modeling or trajectory analysis algorithms are used to generate the motion trajectory of the 3D scanner to reflect the movement path of the 3D scanner in space. The motion trajectory includes timestamps, position coordinates and other motion parameters; it provides accurate spatial coordinate data for image capture and position judgment, ensuring the integrity and accuracy of the scanning path.

[0128] When the reflected image is received, the current 3D scanner coordinates are compared with each coordinate point in the 3D scanner motion trajectory to determine whether the position coordinates of the 3D scanner are within the motion trajectory, that is:

[0129]

[0130] in, is the coordinate set of the 3D scanner motion trajectory, It is The coordinates of the 3D scanner when the secondary reflection image is captured, is the distance in the motion trajectory of the 3D scanner The nearest trajectory point, if , then it is considered that the current position coordinates of the 3D scanner are located in the motion trajectory. It is the position tolerance threshold; it ensures the consistency of the spatial positioning of the image data and the scanning path, thereby reducing the data deviation caused by position errors.

[0131] If the current 3D scanner position is in the trajectory, similarity measurement is performed between the currently captured reflection image and the reflection image at the adjacent position in the trajectory, and the similarity measurement methods include mutual information, normalized cross-correlation, mean square error and structural similarity index;

[0132] Configure a similarity threshold. If the reflection image similarity measurement result exceeds the similarity threshold, it means that the current reflection image is similar to the reflection image at the adjacent position, and there is duplicate reflection image acquisition. Then a signal warning is issued. The signal warning includes visual and audio signals to notify the operator that there is redundant data in the current scan.

[0133] If the similarity measurement result of the reflected image does not exceed the similarity threshold, the currently captured image is considered to be new valid data, and the image acquisition and subsequent processing steps are continued. The motion trajectory of the 3D scanner is continued to be monitored to ensure the consistency of the spatial position of the image data with the scanning path.

[0134] Based on the recorded reflection images and scanning paths, a real-time heat map is generated to show the distribution of scanned and unscanned areas. The color of the heat map is represented by the number of scans and the quality index of the reflection image. For example, the hotter the color, the more intensive the scan, and the colder the color, the less scanned the area. This helps operators optimize scanning strategies, avoid repeatedly scanning the same location, and improve efficiency.

[0135] See also Figure 5 Preferably, the specific steps of performing data preprocessing on the received reflection data include:

[0136] Based on the reflection images captured by the camera, the reflection data of each reflection image is obtained. The reflection data includes the reflection intensity and acquisition distance of each pixel point obtained based on the reflection image, providing basic data for subsequent processing;

[0137] Configure the intensity range threshold and distance range threshold to perform preliminary data filtering on the reflection data, and remove pixels in the reflection image whose reflection intensity and acquisition distance do not meet the intensity range threshold and distance range threshold, that is:

[0138]

[0139] Among them, if the pixel coordinates are Filter coefficient When the value is 1, it means that the pixel point is retained. If the pixel coordinates are Filter coefficient When the value is 0, it means that the pixel is removed. The pixel coordinates are The reflection intensity, The pixel coordinates are The collection distance, [ R min , R max ] is the intensity range, [ d min , d max ] is the distance range; it is used to remove pixels that do not meet the conditions and retain reflection data points that meet the requirements.

[0140] Converting to point cloud data includes: converting the reflection image into point cloud data according to the reflection data filtered by the preliminary data, and converting it into point cloud data. , calculate its corresponding three-dimensional coordinates ,Right now:

[0141]

[0142]

[0143]

[0144] in, is the corresponding pixel coordinate The acquisition distance indicates the distance between the object surface at the pixel and the camera. is the optical center coordinate, which represents the origin in the camera coordinate system and is located at the center of the reflected image. It is a reference point in the two-dimensional image coordinate system and represents the intersection of the camera coordinate system and the image plane. and is the focal length, which indicates the imaging characteristics of the camera in the horizontal and vertical directions; the three-dimensional coordinates of all pixel coordinate points filtered by the preliminary data and reflection intensity As point cloud data output, each point cloud data point is represented as:

[0145]

[0146] in, It is The three-dimensional coordinates of valid point cloud data, is the reflection intensity at that point;

[0147] According to the motion trajectory of the 3D scanner during the capture process, the 3D scanner posture at each reflection data acquisition is calculated. The 3D scanner posture is a transformation matrix represented by a rotation vector and a translation vector, namely:

[0148]

[0149] in, It is The 3D scanner pose transformation matrix during acquisition, is the rotation matrix, which is used to describe the orientation of the 3D scanner. is the translation vector, used to describe the position of the 3D scanner, and They are all vectors, representing the lower half of the transformation matrix; by capturing the posture of the 3D scanner and tracking the movement of the scanner, the spatial position of the point cloud data is ensured to be consistent with the actual scanning position. Through precise posture calculation, the spatial consistency of the point cloud data is guaranteed, which can reflect the actual 3D structure.

[0150] Based on the last reflection data acquisition and the current 3D scanner pose, the transformation matrix from the last acquisition coordinate system to the current coordinate system is calculated. According to the transformation matrix, the point cloud data is transformed into the global coordinate system. The transformation matrix is expressed as:

[0151]

[0152] in, It is from The coordinate system of the first acquisition to the The transformation matrix of the coordinate system at the time of acquisition, It is The inverse transformation matrix of the 3D scanner posture at the time of acquisition, It is The 3D scanner posture transformation matrix during the first acquisition; by calculating the coordinate transformation matrix, each point cloud data is transformed from the local coordinate system to the global coordinate system, so that the point clouds under different acquisition perspectives can be processed in the same reference frame.

[0153] After completing the coordinate transformation of the point cloud data, merge the point cloud data from different perspectives to construct a complete global point cloud data. The density of each point cloud is calculated by the number of neighboring point clouds of each point cloud, that is:

[0154]

[0155] in, It is Point Cloud The density, is the neighborhood volume used to calculate density, is the neighborhood radius used to calculate density, is an indicator function, if If true, the value is 1, otherwise the value is 0. It is the first in the global point cloud data point clouds, It is a point cloud and the distance between them;

[0156] According to the density of the point cloud, the global point cloud data area is divided into sparse areas, normal areas and dense areas, and adaptive density equalization is performed on different types of areas. For sparse areas, a distance-based weighted interpolation method is used to generate new point cloud data based on point cloud data with a closer distance. For dense areas, the voxel grid downsampling method is used to reduce the point cloud density to avoid data redundancy; by calculating the neighborhood density of each point, the distribution of the point cloud is evaluated, and the point cloud density is adjusted through the adaptive density equalization method to balance the density distribution of the point cloud and avoid insufficient data in sparse areas or redundant data in dense areas.

[0157] The spatial position analysis of the global point cloud data after adaptive density equalization is performed. The clustering algorithm is used to remove deviated outliers, and the edge-preserving algorithm is used to remove details, including surface details, holes and sharp edges, while maintaining the main structure of the point cloud to improve the quality of the point cloud data, remove outliers, and make the point cloud more accurately reflect the actual object surface.

[0158] See also Figure 6 Preferably, the specific steps of three-dimensional model generation and optimization include:

[0159] The point cloud data converted by the data processing module is reconstructed into a three-dimensional mesh, and the discrete point cloud data is converted into a triangular mesh. A constraint algorithm is used to maximize the minimum angle of each triangular mesh to help avoid the generation of sharp triangles, improve the quality and visualization of the mesh, avoid instability in subsequent processing, and ensure the smoothness and physical stability of the mesh.

[0160] Calculate the normal vector of each triangle mesh vertex. For each vertex, its normal vector is the weighted average of the normal vectors of the adjacent triangle facets where the vertex is located. By calculating the difference angle between the normal vector of each vertex and the normal vector of the adjacent triangle, the local curvature of the triangle mesh vertex is estimated, that is:

[0161]

[0162] in, It is The local curvature of the vertex, It is The total area of the adjacent triangles of the vertices, Is used to indicate the The set of adjacent triangles of vertices, It is The normal vector of the vertex and the The difference angle between the normal vectors of adjacent triangles; the local curvature can reflect the local geometric characteristics of the target surface, such as the degree of convexity, etc., to help subsequent smoothing and detail retention. The local curvature of the vertex plays a key role in identifying details, surface changes and retaining the effect during smoothing.

[0163] Adaptively smooth the vertices of the triangle mesh, and use the local curvature of the mesh vertex to adjust the smoothing coefficient. When the curvature is large, the smoothing coefficient is small, preserving the details; when the curvature is small, the smoothing coefficient is large, and stronger smoothing is performed, that is:

[0164]

[0165] in, It is The smoothing coefficient of each vertex, It is The local curvature of the vertex, It is a non-negative constant that controls the influence of curvature and determines the degree of influence of curvature on smoothing. It effectively removes surface noise and irregular fluctuations through local curvature while retaining the complex surface details, including edges and bumps. By adjusting the smoothing coefficient, it is possible to finely control which areas are smoothed more strongly and which areas maintain higher resolution and details.

[0166] For each triangle mesh vertex, its position is updated according to the coordinates of other vertices in its neighborhood and the local curvature. The update formula is:

[0167]

[0168] in, It is the updated The position of the vertices, It is The original positions of the vertices, It is The set of other vertices in the neighborhood of a vertex, The first The position of each vertex is updated; this position update process makes the surface of the mesh smoother and avoids over-smoothing in areas with large curvature, preserving the key details of the object. Ultimately, the mesh surface is more coherent and smooth, while maintaining the geometric characteristics of the original data.

[0169] An optimized topology detection algorithm is used to detect self-intersections and identify overlapping areas of the mesh. Topology detection algorithms include Marching Cubes and Dual Contouring. If the mesh has self-intersections or unreasonable connections, topology repair is performed, including: automatic merging of vertices that are close to each other, configuring a merge threshold for distance judgment, and merging two points into one vertex if the distance is less than the merge threshold. A topology optimization algorithm is used to automatically reconstruct triangles with self-intersections. The topology optimization algorithm includes vertex reordering and local constraint repair. This ensures the stability of the mesh's topological structure and avoids incorrect geometric shapes or unusable meshes in subsequent use.

[0170] Using deep learning models such as U-Net, it automatically labels target areas in 3D models, including organs and joints. It also provides an interactive interface for manual labeling through a visual interface, supporting functions such as region selection, label addition, and region editing.

[0171] Preferably, the specific steps of storing the reflection data and the processing results thereof include:

[0172] Capture the reflection intensity and acquisition distance of each reflection image and convert them into data points. Each point includes 3D coordinates and reflection intensity information. For each reflection image collected, save the data of each reflection point in a set format, including JSON and HDF5.

[0173] The filtered valid point cloud data is stored, including the 3D coordinates, reflection intensity, and pose information of each point at the time of acquisition, and the log of the filtering process is stored for subsequent backtracking and analysis.

[0174] Store point cloud data in the global coordinate system, record the relative position relationship between each point, store the point cloud density, area division, denoising and smoothing details after data processing, and store the point cloud data after adaptive density equalization in segments to facilitate subsequent modeling and analysis;

[0175] Stores the 3D model after mesh reconstruction, records the vertices, normal vectors, local curvature and other information of each triangular mesh, stores the mesh topology information and optimized results, including the overall structure and detailed information of the 3D model, and stores the annotation results of the deep learning model on the target area, including labels of characteristic areas such as organs and joints;

[0176] Upload data to the cloud server, provide a data sharing interface, allow remote access, set multi-level permissions to ensure data access security and privacy protection, provide a backup mechanism, perform data backup regularly, and avoid data loss.

[0177] Working principle and its effect:

[0178] The 3D scanning system used in the medical field performs high-precision optical 3D imaging, data acquisition, processing and modeling of the target object, and ultimately generates a 3D digital model that can be used for analysis, diagnosis or treatment. The collaborative work of multiple modules ensures the accuracy and efficiency of the scanning process.

[0179] During the signal transmission phase, a projector projects a light pattern onto the surface of the target object. The reflection of the light pattern is captured by the 3D scanner's camera. Based on the intensity of the reflected light, the projector's brightness and contrast are automatically adjusted to ensure the light pattern is clearly visible across different surface features, thereby ensuring high-quality image data. The camera captures these reflected images in real time, and based on changes in the reflected pattern, the distance between the target object and the camera is calculated. Using triangulation principles, the light pattern changes are analyzed by offset to accurately determine the distance and adjust the scanning settings. If the captured distance meets the preset scanning rules, the camera's shutter speed is automatically adjusted to ensure sufficient lighting and avoid over- or underexposure. Dynamic tracking technology is also used to optimize the image capture process. The received reflected images undergo preprocessing, including removing invalid or erroneous data, performing point cloud data conversion, calculating the scanner's pose transformation, and merging point clouds from different viewpoints to ensure uniform point cloud density and optimize details and outliers. This processed data becomes the basis for subsequent 3D modeling. The processed point cloud data is used to generate a 3D model of the target object. Through the mesh reconstruction algorithm, discrete point cloud data is converted into a triangular mesh, and the mesh is smoothed through local curvature analysis to optimize the surface quality of the model. In addition, the system will perform topological repair on the mesh to eliminate possible self-intersections and unreasonable connections to ensure the reasonable structure of the three-dimensional model. Through the above series of steps, the final generated three-dimensional model can be further used to identify and annotate target areas to support detailed analysis and diagnosis in the medical field. During the annotation process, deep learning technology can be used to automatically identify specific areas, while providing an interactive interface to support manual annotation and editing to improve the accuracy and practicality of the model.

[0180] By automatically adjusting projector parameters and camera settings, the system ensures the quality and consistency of scanned data. Secondly, by dynamically optimizing the acquisition process, duplicate data and redundant scans are avoided, improving work efficiency. Thirdly, optimized point cloud processing and model building techniques make the final 3D model more accurate and suitable for subsequent medical analysis. Finally, deep learning and interactive annotation capabilities enable the system to not only automatically generate models but also accurately annotate target areas, providing strong support for doctors.

[0181] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiment. All technical solutions based on the concept of the present invention are within the scope of protection of the present invention. It should be noted that for those skilled in the art, various improvements and modifications that do not depart from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A three-dimensional scanning system for medical use, characterized in that: It includes a signal transmission module, a data receiving module, a data processing module, a data modeling module and a data storage module; The signal transmission module is used to project a light pattern onto a target object through a projector in the 3D scanner, collect the reflection intensity of the target object through a camera in the 3D scanner, and automatically adjust the brightness and contrast of the projector according to the reflection intensity; The data receiving module is used to capture a reflected image of the light pattern through a camera in the 3D scanner, and use triangulation to obtain a collection distance based on the reflected image. When the collection distance meets the scanning rule, the camera shutter speed is automatically adjusted according to the reflection properties of the light pattern of the target object, the reflection intensity obtained based on the reflected image, and the collection distance to capture the reflected image. The motion trajectory of the 3D scanner is generated by dynamically tracking the reflection data. When the reflected image is received, the current 3D scanner coordinates are compared with each coordinate point in the motion trajectory of the 3D scanner to determine whether the position coordinates of the 3D scanner are within the motion trajectory. If the current 3D scanner position is within the trajectory, similarity measurement is performed between the currently captured reflection image and the reflection image at the adjacent position in the trajectory to determine whether there is redundant data in the current scan and achieve capture optimization; The data processing module performs data preprocessing on the received reflection data, filters invalid or erroneous reflection data, converts it into point cloud data, merges point cloud data from different perspectives, performs density balancing and area division, and removes outliers and details to optimize the point cloud data; The data modeling module is used to generate a three-dimensional model of the target object based on the processed point cloud data, optimize the three-dimensional model through mesh smoothing and mesh layout, and perform target recognition and annotation; The specific steps of automatically adjusting the projector include: Initialize the projector parameters and use the camera in the 3D scanner to capture the reflection image of the target object to obtain the reflection intensity of the target object. The reflection intensity is calculated as follows: ;in, Pixel The reflection intensity at is the pixel point in the reflected image The brightness at To project light patterns on pixels The brightness at Evaluate the reflected image by the reflection intensity The mean and standard deviation of are used to calculate the overall brightness and contrast of the target object surface and automatically adjust the projector parameters: ; ; in, is the adjusted projector brightness, is the initial projector brightness, is the mean brightness of the reflected image, is an adjustable non-negative proportional coefficient of brightness, is the mean brightness of the target reflection image, is the adjusted projector contrast, is the initial projector contrast, is the discrete degree of the reflected image, is an adjustable non-negative contrast ratio, is the contrast of the target reflected image; According to the adjusted projector brightness and contrast, the target object is projected, and the projected image is obtained and automatically adjusted until it converges to the error range, that is: ; ;in, and is the convergence threshold.

2. The three-dimensional scanning system for medical use according to claim 1, wherein: The specific steps of obtaining the acquisition distance and evaluating whether the acquisition distance meets the scanning rules include: Acquire a reflected image, extract key features of the light pattern in the reflected image, perform light pattern offset analysis based on the key features of the light pattern in the reflected image, and acquire an offset amount of the reflected image; Using triangulation, the acquisition distance from the target object is calculated based on the shape of the projector's projected light pattern and the changes in the reflected pattern captured by the camera, that is: in, For the The acquisition distance measured by the light pattern, is the focal length of the camera, It is The light pattern shifts, is the number of light patterns per unit length, is the acquisition distance, is the number of light patterns used to calculate the acquisition distance; Configure the distance threshold. If the collection distance is within the distance threshold range, that is: The first display is performed by a three-dimensional scanner, wherein: is the distance threshold, is the acquisition distance, It is a non-negative distance parameter used to control the allowable range of the acquisition distance. If the acquisition distance is less than the difference between the distance threshold and the distance parameter, the second display is performed through the 3D scanner, otherwise the third display is performed.

3. The three-dimensional scanning system for medical use according to claim 1, wherein: The automatic adjustment of the camera shutter speed specifically includes: The reflection intensity and acquisition distance are obtained based on the reflection image, and the shutter speed is dynamically adjusted in combination with the camera ISO setting, aperture, and exposure value, namely: in, It is The shutter speed of the secondary camera, is a non-negative constant, It is Secondary projected light intensity, It is Secondary reflected light intensity, It is Secondary collection distance, is the atmospheric attenuation coefficient, ISO is the ISO setting of the camera, is the exposure value, is the aperture value.

4. The three-dimensional scanning system for medical use according to claim 1, wherein: The specific steps of dynamically tracking the reflection data include: By utilizing the position sensor of the 3D scanner, the position coordinates of the 3D scanner are recorded in real time to generate the motion trajectory of the 3D scanner; When the reflected image is received, the current 3D scanner coordinates are compared with each coordinate point in the 3D scanner motion trajectory to determine whether the position coordinates of the 3D scanner are within the motion trajectory; If the current three-dimensional scanner position is in the trajectory, a similarity measurement is performed between the currently captured reflection image and the reflection image at the adjacent position in the trajectory; Configure a similarity threshold. If the similarity measurement result of the reflected image exceeds the similarity threshold, a signal warning will be issued to inform the operator that there is redundant data in the current scan. If the similarity measurement result of the reflected image does not exceed the similarity threshold, the currently captured image is considered to be new valid data and the reflected image acquisition will continue. Based on the recorded reflection images and scanning paths, a real-time thermal map is generated to show the distribution of scanned and unscanned areas. The color of the thermal map is represented by the number of scans and the quality index of the reflection image.

5. The three-dimensional scanning system for medical use according to claim 1, wherein: The specific steps of performing data preprocessing on the received reflection data include: Based on the reflection images captured by the camera, reflection data of each reflection image is obtained, where the reflection data includes the reflection intensity and acquisition distance of each pixel point obtained based on the reflection image; Configure the intensity range threshold and distance range threshold to perform preliminary data filtering on the reflection data, and remove pixels in the reflection image whose reflection intensity and acquisition distance do not meet the intensity range threshold and distance range threshold, that is: Among them, if the pixel coordinates are Filter coefficient When the value is 1, it means that the pixel point is retained. If the pixel coordinates are Filter coefficient When the value is 0, it means that the pixel is removed. The pixel coordinates are The reflection intensity, The pixel coordinates are The collection distance, is the intensity range, is the distance range; The conversion into point cloud data includes: performing data conversion on the reflected image according to the reflection data filtered by the preliminary data, converting the reflected image into point cloud data, and converting the three-dimensional coordinates of all pixel coordinate points filtered by the preliminary data into point cloud data. and reflection intensity As point cloud data output, each point cloud data point is represented as: in, It is The three-dimensional coordinates of valid point cloud data, is the reflection intensity at that point.

6. The three-dimensional scanning system for medical use according to claim 5, wherein: The specific step of performing data preprocessing on the received reflection data also includes: Calculating the 3D scanner posture at each reflection data acquisition according to the 3D scanner motion trajectory during the capture process, wherein the 3D scanner posture is a transformation matrix represented by a rotation vector and a translation vector; Based on the last reflection data acquisition and the current 3D scanner pose, the transformation matrix from the last acquisition coordinate system to the current coordinate system is calculated. According to the transformation matrix, the point cloud data is transformed into the global coordinate system. The transformation matrix is expressed as: in, It is from The coordinate system of the first acquisition to the The transformation matrix of the coordinate system at the time of acquisition, It is The inverse transformation matrix of the 3D scanner posture at the time of acquisition, It is The 3D scanner pose transformation matrix during the acquisition.

7. The three-dimensional scanning system for medical use according to claim 1, wherein: The specific steps of merging point cloud data from different viewpoints and performing density balancing and area division include: Merge point cloud data from different perspectives to construct a complete global point cloud data. Calculate the density of each point cloud by the number of neighboring point clouds of each point cloud, that is: in, It is Point Cloud The density, is the neighborhood volume used to calculate density, is the neighborhood radius used to calculate density, is an indicator function, if If true, the value is 1, otherwise the value is 0. It is the first point clouds, It is a point cloud and the distance between them; Based on the density of the point cloud, the global point cloud data is divided into sparse areas, normal areas and dense areas. The density of different types of areas is adaptively balanced. For sparse areas, a distance-based weighted interpolation method is used to generate new point cloud data based on the point cloud data with a closer distance. For dense areas, a voxel grid downsampling method is used to reduce the point cloud density. The spatial position analysis of the global point cloud data after adaptive density equalization is performed, the clustering algorithm is used to remove the deviated outliers, and the edge preservation algorithm is used to remove details.

8. The three-dimensional scanning system for medical use according to claim 1, wherein: The specific steps of generating and optimizing the three-dimensional model include: Perform 3D mesh reconstruction on the point cloud data converted by the data processing module, convert the discrete point cloud data into a triangular mesh, and use a constraint algorithm to maximize the minimum angle of each triangular mesh; Calculate the normal vector of each triangle mesh vertex. For each vertex, its normal vector is the weighted average of the normal vectors of the adjacent triangle facets where the vertex is located. By calculating the difference angle between the normal vector of each vertex and the normal vector of the adjacent triangle, the local curvature of the triangle mesh vertex is estimated, that is: in, It is The local curvature of the vertex, It is The total area of the adjacent triangles of the vertices, Used to indicate the The set of adjacent triangles of vertices, It is The normal vector of the vertex and the The difference angle between the normal vectors of adjacent triangles.

9. The three-dimensional scanning system for medical use according to claim 8, wherein: The specific steps of generating and optimizing the three-dimensional model also include: Adaptively smooth the triangle mesh vertices and adjust the smoothing coefficient using the local curvature of the mesh vertices, that is: in, It is The smoothing coefficient of each vertex, It is The local curvature of the vertex, Is a non-negative constant that controls the influence of curvature; for each triangle mesh vertex, its position is updated according to the coordinates of other vertices in its neighborhood and the local curvature. The update formula is: in, It is the updated The position of the vertices, It is The original positions of the vertices, It is The set of other vertices in the neighborhood of a vertex, The first The position of the vertices; Use optimized topology detection algorithms to detect mesh self-intersections and identify overlapping areas. If the mesh has self-intersections or unreasonable connections, perform topology repair. The target recognition and labeling includes: using a deep learning model to automatically label the target area in the three-dimensional model, and providing an interactive interface for manual labeling through a visual interface.

Images