Three-dimensional ultrasonic equipment integrating photoelectric sensor and motion sensor and reconstruction method

Abstract

The invention relates to the technical field of ultrasonic imaging, in particular to three-dimensional ultrasonic equipment integrating a photoelectric sensor and a motion sensor and a reconstruction method, and the equipment comprises an ultrasonic probe module which is used for obtaining a two-dimensional ultrasonic image sequence; the photoelectric sensing module is used for detecting the translational motion of the probe and generating planar motion information; the motion sensing module is used for detecting the rotary motion of the probe and generating space attitude information; the image processing module is used for performing feature extraction and segmentation on the image sequence through a pre-training image segmentation model to obtain a segmentation result; and the three-dimensional reconstruction module is used for performing space fusion and volume data reconstruction on the image and the segmentation result based on the motion and attitude information to generate a three-dimensional image. According to the invention, real-time high-precision tracking of the spatial pose of the probe is realized, the recognition capability of tissue boundaries and lesions is improved in combination with image segmentation, an accurate three-dimensional reconstruction image is generated through multi-source data fusion, and the visualization and diagnosis reliability of a complex structure is remarkably enhanced.

Description

Technical Field

[0001] This invention relates to the field of ultrasound imaging technology, specifically to a three-dimensional ultrasound device and reconstruction method that integrates photoelectric sensors and motion sensors. Background Technology

[0002] Ultrasound imaging technology, as an important tool in clinical diagnosis, has long relied primarily on two-dimensional image presentation. While this technology offers advantages such as real-time performance, ease of operation, and no radiation, it still has significant limitations in displaying complex anatomical structures and identifying lesions. Traditional two-dimensional ultrasound generates images through single-plane scanning, which can generally meet the needs of routine examinations, but its imaging results are limited by the acoustic window, highly dependent on the operator, and struggle to provide rich spatial and depth information, thus affecting the accuracy and reliability of diagnosis to some extent.

[0003] To improve image information and imaging quality, enhanced 2D imaging techniques have emerged in recent years. These techniques, through multi-angle composite scanning or 2D image stitching, have improved image resolution and information integrity to some extent. However, such improvements still fail to achieve true 3D imaging and cannot fundamentally solve the inherent limitations of traditional 2D ultrasound in spatial structure characterization, detection of minute lesions, and imaging of areas affected by bone or gas interference.

[0004] Therefore, how to overcome the dimensional limitations of two-dimensional imaging and improve the spatial identification ability of complex anatomical structures and lesions while maintaining the advantages of real-time performance and non-invasiveness of ultrasound equipment has become a key issue that urgently needs to be addressed in the development of current ultrasound imaging technology. Summary of the Invention

[0005] To address the above technical problems, this invention provides a technical solution for a three-dimensional ultrasound device and reconstruction method that integrates photoelectric sensors and motion sensors.

[0006] The technical problem solved by this invention can be achieved by the following technical solutions: A three-dimensional ultrasonic device integrating photoelectric sensors and motion sensors, comprising: An ultrasonic probe module is used to transmit and receive multi-frequency ultrasonic signals to obtain a sequence of two-dimensional ultrasonic images. The photoelectric sensing module is connected to the ultrasonic probe module and is used to detect the translational motion of the ultrasonic probe in real time and generate planar motion information. A motion sensing module, connected to the ultrasonic probe module, is used to detect the rotational motion of the ultrasonic probe in real time and generate spatial attitude information. The image processing module, connected to the ultrasound probe module, is used to extract features and segment the two-dimensional ultrasound image sequence using a pre-trained image segmentation model to obtain the segmentation result; The three-dimensional reconstruction module, connected to the photoelectric sensing module, the motion sensing module, and the image processing module, is used to perform spatial fusion and volume data reconstruction on the two-dimensional ultrasound image sequence and the segmentation results based on the planar motion information and the spatial posture information, so as to generate a three-dimensional reconstructed image.

[0007] Preferably, the photoelectric sensing module includes: The light source unit is used to emit illumination light onto the scanning surface; An image sensor is used to acquire a sequence of optical images formed after reflection from the scanned surface; An array signal processor, connected to the image sensor, is used to process the sequence of optical images and determine the direction and amount of movement of the ultrasound probe in the plane by calculating the correlation between consecutive images, so as to generate the planar motion information. The communication interface unit is connected to the array signal processor and is used to transmit the planar motion information to the three-dimensional reconstruction module.

[0008] Preferably, the motion sensing module includes: The data acquisition unit is used to acquire the acceleration and angular velocity measurements of the ultrasonic probe in the X, Y, and Z axes in real time. The motion processing unit, connected to the data acquisition unit, is used to calculate the static tilt angle of the ultrasonic probe based on the acceleration measurement value, and to perform time integration on the angular velocity measurement value to calculate the dynamic angle change of the ultrasonic probe, and to fuse the static tilt angle and the dynamic angle change to generate the spatial attitude information. The output interface unit is connected to the motion processing unit and is used to transmit the spatial attitude information to the three-dimensional reconstruction module.

[0009] Preferably, in the image processing module, the image segmentation model is a multi-scale feature aggregation neural network based on an encoder-decoder structure.

[0010] Preferably, the image processing module includes: The encoding unit is used to perform continuous convolution and downsampling operations on the input image through the encoder to extract multi-layer feature maps with progressively decreasing resolution; The multi-layer feature maps are L1 feature map, L2 feature map, L3 feature map, and L4 feature map, where L1 is the shallowest feature map with the highest resolution, and L4 is the deepest feature map with the lowest resolution. The decoding and segmentation unit, connected to the encoding unit, is used to perform multi-stage upsampling, cross-layer feature concatenation and convolution processing on the multi-layer feature map through the decoder, and output a segmentation map of corresponding resolution at each stage based on a deep supervision mechanism, wherein the segmentation map output in the final stage is used as the segmentation result.

[0011] Preferably, the decoding and segmentation unit performs the following operations: In the first segmentation stage, the L4 feature map is subjected to 1×1 convolution processing to generate a first segmentation map with a resolution of 1 / 8 of the input image. In the second segmentation stage, the L4 feature map is upsampled and then cascaded and fused with the L3 feature map to form the F3 feature. The F3 feature is then subjected to 1×1 convolution to generate a second segmentation map with a resolution of 1 / 4 of the input image. In the third segmentation stage, the L3 feature map is upsampled and then cascaded and fused with the L2 feature map to form the F2 feature. The F2 feature is then subjected to 1×1 convolution to generate a third segmentation map with a resolution of 1 / 2 of the input image. In the fourth segmentation stage, the L2 feature map is upsampled and then cascaded and fused with the L1 feature map to form the F1 feature. The F1 feature is then subjected to 1×1 convolution to generate a fourth segmentation map with the same resolution as the input image. The fourth segmentation map is used as the segmentation result.

[0012] Preferably, the image segmentation model is trained using a training module, the training module comprising: The image preprocessing unit is used to resample and linearly normalize each image in the two-dimensional ultrasound image sequence, and the specific formula is expressed as follows:

[0013] Where x represents the input image, This represents the image after normalization. This represents the maximum grayscale value of the image. Represents the minimum grayscale value of the image; The deep supervised training unit, connected to the image preprocessing unit, is used to input the preprocessed image into the image segmentation model, generate a weighted map corresponding to the resolution of each segmentation stage based on the ground truth segmentation mask, and calculate the weighted binary cross-entropy loss for each segmentation stage. The specific formula is as follows:

[0014] in, This represents the weight corresponding to each pixel. Represents the true value of a pixel. Represents the pixel prediction value; The loss fusion unit, connected to the deep supervised training unit, is used to sum the weighted binary cross-entropy losses from each segmentation stage according to preset weight coefficients to obtain the comprehensive loss. The specific formula is as follows:

[0015] in, (k=1,2,3,4) represents the weighted binary cross-entropy loss for the k-th stage segmentation; a, b, c, and d represent the segmentation loss weights for the four stages, where 0 <a<b<c<d; The model optimization unit, connected to the loss fusion unit, is used to iteratively update the parameters of the image segmentation model based on the comprehensive loss.

[0016] Preferably, in the deep supervised training unit, generating a weighted map corresponding to the resolution of each segmentation stage based on the segmentation ground truth mask includes: For the first segmentation stage and the second segmentation stage, by performing distance transformation on the segmentation ground truth mask, the distance of each pixel in the segmentation ground truth mask to the nearest boundary is calculated, and the obtained distance map is normalized by the maximum and minimum values ​​to generate a first weight map for enhancing the overall localization of the target area. For the third and fourth segmentation stages, the first weight map of the first segmentation stage or the second segmentation stage is reverse-weighted to generate a second weight map for enhancing edge detail learning.

[0017] Preferably, the three-dimensional reconstruction module includes: The coordinate system establishment unit is used to establish a three-dimensional physical space coordinate system with the first frame image of the two-dimensional ultrasound image sequence as a spatial reference, and to place the first frame image and its segmentation result in the three-dimensional physical space coordinate system; The inter-frame pose calculation unit, connected to the coordinate system establishment unit, is used to calculate the relative displacement and relative rotation angle of the current frame image relative to the previous frame image based on the planar motion information and the spatial pose information. A spatial mapping unit, connected to the inter-frame pose calculation unit, is used to map the current frame image and its segmentation result to the corresponding position in the physical space coordinate system according to the relative displacement and the relative rotation angle. The voxel fusion unit, connected to the spatial mapping unit, is used to detect whether there is a voxel intersection region between the mapped current frame image and the existing image frame in the physical space coordinate system. If there is, the pixel values ​​of the voxel intersection region are fused and the pixel values ​​of the first mapped frame are used as the final result. The global trajectory calculation unit, connected to the voxel fusion unit, is used to recursively calculate the cumulative displacement and cumulative rotation angle of each frame image relative to the first frame image based on the relative displacement and relative rotation angle of each frame image, so as to obtain the global pose of all image frames. A 3D volume data generation unit, connected to the global trajectory calculation unit, is used to generate 3D volume data based on the global pose. An interpolation optimization unit, connected to the three-dimensional volume data generation unit, is used to interpolate and fill missing regions in the three-dimensional volume data to generate the three-dimensional reconstructed image.

[0018] A method for reconstructing a three-dimensional ultrasound device integrating photoelectric sensors and motion sensors is also provided, applied to a three-dimensional ultrasound device integrating photoelectric sensors and motion sensors as described above, comprising: Step S1: Establish a three-dimensional physical space coordinate system based on the first frame of two-dimensional ultrasound image; Step S2: The two-dimensional ultrasound image sequence is acquired in real time through the ultrasound probe module, the planar motion information of the ultrasound probe is collected through the photoelectric sensing module, and the spatial attitude information of the ultrasound probe is collected through the motion sensing module. Step S3: Use the image segmentation model in the image processing module to perform frame-by-frame multi-scale feature extraction and segmentation on the two-dimensional ultrasound image sequence to obtain the segmentation result of each frame. Step S4: Based on the planar motion information and the spatial pose information, calculate the relative displacement and relative rotation angle of the current frame image relative to the previous frame image; Step S5: Based on the relative displacement and the relative rotation angle, map the current frame image and its segmentation result to the corresponding position in the three-dimensional physical space coordinate system; Step S6: Determine if the scan has ended. If not, return to step S2 to continue data acquisition and processing. If it has ended, proceed to step S7. Step S7: Perform three-dimensional volume data fusion and reconstruction based on the relative displacement and relative rotation angle of each frame image to generate and output a three-dimensional reconstructed image.

[0019] Beneficial effects: This invention achieves real-time, high-precision tracking of the spatial pose of the ultrasound probe by integrating photoelectric sensors and motion sensors, effectively solving the shortcomings of traditional two-dimensional ultrasound in spatial positioning and attitude perception. At the same time, by combining image segmentation models to extract and segment features from multiple frames of two-dimensional images, the accuracy of tissue boundary identification and the ability to detect minute lesions are significantly improved. Finally, through multi-source data fusion and volume data reconstruction algorithms, a three-dimensional image with accurate spatial information is generated, which greatly improves the visualization ability and diagnostic reliability of complex anatomical structures while maintaining the advantages of real-time ultrasound and non-invasiveness. Attached Figure Description

[0020] Figure 1 This is a block diagram of the three-dimensional ultrasound device of the present invention; Figure 2 This is a block diagram of the photoelectric sensing module of the present invention; Figure 3 This is a block diagram of the motion sensing module of the present invention; Figure 4 This is a block diagram of the image processing module of the present invention; Figure 5 This is a diagram of the image segmentation model of the present invention; Figure 6 This is a block diagram of the training module of the present invention; Figure 7 is a schematic diagram of distance transformation in the deep supervised training unit of the present invention; Figure 8 This is a block diagram of the three-dimensional reconstruction module of the present invention; Figure 9 This is a flowchart of the reconstruction method of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0023] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but this is not intended to limit the scope of the invention.

[0024] Reference Figure 1 This invention provides a three-dimensional ultrasonic device 10 integrating a photoelectric sensor and a motion sensor, comprising: Ultrasonic probe module 100 is used to transmit and receive multi-frequency ultrasonic signals to obtain two-dimensional ultrasonic image sequences; The photoelectric sensing module 200 is connected to the ultrasonic probe module 100 and is used to detect the translational motion of the ultrasonic probe in real time and generate planar motion information. The motion sensing module 300 is connected to the ultrasonic probe module 200 and is used to detect the rotational motion of the ultrasonic probe in real time and generate spatial attitude information. The image processing module 400, connected to the ultrasound probe module 300, is used to perform feature extraction and segmentation processing on the two-dimensional ultrasound image sequence using a pre-trained image segmentation model to obtain the segmentation result; The three-dimensional reconstruction module 500 is connected to the photoelectric sensing module 200, the motion sensing module 300, and the image processing module 400. It is used to perform spatial fusion and volume data reconstruction on the two-dimensional ultrasound image sequence and the segmentation result based on the planar motion information and the spatial posture information to generate a three-dimensional reconstructed image.

[0025] Specifically, in this embodiment of the invention, in response to the technical problems of incomplete spatial structure representation, strong operational dependence, and easy missed diagnosis of small lesions in traditional two-dimensional ultrasound imaging, a pose tracking mechanism that integrates photoelectric and motion sensing is used to capture the probe's motion trajectory in real time. Combined with an image segmentation model, key anatomical structures are extracted, avoiding the information loss caused by human scanning differences and single-plane imaging. This enables automatic stitching of multi-angle and multi-plane data and high-precision three-dimensional volume reconstruction, significantly improving the integrity of imaging complex areas and the reliability of diagnosis.

[0026] Specifically, in practical applications, the operator only needs to move the ultrasound probe in the clinical routine, and the device can automatically record the position and posture changes during the scanning process and simultaneously acquire two-dimensional image sequences; the image processing module 400 performs real-time or offline segmentation on each frame and identifies the target tissue boundary; the three-dimensional reconstruction module 500 accurately fuses the segmented two-dimensional sequence into a three-dimensional model based on the spatial pose data provided by the sensor, and finally generates a three-dimensional ultrasound image that can be used for multi-plane reconstruction and volume rendering.

[0027] As a preferred embodiment of the present invention, refer to Figure 2 The photoelectric sensing module 200 includes: The light source unit 210 is used to emit illumination light onto the scanning surface; Image sensor 220 is used to acquire a sequence of optical images formed after reflection from the scanned surface; An array signal processor 230, connected to the image sensor 220, is used to process the sequence of optical images and determine the direction and amount of movement of the ultrasound probe in the plane by calculating the correlation between consecutive images, so as to generate the planar motion information. The communication interface unit 240 is connected to the array signal processor 230 and is used to transmit the planar motion information to the three-dimensional reconstruction module.

[0028] Specifically, since traditional ultrasound probes are difficult to track their planar motion trajectory in real time and accurately during free scanning, resulting in the loss of spatial position information in the two-dimensional image sequence, the accuracy and stability of three-dimensional reconstruction are affected. In this embodiment of the invention, LED light-emitting diodes are preferably used as the light source unit 210 to provide a uniform and stable lighting environment for the scanning surface. At the same time, the image sensor 220 is used to acquire optical images at high frame rate and high resolution to capture the microscopic texture details of the scanning surface in real time to form a continuous image sequence. The digital signal processor (DSP) 230 uses intelligent image correlation algorithms to perform high-speed comparison and calculation on the continuous images to accurately analyze the translational motion trajectory of the ultrasound probe in the plane. Finally, the high-precision planar motion data is uploaded to the three-dimensional reconstruction module 500 through the communication interface unit 240.

[0029] More specifically, in the embodiments of the present invention, a high-performance optical sensor, such as PAW3395, is preferably used. It has high resolution, high frame rate and low power consumption characteristics, and supports intelligent surface adaptation technology. It can achieve stable and accurate position tracking under various tissue surfaces and scanning conditions, effectively ensuring the accuracy and reliability of the basic data required for three-dimensional reconstruction.

[0030] By introducing the aforementioned optical sensing mechanism, the problems of image registration misalignment and spatial positioning distortion caused by probe motion uncertainty in traditional ultrasound reconstruction are effectively solved. This technology directly captures the relative motion between the probe and the skin surface in a non-contact manner, which not only significantly reduces the dependence of motion information detection on operator technique and experience, but also provides high-precision and high-reliability spatial position coding for subsequent three-dimensional volume data reconstruction, fundamentally improving the system's adaptability to complex scanning paths and the geometric accuracy of the final three-dimensional imaging.

[0031] As a preferred embodiment of the present invention, refer to Figure 3 The motion sensing module 300 includes: The data acquisition unit 310 is used to acquire the acceleration and angular velocity measurements of the ultrasonic probe in the X, Y, and Z axis directions in real time. The motion processing unit 320, connected to the data acquisition unit 310, is used to calculate the static tilt angle of the ultrasonic probe based on the acceleration measurement value, and to perform time integration on the angular velocity measurement value to calculate the dynamic angle change of the ultrasonic probe, and to fuse the static tilt angle and the dynamic angle change to generate the spatial attitude information. The output interface unit 330 is connected to the motion processing unit 320 and is used to transmit the spatial attitude information to the three-dimensional reconstruction module.

[0032] Specifically, considering that a single optical sensor can only accurately detect the translational motion of the probe in a two-dimensional plane, but cannot effectively perceive its rotation and tilt attitude in three-dimensional space, this will result in the reconstructed three-dimensional model lacking correct spatial orientation information. In this embodiment of the invention, the accelerometer and gyroscope built into the data acquisition unit 310 synchronously acquire the original data of the probe's linear acceleration and rotational angular velocity in three-dimensional space, providing a high-precision data source for attitude calculation. The motion processing unit 320, as the core computing component, calculates the static attitude angle of the probe relative to the direction of gravity based on the acceleration data, and tracks the probe's real-time rotation and dynamic deflection by performing time-series integration on the gyroscope angular velocity data. Finally, data fusion algorithms such as complementary filtering or Kalman filtering are used to integrate the static angle and dynamic changes, outputting stable and accurate comprehensive spatial attitude information.

[0033] More specifically, in this embodiment of the invention, the MPU6050 is preferably used as the core motion sensor. Its highly integrated characteristics can provide high-precision six-axis motion sensing. With its excellent performance and stability, this sensor is widely used in consumer electronics and industrial equipment. It can effectively meet the high precision and high reliability requirements of ultrasonic probes for attitude measurement under complex scanning actions, and ensure the quality of attitude data required for three-dimensional reconstruction.

[0034] By combining the motion sensing module 300 with the photoelectric sensing module 200, a multi-source information fusion probe spatial pose full-parameter tracking system was constructed. Optical sensing provides high-precision planar displacement data, while motion sensing supplements the crucial three-dimensional rotational attitude information. This fusion strategy effectively overcomes the limitations of single-sensor technology, ensuring that the probe's spatial position (X, Y, Z translation) and attitude (pitch, yaw, roll rotation) can be captured synchronously and accurately under any free scanning path. This provides complete and reliable six-degree-of-freedom spatial transformation parameters for subsequent three-dimensional reconstruction algorithms, fundamentally solving the problems of three-dimensional model distortion, misalignment, and geometric distortion caused by missing attitude information, and significantly improving the system's robustness and reconstruction accuracy in complex clinical application scenarios.

[0035] In a preferred embodiment of the present invention, the image segmentation model in the image processing module 400 is a multi-scale feature aggregation neural network based on an encoder-decoder structure.

[0036] Specifically, considering the complexity of tissue structures, the diversity of lesion scales, and the ambiguity of boundary information in ultrasound images, traditional single-scale segmentation methods struggle to simultaneously consider both global contextual semantics and local detailed features. In this embodiment of the invention, an encoder-decoder network architecture that deeply integrates multi-scale features and introduces a deep supervision mechanism is employed. This network architecture mainly consists of two parts: an encoder and a decoder. The encoder extracts multi-scale features from the input image progressively through continuous convolution and downsampling operations, forming a multi-level feature representation from low to high. The decoder, on the other hand, gradually restores spatial resolution and fuses feature information from different scales through feature upsampling and cross-layer cascading operations.

[0037] This image segmentation model employs a deep supervision mechanism, generating an auxiliary segmentation output at each stage of the decoding process. This strengthens the extraction and constraint of key segmentation information from each layer of features, ensuring the network can effectively learn segmentation features from coarse to fine. Ultimately, by fusing high-level semantic information with low-level detail features, the model significantly enhances the representation ability of local fine structures while fully preserving the global contextual information of the image, thereby comprehensively improving the segmentation accuracy and robustness for complex tissue boundaries and small lesion areas.

[0038] As a preferred embodiment of the present invention, refer to Figure 4 The image processing module 400 includes: The encoding unit 410 is used to perform continuous convolution and downsampling operations on the input image through the encoder to extract multi-layer feature maps with progressively decreasing resolution. The multi-layer feature maps are L1 feature map, L2 feature map, L3 feature map, and L4 feature map, where L1 is the shallowest feature map with the highest resolution, and L4 is the deepest feature map with the lowest resolution. The decoding and segmentation unit 420 is connected to the encoding unit 410 and is used to perform multi-stage upsampling, cross-layer feature concatenation and convolution processing on the multi-layer feature map through the decoder, and output a segmentation map of corresponding resolution at each stage based on the deep supervision mechanism, wherein the segmentation map output in the final stage is the segmentation result.

[0039] Specifically, considering the complex tissue structures, blurred lesion boundaries, and varying scales in ultrasound images, direct single-resolution segmentation can easily lead to loss of detail and missegmentation. In this embodiment of the invention, a series of continuous convolution and downsampling operations are first performed by the encoding unit 410 to extract image features step by step. The input image (W×H) is processed through multiple convolutional blocks. After each downsampling layer, the spatial size of the feature map is halved, ultimately obtaining a feature representation with 1 / 8 of the original resolution (i.e., W / 8 × H / 8). During this process, the feature map output from each layer is further enhanced and compressed through 3×3 convolution, generating feature maps with different semantic levels and resolutions, labeled L1, L2, L3, and L4, where L1 is the shallowest layer with the highest resolution, and L4 is the deepest layer with the richest semantic information but the lowest resolution.

[0040] Next, the decoding and segmentation unit 420 employs a progressive upsampling structure symmetrical to the encoding path. In each segmentation stage, the deep feature map is upsampled and then concatenated across layers with the corresponding shallow high-resolution features, achieving the fusion of high semantic information and detailed features. This structure effectively compensates for the spatial details lost during downsampling, significantly improving the localization accuracy of target edges and fine structures, thereby generating more accurate segmentation results at different scales.

[0041] It is evident that by introducing this multi-scale encoding-decoding architecture and deep supervision mechanism, the image processing module 400 can fully utilize the complementary advantages of features at different levels. That is, it captures rich global contextual semantic information in the encoding path and gradually restores accurate spatial details and boundary positioning in the decoding path, ultimately achieving high-precision and robust segmentation of complex anatomical structures and minute lesions, significantly improving the automation level and diagnostic reliability of ultrasound image analysis.

[0042] In a preferred embodiment of the present invention, the decoding and segmentation unit 420 performs the following operations: In the first segmentation stage, the L4 feature map is subjected to 1×1 convolution processing to generate a first segmentation map with a resolution of 1 / 8 of the input image. In the second segmentation stage, the L4 feature map is upsampled and then cascaded and fused with the L3 feature map to form the F3 feature. The F3 feature is then subjected to 1×1 convolution to generate a second segmentation map with a resolution of 1 / 4 of the input image. In the third segmentation stage, the L3 feature map is upsampled and then cascaded and fused with the L2 feature map to form the F2 feature. The F2 feature is then subjected to 1×1 convolution to generate a third segmentation map with a resolution of 1 / 2 of the input image. In the fourth segmentation stage, the L2 feature map is upsampled and then cascaded and fused with the L1 feature map to form the F1 feature. The F1 feature is then subjected to 1×1 convolution to generate a fourth segmentation map with the same resolution as the input image. The fourth segmentation map is used as the segmentation result.

[0043] Specifically, because the target tissue structure in ultrasound images is complex, has large scale differences, and blurred boundaries, in order to gradually restore spatial details and achieve accurate segmentation, in this embodiment of the invention, referring to... Figure 5 The four segmentation stages of the decoding and segmentation unit 420 unfold sequentially, forming an upsampling reconstruction process symmetrical to the encoding path, specifically including: In the first segmentation stage, the deep semantic feature L4 is directly convolved with 1×1 to output a low-resolution (W / 8×H / 8) coarse segmentation map H1 to capture the approximate region of the target. In the second segmentation stage, L4 is upsampled and concatenated with L3 to form F3 features, which are then convolved with 1×1 to generate a medium-resolution (W / 4×H / 4) segmentation map H2, gradually restoring spatial details. In the third segmentation stage, L3 is further upsampled and concatenated with L2 to form F2 features. After convolution, a medium-to-high resolution (W / 2×H / 2) segmentation result H3 is obtained, which enhances the edge structure information. In the fourth segmentation stage, L2 upsampling is fused with high-resolution shallow features L1 to form F1, and then a fine segmentation map H4 with the same size as the original image (W×H) is output through 1×1 convolution. This fully integrates multi-scale context and detail features, thereby completing full-resolution segmentation prediction from coarse to fine.

[0044] As can be seen, this network architecture progressively compresses spatial dimensions and enriches semantic information through the encoder, and restores spatial details and fuses semantic information through step-by-step upsampling and cross-layer connections in the decoder. This progressive reconstruction mechanism, guided by multi-scale feature aggregation and deep supervision, achieves efficient integration of global contextual information and local detailed features, significantly improving the model's sensitivity to complex tissue boundaries and its ability to identify minute lesions, thus ensuring the accuracy and robustness of the final segmentation results.

[0045] As a preferred embodiment of the present invention, refer to Figure 6 The image segmentation model is trained using a training module 600, which includes: Image preprocessing unit 610 is used to resample and linearly normalize each image in the two-dimensional ultrasound image sequence; The deep supervised training unit 620 is connected to the image preprocessing unit 610. It is used to input the preprocessed image into the image segmentation model, generate a weighted image corresponding to the resolution of each segmentation stage based on the segmentation ground truth mask, and calculate the weighted binary cross-entropy loss of each segmentation stage respectively. The loss fusion unit 630, connected to the deep supervised training unit 620, is used to sum the weighted binary cross-entropy losses of each segmentation stage according to a preset weight coefficient to obtain the comprehensive loss. The model optimization unit 640, connected to the loss fusion unit 630, is used to iteratively update the parameters of the image segmentation model based on the comprehensive loss.

[0046] Specifically, in this embodiment of the invention, the input image is first preprocessed by the image preprocessing unit 610, which resamples it to a uniform size (e.g., 512×512 pixels) and normalizes the grayscale values ​​to between 0 and 1 using linear normalization. The specific formula is expressed as follows:

[0047] Where x represents the input image, This represents the image after normalization. This represents the maximum grayscale value of the image. This represents the minimum grayscale value of the image.

[0048] Next, the preprocessed image is input into the image segmentation model through the deep supervised training unit 620. A weighted map corresponding to the resolution of each segmentation stage is generated based on the ground truth segmentation mask, and the weighted binary cross-entropy loss (bce) for each segmentation stage is calculated. The bce loss function is expressed by the formula:

[0049] in, This represents the weight corresponding to each pixel. Represents the true value of a pixel. This represents the pixel prediction value.

[0050] Then, the weighted BCE losses of the four segmentation stages are weighted and summed according to preset weight coefficients by the loss fusion unit 630 to obtain the comprehensive loss, as shown in the following formula:

[0051] in, (k=1,2,3,4) represents the weighted binary cross-entropy loss for the k-th stage segmentation; a, b, c, and d represent the segmentation loss weights for the four stages, where 0 <a<b<c<d。

[0052] Finally, the model optimization unit 640 optimizes the model based on the comprehensive loss. The parameters of the image segmentation model are iteratively optimized to continuously minimize the comprehensive loss in order to improve the segmentation accuracy and generalization ability of the model, and finally a well-trained image segmentation model is obtained.

[0053] In a preferred embodiment of the present invention, the deep supervised training unit 620 generates a weighted map corresponding to the resolution of each segmentation stage based on the segmentation ground truth mask, including: For the first segmentation stage and the second segmentation stage, by performing distance transformation on the segmentation ground truth mask, the distance of each pixel in the segmentation ground truth mask to the nearest boundary is calculated, and the obtained distance map is normalized by the maximum and minimum values ​​to generate a first weight map for enhancing the overall localization of the target area. For the third and fourth segmentation stages, the first weight map of the first segmentation stage or the second segmentation stage is reverse-weighted to generate a second weight map for enhancing edge detail learning.

[0054] Specifically, since different resolution segmentation stages require attention to different image features, the low-resolution stages (first and second segmentation stages) focus on the overall localization of the target region, while the high-resolution stages (third and fourth segmentation stages) need to strengthen the learning of edge details. Therefore, in the deep supervised training unit 620, assuming the segmentation mask (i.e., the grayscale image identifying the target region) is as follows... Figure 7a and Figure 7b The ellipse shown is used to perform a distance transformation on the segmented target, that is, to calculate the nearest distance between each pixel within the ellipse and the boundary.

[0055] Subsequently, the distance-transformed mask (with pixel values ​​D) is normalized to its maximum and minimum values, and the minimum value of the distance map D is calculated. and maximum value Normalizing each pixel value in the distance map yields the first weight map used in the loss function for the first and second segmentation stages. The specific normalization formula is as follows: .

[0056] For the third and fourth segmentation stages, the first weighted graph will be... Combined with the original mask to generate a second weight graph This makes the loss function focus more on the edge region information of the segmented target. The specific transformation formula is expressed as: .

[0057] By employing the aforementioned differentiated weighting strategy, the network focuses on the overall existence of the target at a shallow level and on the fine detail of the boundary at a deep level during training. This effectively improves the feature learning ability and final segmentation accuracy of the image segmentation model at different resolutions.

[0058] As a preferred embodiment of the present invention, refer to Figure 8 The three-dimensional reconstruction module 500 includes: The coordinate system establishment unit 510 is used to establish a three-dimensional physical space coordinate system with the first frame image of the two-dimensional ultrasound image sequence as a spatial reference, and to place the first frame image and its segmentation result in the three-dimensional physical space coordinate system. The inter-frame pose calculation unit 520, connected to the coordinate system establishment unit 510, is used to calculate the relative displacement and relative rotation angle of the current frame image relative to the previous frame image based on the planar motion information and the spatial pose information. The spatial mapping unit 530, connected to the inter-frame pose calculation unit 520, is used to map the current frame image and its segmentation result to the corresponding position in the physical space coordinate system according to the relative displacement and the relative rotation angle. The voxel fusion unit 540 is connected to the spatial mapping unit 530 and is used to detect whether there is a voxel intersection region between the mapped current frame image and the existing image frame in the physical space coordinate system. If there is, the pixel values ​​of the voxel intersection region are fused and the pixel values ​​of the first mapped frame are used as the final result. The global trajectory calculation unit 550 is connected to the voxel fusion unit 540 and is used to recursively calculate the cumulative displacement and cumulative rotation angle of each frame image relative to the first frame image based on the relative displacement and relative rotation angle of each frame image, so as to obtain the global pose of all image frames. The three-dimensional volume data generation unit 560 is connected to the global trajectory calculation unit 550 and is used to generate three-dimensional volume data based on the global pose. The interpolation optimization unit 570 is connected to the three-dimensional volume data generation unit 560 and is used to interpolate and fill the missing regions in the three-dimensional volume data to generate the three-dimensional reconstructed image.

[0059] Specifically, in this embodiment of the invention, the three-dimensional reconstruction module 500 works collaboratively according to the following process: First, the coordinate system establishment unit 510 establishes a three-dimensional physical space coordinate system using the first frame image of the two-dimensional ultrasound image sequence as the spatial reference. Specifically, the plane containing the first frame image is set as the XOY plane, and the direction perpendicular to this plane is the Z-axis direction. This unit inputs the first frame image to the image processing module 400 to obtain its segmentation result, and places the image and the segmentation result together at the initial position of the three-dimensional coordinate system.

[0060] During continuous scanning, the inter-frame pose calculation unit 520 calculates the relative displacement (Δd) and relative rotation angle (Δθ) of the current frame (e.g., frame t+1) relative to the previous frame (frame t) based on the planar motion information (including offset direction and offset distance between consecutive frames) provided by the photoelectric sensing module 200 and the spatial pose information (including relative rotation angle between consecutive frames) provided by the motion sensing module 300, thereby obtaining the inter-frame relative pose.

[0061] The spatial mapping unit 530 performs corresponding translation and rotation transformations on the current frame image and its segmentation results according to the relative pose, and maps them to the corresponding positions in the three-dimensional physical space coordinate system.

[0062] Subsequently, the voxel fusion unit 540 detects whether there are voxel intersection regions between the mapped current frame and existing image frames in three-dimensional space. If an intersection exists, the pixel values ​​in that region are fused, following the "first scan priority" principle, with the pixel values ​​of the first mapped frame as the final result, effectively avoiding data conflicts caused by overlapping scan paths or excessive rotation angles.

[0063] After the scan is completed, the global trajectory calculation unit 550 recursively calculates the cumulative displacement and cumulative rotation angle of each frame relative to the first frame based on the relative pose information between each frame, thereby determining the global pose of all frames in a unified coordinate system.

[0064] The three-dimensional volume data generation unit 560 integrates all image data after voxel fusion processing to generate initial three-dimensional volume data based on the global pose.

[0065] Finally, the interpolation optimization unit 570 interpolates and fills in the missing regions in the initial 3D volume data caused by scan sparsity (e.g., using trilinear interpolation) to generate a complete and continuous 3D reconstructed image.

[0066] Through the coordinated operation of the above units, this invention achieves stable reconstruction from two-dimensional ultrasound sequences to high-precision, highly consistent three-dimensional models.

[0067] Reference Figure 9 The present invention also provides a method for reconstructing a three-dimensional ultrasound device integrating a photoelectric sensor and a motion sensor, applicable to a three-dimensional ultrasound device integrating a photoelectric sensor and a motion sensor as described above, comprising: Step S1: Establish a three-dimensional physical space coordinate system based on the first frame of two-dimensional ultrasound image; Step S2: The ultrasonic probe module 100 acquires a two-dimensional ultrasonic image sequence in real time, the photoelectric sensor module 200 collects the planar motion information of the ultrasonic probe, and the motion sensor module 300 collects the spatial attitude information of the ultrasonic probe. Step S3: The image segmentation model in the image processing module 400 is used to perform frame-by-frame multi-scale feature extraction and segmentation on the two-dimensional ultrasound image sequence to obtain the segmentation result of each frame. Step S4: Based on the planar motion information and the spatial pose information, calculate the relative displacement and relative rotation angle of the current frame image relative to the previous frame image; Step S5: Based on the relative displacement and the relative rotation angle, map the current frame image and its segmentation result to the corresponding position in the three-dimensional physical space coordinate system; Step S6: Determine if the scan has ended. If not, return to step S2 to continue data acquisition and processing. If it has ended, proceed to step S7. Step S7: Perform three-dimensional volume data fusion and reconstruction based on the relative displacement and relative rotation angle of each frame image to generate and output a three-dimensional reconstructed image.

[0068] Specifically, in this embodiment of the invention, the planar motion data collected by the photoelectric sensing module 200 and the spatial attitude data collected by the motion sensing module 300 are fused together, and the two-dimensional ultrasound sequence is processed frame by frame by frame in the image processing module 400. The relative displacement and rotation angle between adjacent frames are accurately calculated, and each frame image and its segmentation result are mapped to a unified three-dimensional physical space coordinate system according to the transformation relationship. Finally, high-precision three-dimensional structure reconstruction is achieved through multi-frame data fusion.

[0069] More specifically, since the operation method may cause fluctuations or rotations in the probe's movement path during manual ultrasound scanning, resulting in the intersection of scanning planes, in step S5, pixel value fusion processing is performed on the voxel intersection area. Specifically, when multiple image frames are detected to have data overlap at the same voxel position in the physical space coordinate system, the pixel gray value or segmentation probability value at that position is averaged to smooth the stitching marks and reduce motion artifacts.

[0070] In addition, after the scanning is completed, due to the sparsity of two-dimensional data in 3D space, some spaces will not generate data results. Therefore, in step S7, the missing areas are filled by interpolation. Specifically, a trilinear interpolation algorithm can be used to estimate the voxel values ​​of the missing areas based on the values ​​of the surrounding known voxel points, so as to generate continuous and complete three-dimensional volume data, improve the visual effect and usability of the reconstructed image.

[0071] The following embodiment details the specific implementation process of this three-dimensional ultrasound reconstruction method: Example 1: First, a three-dimensional physical space coordinate system is established using the first frame of the two-dimensional ultrasound image sequence as the spatial reference. Specifically, the imaging plane containing the first frame image is defined as the xoy plane, and the direction perpendicular to this plane is defined as the z-axis. The ultrasound image data of the first frame is saved and input into the pre-trained segmentation model in the image processing module 400 to obtain the corresponding segmentation result.

[0072] The scanning process continues. When the scan reaches frame t+1, the spatial position of frame t is used as a reference: the photoelectric sensing module 200 calculates and records the translation direction and displacement of the probe from frame t to frame t+1 in real time, and the frame t+1 is initially positioned in the physical coordinate system based on this displacement; at the same time, the motion sensing module 300 calculates and records the rotation angle of the frame relative to frame t, and performs corresponding rotation transformation on the image data of frame t+1, thereby accurately restoring the pose of the frame in the real coordinate system.

[0073] If voxel overlap is detected in the physical space between two consecutive frames of image data during the mapping process (i.e., the same spatial location is scanned multiple times), the pixel values ​​repeatedly acquired at that location are averaged and fused to eliminate the repeated acquisition phenomenon caused by fluctuations in the scanning method and improve data consistency. It should be noted that if the same voxel is scanned multiple times, this embodiment preferably uses the value acquired the first time as the final result to avoid registration errors that may be introduced by subsequent frames. Simultaneously, the (t+1)th frame image is also input into the segmentation model to obtain its target segmentation result.

[0074] Subsequently, the system continuously scans and repeats the above pose calculation and mapping process, recording the relative displacement and rotation angle of each frame relative to the previous frame. Based on this incremental pose information, the global pose transformation matrix of each frame relative to the first frame is calculated recursively, thereby determining the precise distribution of each frame image and its segmentation result in the physical coordinate system. The segmentation result preferentially uses the highest resolution segmentation map output by the model decoder.

[0075] Furthermore, after obtaining the offset and rotation angle of frame t relative to frame t+1, the cumulative offset and rotation angle relative to frame 1 can be calculated using the chain transformation formula. Based on this, the pixel and segmentation information of all frames can be fused to generate preliminary 3D volume data.

[0076] Finally, considering the sparse distribution of two-dimensional image sequences in three-dimensional space, interpolation processing (such as trilinear interpolation) is performed on the missing regions in the reconstructed data after scanning to fill data gaps, improve the visual continuity and geometric integrity of the reconstruction results, and finally output a high-fidelity three-dimensional reconstructed image.

[0077] The above description is merely a preferred embodiment of the present invention and does not limit the implementation and protection scope of the present invention. Those skilled in the art should realize that any equivalent substitutions and obvious changes made based on the description and illustrations of the present invention should be included within the protection scope of the present invention.

Claims

1. A three-dimensional ultrasonic device integrating a photoelectric sensor and a motion sensor, characterized in that, include: An ultrasonic probe module is used to transmit and receive multi-frequency ultrasonic signals to obtain a sequence of two-dimensional ultrasonic images. The photoelectric sensing module is connected to the ultrasonic probe module and is used to detect the translational motion of the ultrasonic probe in real time and generate planar motion information. A motion sensing module, connected to the ultrasonic probe module, is used to detect the rotational motion of the ultrasonic probe in real time and generate spatial attitude information. The image processing module, connected to the ultrasound probe module, is used to extract features and segment the two-dimensional ultrasound image sequence using a pre-trained image segmentation model to obtain the segmentation result; The three-dimensional reconstruction module, connected to the photoelectric sensing module, the motion sensing module, and the image processing module, is used to perform spatial fusion and volume data reconstruction on the two-dimensional ultrasound image sequence and the segmentation results based on the planar motion information and the spatial posture information, so as to generate a three-dimensional reconstructed image.

2. The three-dimensional ultrasonic device integrating a photoelectric sensor and a motion sensor according to claim 1, characterized in that, The photoelectric sensing module includes: The light source unit is used to emit illumination light onto the scanning surface; An image sensor is used to acquire a sequence of optical images formed after reflection from the scanned surface; An array signal processor, connected to the image sensor, is used to process the sequence of optical images and determine the direction and amount of movement of the ultrasound probe in the plane by calculating the correlation between consecutive images, so as to generate the planar motion information. The communication interface unit is connected to the array signal processor and is used to transmit the planar motion information to the three-dimensional reconstruction module.

3. A three-dimensional ultrasonic device integrating a photoelectric sensor and a motion sensor according to claim 1, characterized in that, The motion sensing module includes: The data acquisition unit is used to acquire the acceleration and angular velocity measurements of the ultrasonic probe in the X, Y, and Z axes in real time. The motion processing unit, connected to the data acquisition unit, is used to calculate the static tilt angle of the ultrasonic probe based on the acceleration measurement value, and to perform time integration on the angular velocity measurement value to calculate the dynamic angle change of the ultrasonic probe, and to fuse the static tilt angle and the dynamic angle change to generate the spatial attitude information. The output interface unit is connected to the motion processing unit and is used to transmit the spatial attitude information to the three-dimensional reconstruction module.

4. A three-dimensional ultrasonic device integrating a photoelectric sensor and a motion sensor according to claim 1, characterized in that, In the image processing module, the image segmentation model is a multi-scale feature aggregation neural network based on an encoder-decoder structure.

5. A three-dimensional ultrasonic device integrating a photoelectric sensor and a motion sensor according to claim 4, characterized in that, The image processing module includes: The encoding unit is used to perform continuous convolution and downsampling operations on the input image through the encoder to extract multi-layer feature maps with progressively decreasing resolution; The multi-layer feature maps are L1 feature map, L2 feature map, L3 feature map, and L4 feature map, where L1 is the shallowest feature map with the highest resolution, and L4 is the deepest feature map with the lowest resolution. The decoding and segmentation unit, connected to the encoding unit, is used to perform multi-stage upsampling, cross-layer feature concatenation and convolution processing on the multi-layer feature map through the decoder, and output a segmentation map of corresponding resolution at each stage based on a deep supervision mechanism, wherein the segmentation map output in the final stage is used as the segmentation result.

6. A three-dimensional ultrasonic device integrating a photoelectric sensor and a motion sensor according to claim 5, characterized in that, The decoding and segmentation unit performs the following operations: In the first segmentation stage, the L4 feature map is subjected to 1×1 convolution processing to generate a first segmentation map with a resolution of 1 / 8 of the input image. In the second segmentation stage, the L4 feature map is upsampled and then cascaded and fused with the L3 feature map to form the F3 feature. The F3 feature is then subjected to 1×1 convolution to generate a second segmentation map with a resolution of 1 / 4 of the input image. In the third segmentation stage, the L3 feature map is upsampled and then cascaded and fused with the L2 feature map to form the F2 feature. The F2 feature is then subjected to 1×1 convolution to generate a third segmentation map with a resolution of 1 / 2 of the input image. In the fourth segmentation stage, the L2 feature map is upsampled and then cascaded and fused with the L1 feature map to form the F1 feature. The F1 feature is then subjected to 1×1 convolution to generate a fourth segmentation map with the same resolution as the input image. The fourth segmentation map is used as the segmentation result.

7. A three-dimensional ultrasonic device integrating a photoelectric sensor and a motion sensor according to claim 1, characterized in that, The image segmentation model is trained through a training module, which includes: The image preprocessing unit is used to resample and linearly normalize each image in the two-dimensional ultrasound image sequence, and the specific formula is expressed as follows: ； Where x represents the input image, This represents the image after normalization. This represents the maximum grayscale value of the image. Represents the minimum grayscale value of the image; The deep supervised training unit, connected to the image preprocessing unit, is used to input the preprocessed image into the image segmentation model and calculate the weighted binary cross-entropy loss for each segmentation stage, as expressed by the following formula: ； in, This represents the weight corresponding to each pixel. Represents the true value of a pixel. Represents the pixel prediction value; The loss fusion unit, connected to the deep supervised training unit, is used to sum the weighted binary cross-entropy losses from each segmentation stage according to preset weight coefficients to obtain the comprehensive loss. The specific formula is as follows: ； in, (k=1,2,3,4) represents the weighted binary cross-entropy loss for the k-th stage segmentation; a, b, c, and d represent the segmentation loss weights for the four stages, where 0 <a<b<c<d; The model optimization unit, connected to the loss fusion unit, is used to iteratively update the parameters of the image segmentation model based on the comprehensive loss.

8. A three-dimensional ultrasonic device integrating a photoelectric sensor and a motion sensor according to claim 7, characterized in that, In the deep supervised training unit, a weighted map corresponding to the resolution of each segmentation stage is generated based on the segmentation ground truth mask, including: For the first segmentation stage and the second segmentation stage, by performing distance transformation on the segmentation ground truth mask, the distance of each pixel in the segmentation ground truth mask to the nearest boundary is calculated, and the obtained distance map is normalized by the maximum and minimum values ​​to generate a first weight map for enhancing the overall localization of the target area. For the third and fourth segmentation stages, the first weight map of the first segmentation stage or the second segmentation stage is reverse-weighted to generate a second weight map for enhancing edge detail learning.

9. A three-dimensional ultrasonic device integrating a photoelectric sensor and a motion sensor according to claim 1, characterized in that, The three-dimensional reconstruction module includes: The coordinate system establishment unit is used to establish a three-dimensional physical space coordinate system with the first frame image of the two-dimensional ultrasound image sequence as a spatial reference, and to place the first frame image and its segmentation result in the three-dimensional physical space coordinate system. The inter-frame pose calculation unit, connected to the coordinate system establishment unit, is used to calculate the relative displacement and relative rotation angle of the current frame image relative to the previous frame image based on the planar motion information and the spatial pose information. A spatial mapping unit, connected to the inter-frame pose calculation unit, is used to map the current frame image and its segmentation result to the corresponding position in the physical space coordinate system according to the relative displacement and the relative rotation angle. The voxel fusion unit, connected to the spatial mapping unit, is used to detect whether there is a voxel intersection region between the mapped current frame image and the existing image frame in the physical space coordinate system. If there is, the pixel values ​​of the voxel intersection region are fused and the pixel values ​​of the first mapped frame are used as the final result. The global trajectory calculation unit, connected to the voxel fusion unit, is used to recursively calculate the cumulative displacement and cumulative rotation angle of each frame image relative to the first frame image based on the relative displacement and relative rotation angle of each frame image, so as to obtain the global pose of all image frames. A 3D volume data generation unit, connected to the global trajectory calculation unit, is used to generate 3D volume data based on the global pose. An interpolation optimization unit, connected to the three-dimensional volume data generation unit, is used to interpolate and fill missing regions in the three-dimensional volume data to generate the three-dimensional reconstructed image.

10. A method for reconstructing a three-dimensional ultrasonic device integrating a photoelectric sensor and a motion sensor, applied to a three-dimensional ultrasonic device integrating a photoelectric sensor and a motion sensor as described in any one of claims 1-9, characterized in that, include: Step S1: Establish a three-dimensional physical space coordinate system based on the first frame of two-dimensional ultrasound image; Step S2: The two-dimensional ultrasound image sequence is acquired in real time through the ultrasound probe module, the planar motion information of the ultrasound probe is collected through the photoelectric sensing module, and the spatial attitude information of the ultrasound probe is collected through the motion sensing module. Step S3: Use the image segmentation model in the image processing module to perform frame-by-frame multi-scale feature extraction and segmentation on the two-dimensional ultrasound image sequence to obtain the segmentation result of each frame. Step S4: Based on the planar motion information and the spatial pose information, calculate the relative displacement and relative rotation angle of the current frame image relative to the previous frame image; Step S5: Based on the relative displacement and the relative rotation angle, map the current frame image and its segmentation result to the corresponding position in the three-dimensional physical space coordinate system; Step S6: Determine if the scan has ended. If not, return to step S2 to continue data acquisition and processing. If it has ended, proceed to step S7. Step S7: Perform three-dimensional volume data fusion and reconstruction based on the relative displacement and relative rotation angle of each frame image to generate and output a three-dimensional reconstructed image.

Images