Image processing method for handheld imaging device

Abstract

This invention relates to an image processing method for a handheld imaging device. The method acquires surface image data of the object under test through an optical scanning module and establishes a surface model and a world coordinate system. At the same time, it acquires the spatial position and attitude information of the device. Then, it uses a radar module to acquire internal structure measurement data and transmits it to the host. The host maps the internal structure measurement data to the corresponding position in the world coordinate system based on the spatial position and attitude information, so that it corresponds to the spatial position of the surface model. Alignment and fusion are then performed to generate a fused imaging result and output it for display.

Description

Technical Field

[0001] This invention relates to a method, particularly an image processing method for a handheld imaging device. Background Technology

[0002] With the development of non-destructive testing technology, various imaging methods have been widely used in fields such as building inspection, industrial inspection and medical imaging. Among them, optical imaging technology, due to its advantages of high resolution and fast scanning, is often used to obtain information on the appearance and surface morphology of objects, and can further build three-dimensional surface models.

[0003] However, optical imaging technology can only measure visible surfaces and cannot penetrate the surface of an object to obtain information about its internal structure, which limits its application range.

[0004] On the other hand, penetrating detection technologies such as radar, ultrasound, and X-rays can be used to obtain information about the internal structure of objects. Radar technology has gradually gained attention due to its advantages of non-contact measurement and relative safety. However, radar imaging resolution is usually low, and its echo signals lack clear spatial positioning information, making it difficult for the obtained images to correspond to the spatial position of the actual objects, thus limiting its application in fine imaging.

[0005] In addition, to improve radar imaging quality, existing technologies mostly rely on fixed scanning equipment, such as scanning systems mounted on mechanical rails or robotic arms, which measure through pre-planned scanning paths. Although this type of system can achieve better imaging results under stable conditions, its equipment is bulky, expensive, and lacks mobility, making it unsuitable for rapid on-site detection or handheld applications.

[0006] Existing technologies also attempt to combine data from different sensors in order to improve imaging performance. For example, optical data and radar data are integrated to simultaneously acquire surface and internal information.

[0007] However, this technology still largely relies on fixed equipment or a stable scanning environment, and during the data fusion process, it still faces the problem of data misalignment between different sensors, especially when the device moves or vibrates, making it even more difficult to ensure the accuracy and consistency of the data.

[0008] In handheld applications, the device will continuously change position and attitude during operation. If there is no effective positioning and correction mechanism, the measurement data obtained by the radar will be difficult to accurately map to the corresponding position of the object under test, thus affecting the imaging quality. In addition, radar measurements are usually point data or low-resolution data. Without appropriate data accumulation and fusion methods, it is difficult to reconstruct the complete three-dimensional internal structure.

[0009] On the other hand, while existing optical scanning technology can establish the three-dimensional structure of a scene and obtain the spatial position information of the device through positioning algorithms, it can only provide surface information and cannot extend to the analysis of the internal structure. Therefore, how to effectively combine optical positioning capabilities with transmissive sensing technology to obtain high-resolution surface information and internal structure information at the same time remains an important issue.

[0010] Furthermore, the process of multi-sensor fusion also involves the conversion, alignment, and integration of different data types. For example, how to map the volume or reflection information obtained by radar to the surface model established by optics, and further present it in a visual way so that users can intuitively understand the relationship between the internal and external structures, is also a difficulty that existing technologies still need to overcome.

[0011] Furthermore, conventional optical imaging technology can only obtain surface information and cannot penetrate the interior, while penetrating technologies such as radar lack precise spatial positioning capabilities and rely heavily on fixed equipment for measurement.

[0012] Therefore, how to simultaneously acquire surface information and internal structural information of the object under test under handheld operation conditions, and accurately align and fuse the two to form a visual imaging result, is a problem that those in the technical field must solve. Summary of the Invention

[0013] One objective of this invention is to provide a handheld imaging device and its image processing method. This device establishes a surface model and spatial positioning information of the object under test using an optical scanning module, and combines this with a radar module to obtain internal structural information of the object. The radar measurement data is then mapped to the corresponding spatial location and aligned and fused with the surface model to produce a visualized imaging result that incorporates both surface and internal information. Therefore, this invention enables highly mobile inspection under handheld operation conditions and effectively improves the completeness and accuracy of the imaging.

[0014] To achieve the above objectives, the present invention provides an image processing method for a handheld imaging device. This method involves detecting the surface structure and internal structure of an object under test using a handheld imaging device. The steps include: executing a scanning procedure using an optical scanning module of the handheld imaging device to acquire surface image data of the object under test; establishing a surface model and a world coordinate system of the object under test from the surface image data; simultaneously acquiring the spatial position and attitude information of the handheld imaging device in the world coordinate system during the scanning procedure; acquiring measurement data of the internal structure of the object under test using a radar module of the handheld imaging device; and using a front-end data processing unit of the handheld imaging device... After the data integration and transmission unit integrates the surface image data and the internal structure measurement data, it transmits them to the back-end data integration and transmission unit of a host through a transmission module of the handheld imaging device. The host receives the surface image data and the internal structure measurement data through a computing unit, and maps the internal structure measurement data to a corresponding position in the world coordinate system based on the spatial position and attitude information, so that the internal structure measurement data corresponds to a spatial position of the surface model. The computing unit aligns and fuses the mapped internal structure measurement data with the surface model to produce a fused imaging result, which is then output through a display unit.

[0015] The present invention provides an embodiment in which the surface model is a three-dimensional surface model.

[0016] The present invention provides an embodiment in which the spatial position and attitude information is six-degree-of-freedom attitude information.

[0017] The present invention provides an embodiment in which the spatial position and attitude information is continuously updated during handheld scanning.

[0018] The present invention provides an embodiment in which the internal structure measurement data is spatially aligned based on the spatial position and attitude information.

[0019] The present invention provides an embodiment in which the radar module comprises a plurality of radar elements.

[0020] The present invention provides an embodiment in which the plurality of radar elements are arranged in a regular or irregular manner.

[0021] The present invention provides an embodiment in which the front-end data integration and transmission unit is a field-programmable logic gate array.

[0022] The present invention provides an embodiment in which the field-programmable logic gate array is used to perform data integration, data format conversion or high-speed data transmission.

[0023] The present invention provides an embodiment in which the back-end data integration and transmission unit is a field-programmable logic gate array.

[0024] The present invention provides an embodiment in which the field-programmable logic gate array is used to perform data integration, data format conversion or high-speed data transmission.

[0025] The present invention provides an embodiment in which the step of receiving surface image data and internal structure measurement data through a computing unit of the host, and mapping the internal structure measurement data to a corresponding position in the world coordinate system according to the spatial position and attitude information, so that the internal structure measurement data corresponds to a spatial position of the surface model, includes the step of converting the internal structure measurement data into spatial volume data in the world coordinate system.

[0026] The present invention provides an embodiment in which the spatial volume data is formed into a three-dimensional voxel model.

[0027] The present invention provides an embodiment in which, before the step of the computing unit aligning and fusing the mapped internal structure measurement data with the surface model to generate a fused imaging result, the method includes the steps of performing a noise filtering procedure, a resolution enhancement procedure, or a signal compensation processing procedure on the internal structure measurement data.

[0028] The present invention provides an embodiment in which the step of the computing unit aligning and fusing the mapped internal structure measurement data with the surface model to generate a fused imaging result includes the step of: sampling the internal structure measurement data along one direction of the surface model to obtain internal information corresponding to the surface model.

[0029] The present invention provides an embodiment in which the fused imaging result is superimposed on the surface model in the form of a heat map.

[0030] The present invention provides an embodiment in which the fused imaging result includes a cross-sectional image or a monolithic rendered image. Attached Figure Description

[0031] Figure 1A This is a schematic flowchart of one embodiment of the present invention; Figure 1B This is a block diagram illustrating one embodiment of the present invention; Figure 2 This is a schematic flowchart of one embodiment of the present invention; Figure 3 This is a schematic flowchart of one embodiment of the present invention; and Figure 4 This is a flowchart illustrating one embodiment of the present invention.

[0032] Figure label: 10: Handheld imaging device; 12: Optical scanning module; 14: Radar module; 16: Front-end data integration and transmission unit; 20: Host computer; 22: Backend data integration and transmission unit; 24: Arithmetic unit; 26: Power management components; 28: Power supply components; 30: Display unit; 40: External power supply; Steps: S10, S20, S30, S40, S40-1, S50, S50-1, S50-2, S60. Detailed Implementation

[0033] To enable your review committee to have a better understanding of the features and effects of this invention, the following preferred embodiments and detailed descriptions are provided: In conventional imaging technologies, optical sensing methods have the advantages of high resolution and fast scanning, but they can only obtain the appearance and surface information of the object under test and cannot penetrate the object to obtain its internal structure. While penetrating sensing technologies such as radar can obtain internal information, they have the problems of low resolution and lack of accurate spatial positioning capabilities, making it difficult to correspond the measurement results to the actual object position. However, existing penetrating detection devices mostly rely on fixed scanning platforms or mechanical tracks for measurement, which are not only bulky and expensive, but also lack mobility and cannot meet the needs of handheld operation.

[0034] This invention establishes a surface model and spatial positioning information of the object under test through an optical scanning module, and obtains the position and attitude of the device in real time during handheld operation. It then combines the internal structural information of the object under test with the radar module, and maps the radar measurement data to the corresponding world coordinate position based on the position and attitude information. This data is then aligned and fused with the surface model to form a visualized imaging result that combines surface and internal information.

[0035] The invention will be described in detail below by way of the drawings illustrating various embodiments thereof. However, the concept of the invention may be embodied in many different forms and should not be construed as being limited to the exemplary embodiments set forth herein.

[0036] First, please refer to Figure 1A, which is a flowchart of one embodiment of the present invention, and also refer to Figure 1B, which is a block diagram of one embodiment of the present invention. As shown in the figures, this embodiment is based on the integration of multiple sensors, and obtains surface image data of the object under test through an optical scanning module and obtains internal structure measurement data through a radar module.

[0037] This invention further maps internal structure measurement data to a world coordinate system by establishing spatial position and attitude information, so that the internal structure information corresponds to the spatial position of the surface model and is aligned and fused to produce a visual image that combines surface information and internal structure information.

[0038] Referring again to Figures 1A and 1B, as shown, the image processing method of the handheld imaging device in this embodiment includes the following steps: Step S10: The scanning procedure is performed through the optical scanning module of the handheld imaging device to obtain the surface image data of the object under test, and the surface model and world coordinate system of the object under test are established from the surface image data. At the same time, the spatial position and attitude information of the handheld imaging device in the world coordinate system are obtained during the scanning procedure. Step S20: Obtain measurement data of the internal structure of the object under test using the radar module of the handheld imaging device; Step S30: After integrating the surface image data and internal structure measurement data by the front-end data integration and transmission unit of the handheld imaging device, the data is transmitted to the back-end data integration and transmission unit of the host. Step S40: Receive surface image data and internal structure measurement data through the computing unit of the host computer, and map the internal structure measurement data to the corresponding position in the world coordinate system according to the spatial position and attitude information, so that the internal structure measurement data can correspond to the spatial position of the surface model. Step S50: The computing unit aligns and fuses the mapped internal structure measurement data with the surface model to generate a fused imaging result; and Step S60: Output the fused imaging result through the display unit.

[0039] Please refer to Figure 1B. As shown in the figure, in one embodiment, the handheld imaging device 10 is electrically connected to a host 20. The handheld imaging device 10 includes an optical scanning module 12, a radar module 14, and a front-end data integration and transmission unit 16.

[0040] The optical scanning module 12 is used to acquire surface image data of one of the objects under test, the radar module 14 is used to acquire internal structural measurement data of one of the objects under test, and the front-end data integration and transmission unit 16 is used to integrate the surface image data and the internal structural measurement data and transmit them.

[0041] Please refer to Figure 1B. The host 20 includes a back-end data integration and transmission unit 22, a computing unit 24, a power management element 26, a power supply element 28, and a display unit 30.

[0042] The back-end data integration and transmission unit 22 is used to receive and process data from the front-end data integration and transmission unit 16, and the computing unit 24 is used to perform data processing, spatial alignment, data fusion and imaging operations to generate fused imaging results.

[0043] In this embodiment, the host 20 further includes an external power supply 40, which is used to provide input power. The power management element 26 is used to receive and manage the power provided by the external power supply 40, and to perform power distribution and control according to the needs of each element. The power supply element 28 is used to provide stable power to each computing and transmission module to maintain the normal operation of the system.

[0044] The display unit 30 is used to output the fused imaging results, allowing the user to observe the surface information and internal structural information of one of the objects under test.

[0045] Next, please refer to Figures 1A and 1B again. As described in step S10, the image processing method of the handheld imaging device in this embodiment detects a surface structure and an internal structure of the object under test through the handheld imaging device 10. A scanning procedure is performed by the optical scanning module 12 of the handheld imaging device 10 to obtain surface image data of the object under test. A surface model of the object under test and a world coordinate system are established from the surface image data. At the same time, during the execution of the scanning procedure, the spatial position and attitude information of the handheld imaging device 10 in the world coordinate system are obtained. The spatial position and attitude information is six-degree-of-freedom attitude information.

[0046] The six degrees of freedom attitude information is used to describe the position and orientation of the handheld imaging device in three-dimensional space. The six degrees of freedom attitude information includes three translational degrees of freedom and three rotational degrees of freedom. The three translational degrees of freedom correspond to the position coordinates in the X-axis, Y-axis and Z-axis directions, respectively, while the three rotational degrees of freedom correspond to the rotation angles (such as pitch angle, yaw angle and roll angle) around the X-axis, Y-axis and Z-axis, respectively, to describe the attitude changes of the handheld imaging device.

[0047] By using six degrees of freedom attitude information, the pose of the handheld imaging device in the world coordinate system can be established in real time, and the internal structure measurement data obtained subsequently can be accurately mapped to the corresponding spatial position, thereby improving the accuracy of data alignment and fusion.

[0048] In this embodiment, the optical scanning module 12 includes a lens and an optical sensor. The lens guides the reflected light from the object under test into the optical sensor to capture the surface image data of the object under test.

[0049] In this embodiment, the surface model is a three-dimensional surface model used to represent the external geometry of the object under test.

[0050] In this embodiment, the optical sensor may be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) image sensor to convert light signals into corresponding electrical signals. Furthermore, the optical scanning module 12 can continuously capture multiple image data during handheld scanning to establish a three-dimensional surface model of the object under test and the world coordinate system.

[0051] In this embodiment, the spatial position and attitude information is used to describe the spatial position and orientation of the handheld imaging device 10 relative to the world coordinate system, and can be continuously updated as the handheld imaging device 10 moves.

[0052] By obtaining this spatial position and attitude information, the subsequent internal structure measurement data can be accurately mapped to the spatial position in the world coordinate system, thereby improving the accuracy of fused imaging.

[0053] Please refer to Figure 1A again. In this embodiment, optical image data and radar measurement data are acquired simultaneously through a multi-sensor synchronization mechanism, and time synchronization and cross-sensor correction are performed.

[0054] In terms of attitude estimation, the pose information of the handheld imaging device can be obtained through simultaneous localization and mapping (SLAM) or visual inertial odometry (VIO) methods, which can be represented as: T_w_cam(t) Where T_w_cam(t) represents the transformation relationship of the handheld imaging device relative to the world coordinate system at time t.

[0055] In addition, the surface image data obtained by the optical scanning module can be used to build a three-dimensional surface model of the scene through the truncated symbolic distance function (TSDF), and further converted into a mesh representation.

[0056] Next, as described in step S20, the radar module 14 of the handheld imaging device 10 is used to obtain internal structural measurement data of the object under test. In this embodiment, the radar module 14 is used to emit an electromagnetic wave signal and receive an echo signal reflected from the inside of the object under test. Based on the time delay, intensity, or frequency change of the echo signal, the internal structural information of the object under test is calculated. Through the aforementioned method, internal structural measurement data including depth information can be obtained to reflect the material distribution or structural characteristics inside the object under test.

[0057] In this embodiment, the radar module 14 includes a plurality of radar units, which are arranged in a regular or irregular manner.

[0058] In this embodiment, as described in steps S30 and S40, the front-end data integration and transmission unit 16 of the handheld imaging device 10 integrates the surface image data and the internal structure measurement data, and then transmits them to the back-end data integration and transmission unit 22 of the host 20. The computing unit 24 of the host 20 receives the surface image data and the internal structure measurement data, and maps the internal structure measurement data to a corresponding position in the world coordinate system based on the spatial position and attitude information, so that the internal structure measurement data corresponds to a spatial position of the surface model, so that the internal structure information can be accurately aligned with the surface model, thereby improving the spatial consistency and accuracy of the fused imaging.

[0059] In this embodiment, the computing unit 24 is an industrial computer, personal computer or tablet computer, and the front-end data integration and transmission unit 16 and / or the back-end data integration and transmission unit 22 can be a field-programmable gate array (FPGA) to perform data integration, data format conversion or high-speed data transmission.

[0060] In this embodiment, please refer to Figure 2 This is a flowchart illustrating one embodiment of the present invention. As shown in the figure, it describes the process of converting internal structure measurement data to a world coordinate system. The process includes the following steps after the internal structure measurement data is mapped to its corresponding position in the world coordinate system: Step S40-1: Convert the internal structure measurement data to spatial volume data in the world coordinate system.

[0061] As described in step S40-1, after the internal structure measurement data is mapped to the corresponding position in the world coordinate system, it can be further converted into spatial volume data. The spatial volume data is composed of a plurality of data units corresponding to spatial positions, which are used to represent the distribution of the interior of the object under test in three-dimensional space. By converting the internal structure measurement data into spatial volume data, the internal information can be presented in a structured manner, which is conducive to subsequent alignment and fusion processing.

[0062] In this step, radar measurement data is mapped to the world coordinate system based on time-synchronized attitude information. The transformation relationship can be expressed as follows: T_w_em(t) = T_w_cam(t) · T_cam_em Where T_w_em(t) represents the transformation relationship between the radar coordinate system and the world coordinate system at time t, while T_cam_em is the relative pose transformation relationship between the optical scanning module and the radar module.

[0063] Through the above conversion, radar measurement data can be gradually accumulated into the world volume to form spatial volume data, which can be further converted into a three-dimensional voxel model to describe the structural distribution of the object under test in three-dimensional space.

[0064] Furthermore, at each point p of the surface model, samples can be taken from the spatial volume data along its normal direction to obtain the corresponding internal information.

[0065] In this embodiment, the sampling can be carried out according to different strategies, such as taking the maximum value, average value, or cross-sectional sampling, to reflect the internal structural features corresponding to the surface location.

[0066] The sampling process involves extracting measurement information corresponding to the location of the surface model from the spatial volume data to establish a spatial correspondence between the internal structure and the surface model.

[0067] This allows the internal structural information to accurately correspond to the spatial position of the surface model, thus completing the alignment and fusion of heterogeneous data.

[0068] By establishing the aforementioned spatial volume data and sampling mechanism along the surface model, the internal structure measurement data can accurately correspond to the spatial position of the surface model, which is the key technical means for heterogeneous sensing data fusion in this embodiment.

[0069] In addition, in this embodiment, the three-dimensional voxel model disclosed above, wherein each voxel corresponds to a volume element inside the object under test and stores corresponding measurement information, such as reflection intensity, material characteristics or other structural information, can more accurately describe the internal structure of the object under test and improve the accuracy and spatial consistency when aligning and fusing with the surface model.

[0070] Therefore, this embodiment combines the spatial position and attitude information obtained above to convert and map the internal structure measurement data to the corresponding position in the world coordinate system, thereby enabling the internal structure measurement data to correspond to the spatial position of the surface model, so that the internal structure information can be accurately aligned with the surface model, and the spatial consistency and accuracy of the fused imaging can be improved.

[0071] Next, in this embodiment, as described in steps S50 to S60, the computing unit 24 aligns and fuses the mapped internal structure measurement data with the surface model to generate a fused imaging result, and outputs the fused imaging result through a display unit 30. The fused imaging result is a visual image information that combines surface information and internal structure information. Furthermore, the display unit 30 can be an augmented reality (AR) glasses, virtual reality (VR) glasses, or a liquid crystal display screen, etc. This embodiment is not limited to these; any device that can be used to display the fused imaging result can be used as an implementation of the display unit 30.

[0072] In this embodiment, before performing the alignment and fusion in step S50, the internal structure measurement data can be preprocessed. Please refer to [reference needed]. Figure 3 As shown in the figure, noise filtering, resolution enhancement, or signal compensation processing can be performed on internal structure measurement data to improve data quality and the accuracy of subsequent fusion. The steps include: Step S50-1: Perform noise filtering, resolution enhancement, or signal compensation processing on the internal structure measurement data.

[0073] Among these, a noise filtering procedure, a resolution enhancement procedure, or a signal compensation processing procedure can be applied to the internal structure measurement data to improve the quality of the internal structure information.

[0074] Among them, the noise filtering procedure can be used to reduce measurement errors, the resolution enhancement procedure can improve spatial details, and the signal compensation procedure can correct signal attenuation or distortion caused by material differences.

[0075] Furthermore, this noise filtering procedure is used to reduce measurement errors. Therefore, in this embodiment, in order to reduce the influence of noise generated during handheld scanning, a noise suppression mechanism combining temporal coherence and incoherence can be adopted to improve measurement stability.

[0076] During the scanning process, the spatial position and posture of the handheld device may undergo slight changes in a short period of time, which may cause the phase of the measurement signal to be unstable or produce a phase shift similar to the Doppler effect. This makes it easy to cause image blurring or reduced resolution when performing long-term coherent accumulation.

[0077] In this embodiment, continuously acquired radar measurement data can be divided into multiple time segments (batch), and coherent processing can be performed within each time segment to preserve the phase information and resolution capability of the signal; while between different time segments, an incoherent accumulation method is used for integration to reduce the accumulation of phase errors caused by device shaking.

[0078] Furthermore, within each time segment, the measurement data can be first processed by noise filtering or signal enhancement before coherent integration. Between the outputs of each time segment, energy accumulation or statistical integration is performed to form the final internal structure measurement data. Through this hybrid processing method of time segmentation, noise and errors introduced by handheld operation can be effectively suppressed while maintaining image resolution.

[0079] Furthermore, in this embodiment, to further improve the spatial resolution and boundary clarity of the internal structure measurement data, resolution enhancement processing can be performed through multimodal data fusion. Specifically, high-resolution edge or structural information in surface image data (such as RGB images or near-infrared images) obtained by the optical scanning module can be used as guiding information for the internal structure measurement data.

[0080] In this embodiment, edge features or geometric structure information of the surface model can be extracted from the optical image first, and the edge features can be used as structural constraints to enhance the resolution of the internal structure measurement data, so that the internal structure information can maintain the same boundary characteristics as the surface model at the corresponding spatial position.

[0081] In particular, when performing resolution enhancement, edge-guided processing can be used to smooth or sharpen the internal structural measurement data in the boundary area, while avoiding blurring caused by crossing the structural boundary, thereby improving the image's recognizability and detail.

[0082] Furthermore, under multi-frame or multi-modal data conditions, motion compensation and super-resolution technologies can be combined to align and reconstruct data from different time points or different sensing sources, thereby improving the spatial resolution and detail of internal structural measurement data.

[0083] By employing the aforementioned resolution enhancement mechanism, the fused imaging results can maintain the consistency of internal structural information while possessing higher boundary sharpness and image quality, and can effectively improve the problems of insufficient resolution or blurred boundaries in traditional internal imaging techniques.

[0084] Furthermore, in this embodiment, it is explained that the medium compensation procedure is to improve the accuracy of internal structure measurement data. Since the propagation speed of electromagnetic waves may change due to the difference in dielectric constant when propagating in different materials, resulting in errors in the measurement distance, it is necessary to correct it through a compensation mechanism.

[0085] Specifically, the dielectric effect can be regarded as a correction parameter to be estimated, and the equivalent propagation characteristics inside the test object, such as equivalent propagation velocity or equivalent dielectric constant, can be estimated by analyzing measurement data from multiple frequencies or multiple perspectives.

[0086] In this embodiment, measurement results at different frequencies or observation angles can be compared, and estimation can be performed using consistency conditions to reduce errors caused by phase accumulation or distance calculation, thereby correcting the distance information of the internal structure measurement data.

[0087] In addition, when performing distance estimation, phase unwrapping or multi-frequency distance estimation methods can be combined to improve the continuity and accuracy of measurement results and reduce the impact of distance ambiguity.

[0088] By using the aforementioned medium compensation mechanism, the internal structure measurement data can be made closer to the actual structure distribution, thereby improving the accuracy and reliability of subsequent spatial alignment and fusion, and improving the distance deviation problem caused by medium differences in traditional penetration measurement.

[0089] Next, in step S50, please refer to Figure 4 The figure illustrates a flowchart of one embodiment of the present invention, showing the sampling process corresponding to the internal structure measurement data and the surface model. By sampling along the direction of the surface model, internal information corresponding to the surface model can be obtained. The steps include: Step S50-2: Sample the internal structure measurement data along the direction of the surface model to obtain the internal information corresponding to the surface model.

[0090] Furthermore, in this embodiment, the fused imaging result can be a heat map superimposed on the surface model, or a cross-sectional image or a single rendered image, to present the internal structural distribution of the object under test.

[0091] The rendering process is further explained below. After data fusion is completed, the internal structure information and surface model are integrated through rendering. The display result can be represented as follows: R_t = R(M, V, P_t) Where R_t is the output image, R is the rendering function, M is the surface model, V is the volume data, and P_t is the corresponding pose information.

[0092] Through the above processing, internal structural information can be superimposed on the surface model in the form of a heat map, or further generated as a cross-sectional image or a volume rendering image, to present the internal structural distribution of the object under test.

[0093] Compared to conventional techniques that can only obtain surface information or can only perform internal scanning without spatial alignment, this embodiment enables the surface model and internal structural data to correspond precisely through the establishment of spatial position and attitude information, volume data modeling, and heterogeneous data fusion mechanism, which significantly improves the accuracy, resolution, and practicality of imaging.

[0094] In this embodiment, the volume rendering image refers to the volume rendering process performed on the three-dimensional voxel model formed by converting the internal structure measurement data. By performing optical mapping and weighted calculation on the measurement information of each voxel (such as reflection intensity or material characteristics), a three-dimensional visualization image that can present depth information is generated, so that the internal structure of the object under test can be presented in a three-dimensional form, thereby improving the intuitiveness and accuracy of image interpretation.

[0095] The advantage of this embodiment is that it can simultaneously acquire the external geometry and internal structure distribution of the object under test during handheld scanning, and improve the spatial consistency and resolution of the imaging through spatial alignment and data fusion technology. Moreover, it can achieve the technical effect of highly mobile and non-destructive detection without relying on a fixed scanning platform.

[0096] Next, in order to make the features of this embodiment clearer, a practical application example is given here to illustrate the following implementation example: In a practical application, the handheld imaging device 10 of the present invention can be applied to the non-destructive inspection of building walls.

[0097] The user holds the handheld imaging device 10 to scan a wall surface to be tested. During the scanning process, the optical scanning module 12 continuously acquires the surface image data of the wall surface, and establishes the three-dimensional surface model and world coordinate system of the wall surface through visual positioning and simultaneous localization mapping (SLAM) technology. At the same time, the user obtains the spatial position and attitude information of the handheld imaging device 10 in the world coordinate system.

[0098] In addition, the radar module 14 simultaneously transmits electromagnetic wave signals and receives echo signals during the scanning process to obtain structural measurement data of the interior of the wall, such as information on pipeline location, steel reinforcement distribution, or void defects.

[0099] In this embodiment, the radar module 14 may include a plurality of radar units, and the internal imaging resolution is improved through array arrangement and beamforming or synthetic aperture radar (SAR) processing.

[0100] Next, the front-end data integration and transmission unit 16 integrates the surface image data and the internal structure measurement data and transmits them to the host 20. The computing unit 24 then maps the internal structure measurement data to the corresponding position in the world coordinate system based on the spatial position and attitude information, thereby forming the spatial volume data corresponding to the three-dimensional surface model, and can further form the three-dimensional voxel model.

[0101] During the data fusion stage, the computing unit 24 can sample the volume data of the space along the normal direction of the surface model to obtain the internal information corresponding to the wall, and superimpose the internal information on the surface model in the form of a heat map, so that users can intuitively identify abnormal areas inside the wall, such as pipeline locations or structural defects.

[0102] Furthermore, the fused imaging results can also be presented as cross-sectional images or volumetric rendering images, allowing the internal structure to be displayed in three dimensions, thus improving the accuracy of interpretation.

[0103] In this way, the present invention can simultaneously obtain surface information and internal structural information of the object under test, and achieve real-time, non-destructive imaging with high spatial resolution through handheld scanning.

[0104] In another embodiment, the handheld imaging device 10 of the present invention can be applied to industrial equipment inspection or security inspection. For example, in industrial maintenance scenarios, it can scan the outer shell of mechanical equipment to simultaneously obtain the appearance structure and the condition of internal parts, and determine whether there is corrosion, loosening or abnormal wear inside by fusing the imaging results.

[0105] In another implementation example, in security or customs inspection scenarios, users can scan packages or luggage, obtain appearance information through the optical scanning module 12, and obtain internal structure information through the radar module 14. After alignment and fusion, the location and distribution of suspected abnormal objects can be displayed on the display unit 30, improving inspection efficiency and accuracy.

[0106] The embodiments described above are image processing methods for handheld imaging devices. The method acquires surface image data of the object under test through an optical scanning module and establishes a surface model and a world coordinate system. It also acquires internal structure measurement data through a radar module. Based on spatial position and attitude information, the internal structure measurement data is mapped to the world coordinate system so that the internal structure information corresponds to the spatial position of the surface model. Alignment and fusion processing are then performed to produce a fused imaging result that combines surface information and internal structure information.

[0107] By employing the aforementioned technical means, this invention can simultaneously acquire the external geometric shape and internal structural distribution of the object under test during handheld scanning. Through spatial alignment, volume modeling, and fusion display processing, it improves the spatial consistency, resolution, and visualization effect of the imaging, thereby achieving the technical effects of non-destructive testing, real-time imaging, and highly mobile applications. Compared with conventional technologies that can only acquire surface information or lack spatial alignment capabilities, this invention can effectively integrate surface and internal information and perform precise correspondence, resulting in a significant technological improvement.

[0108] However, the above description is only a preferred embodiment of the present invention and is not intended to limit the scope of the present invention. All equivalent changes and modifications made to the shape, structure, features and spirit described in the claims of the present invention should be included in the scope of the claims of the present invention.

Claims

1. An image processing method for a handheld imaging device, characterized in that, the method detects the surface structure and internal structure of an object under test through a handheld imaging device, wherein, The steps include: A scanning procedure is performed through one of the optical scanning modules of the handheld imaging device to obtain surface image data of one of the objects under test, and a surface model of one of the objects under test and a world coordinate system are established from the surface image data. At the same time, during the execution of the scanning procedure, the spatial position and attitude information of the handheld imaging device in the world coordinate system are obtained. The internal structure measurement data of the object under test is obtained using the radar module of one of the handheld imaging devices. After the surface image data and the internal structure measurement data are integrated by the front-end data integration and transmission unit of one of the handheld imaging devices, they are transmitted to the back-end data integration and transmission unit of one of the main units. The host computer receives surface image data and internal structure measurement data through one of its computing units, and maps the internal structure measurement data to a corresponding position in the world coordinate system based on the spatial position and attitude information, so that the internal structure measurement data corresponds to a spatial position of the surface model. The processing unit aligns and fuses the mapped internal structure measurement data with the surface model to produce a fused imaging result; and The fused imaging result is output through a display unit, wherein the fused imaging result is a visualized image information containing surface information and internal structure information.

2. The image processing method of the handheld imaging device as described in claim 1, characterized in that, The surface model is a three-dimensional surface model.

3. The image processing method of the handheld imaging device as described in claim 1, characterized in that, The spatial position and attitude information is six-degree-of-freedom attitude information.

4. The image processing method of the handheld imaging device as described in claim 1, characterized in that, The spatial location and attitude information is continuously updated during the handheld scanning process.

5. The image processing method of the handheld imaging device as described in claim 1, characterized in that, The internal structure measurement data is spatially aligned based on the spatial position and attitude information.

6. The image processing method of the handheld imaging device as described in claim 1, characterized in that, The radar module contains a plurality of radar elements, which are arranged in a regular or irregular manner.

7. The image processing method of the handheld imaging device as described in claim 1, characterized in that, The front-end data integration and transmission unit is a field-programmable gate array, which is used to perform data integration, data format conversion or high-speed data transmission.

8. The image processing method of the handheld imaging device as described in claim 1, characterized in that, The back-end data integration and transmission unit is a field-programmable gate array (FPGA), which is used to perform data integration, data format conversion, or high-speed data transmission.

9. The image processing method of the handheld imaging device as described in claim 1, characterized in that, The step of receiving surface image data and internal structure measurement data through a computing unit of the host computer, and mapping the internal structure measurement data to a corresponding position in the world coordinate system based on the spatial position and orientation information, so that the internal structure measurement data corresponds to a spatial position of the surface model, includes the following steps: The internal structural measurement data is converted into spatial volume data in one of the world coordinate systems, wherein the spatial volume data forms a three-dimensional voxel model.

10. The image processing method of the handheld imaging device as described in claim 1, characterized in that, Before the step in which the computing unit aligns and fuses the mapped internal structure measurement data with the surface model to generate a fused imaging result, the following steps are included: Perform a noise filtering procedure, a resolution enhancement procedure, or a signal compensation procedure on the internal structure measurement data.

11. The image processing method of the handheld imaging device as described in claim 1, characterized in that, The step of aligning and fusing the mapped internal structure measurement data with the surface model in the computing unit to generate a fused imaging result includes the following steps: The internal structure measurement data is sampled along one direction of the surface model to obtain the internal information corresponding to the surface model.

12. The image processing method of the handheld imaging device as described in claim 1, characterized in that, The fused imaging result is superimposed on the surface model in the form of a heatmap.

13. The image processing method of the handheld imaging device as described in claim 1, characterized in that, The fused imaging result includes either a cross-sectional image or a single rendered image.

Images