A three-dimensional model picture matching method and system for disease identification

Abstract

The application discloses a three-dimensional model picture matching method and system for disease identification, and belongs to the technical field of three-dimensional model construction. The system is used for realizing the method. The method comprises the following steps: acquiring two groups of picture data of the same object, namely picture data A and B. The resolution of the picture data A is higher than that of the picture data B. Distortion correction is performed on the picture data A and B. Image matching is performed on the non-distortion picture A and the non-distortion picture B. A three-dimensional model and a camera perspective projection model are constructed based on the picture data B. A depth map is constructed based on the three-dimensional model and the camera perspective projection model of the picture data B. The non-distortion picture B is mapped onto a model texture through the camera perspective projection model and the depth map. The non-distortion picture A is used to replace the content of the non-distortion picture B, and the content of the non-distortion picture B is hidden. The application can reduce the data amount when three-dimensional modeling is performed through high-quality pictures, and reduce the modeling time.

Description

Technical Field

[0001] This invention relates to the field of 3D model construction technology, specifically to a 3D model image matching method and system for disease identification. Background Technology

[0002] Currently, in some engineering projects involving image-based defect identification, the workflow is as follows: taking photos, identifying defects in the photos, creating a 3D model of the project based on the photos, and then mapping the photos onto the 3D model to locate the corresponding defects. However, this method requires high-precision photos for defect identification, hence the aerial photographs used are all high-precision, and the 3D models built from these photos are also high-quality models. But in most projects, the main purpose of the 3D model is to provide a visual representation of the defect location, with lower quality requirements. While building a 3D model from high-precision aerial photographs offers high accuracy and detail, it also results in a large data volume, long processing time, and high resource consumption for displaying the model, making it wasteful for practical engineering work. A relevant technical solution, such as the one disclosed in patent application CN202310893947.9 (Invention Title: A Method for 3D Reconstruction and Defect Identification of Buildings Based on NeRF 5D Neural Radiation Field), extracts the 3D coordinates of the image and the camera pose during image acquisition to form the feature input for a 3D reconstruction-semantic segmentation network used for 3D reconstruction, and further realizes 3D reconstruction of buildings based on a three-branch network model.

[0003] Therefore, two cameras can be set up when taking photos: one for high-quality images and the other for general images. The high-quality images are used for lesion identification, while the general images are used to build a 3D model, thus completing the task. However, the drawback of this method is that the 3D model is built from general images, while the lesions are identified from high-quality aerial photographs. How to map the two onto the 3D model and ensure that the lesions are correctly positioned is a problem that urgently needs to be solved. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a method for matching three-dimensional model images for disease identification. In application, it performs three-dimensional modeling using ordinary photos and maps high-quality photos onto the three-dimensional model, thereby reducing the amount of data and modeling time when performing three-dimensional modeling using high-quality photos. At the same time, the model established has high accuracy in disease identification.

[0005] The objective of this invention is mainly achieved through the following technical solution: Firstly, this application discloses a method for matching three-dimensional model images for disease identification, comprising the following steps:

[0006] Two sets of photographic data of the same object are acquired and denoted as photographic data A and photographic data B, respectively. The resolution of photographic data A is higher than that of photographic data B.

[0007] Distortion correction is performed on the photo data A and photo data B, and they are respectively denoted as distortion-free photo A and distortion-free photo B;

[0008] Perform image matching between distortion-free image A and distortion-free image B;

[0009] Based on the photo data B, construct a 3D model and a camera perspective projection model;

[0010] A depth map is constructed based on the 3D model and camera perspective projection model of the photo data B;

[0011] The distortion-free photo B is mapped onto the model texture using a camera perspective projection model and a depth map.

[0012] Based on the image matching data of distortion-free photo A and distortion-free photo B, the content of distortion-free photo B is replaced with distortion-free photo A and the content of distortion-free photo B is hidden, thus completing the mapping of photo data A onto the 3D model constructed from photo data B.

[0013] In existing technologies, 3D modeling using high-precision photographs requires significant resources, involves large amounts of data, and is time-consuming. Conversely, 3D modeling using ordinary photographs cannot meet the requirements for disease identification. Therefore, achieving high-precision modeling with limited resources is a problem that needs to be solved. In this application, for high-precision photograph data A and low-precision photograph data B, after image distortion correction of photograph data A and B, image matching is performed on the distortion-free photographs A and B. Then, 3D modeling and camera perspective projection modeling are performed using the photograph data. A depth map is constructed using the 3D model and camera perspective projection model of photograph data B. Based on the camera perspective projection model and depth map of photograph data B, the distortion-free photograph data B is mapped onto its 3D model. Since image matching was performed on the distortion-free photograph data A and B before modeling, the content of the distortion-free photograph B can be directly replaced with the content of the distortion-free photograph A on the 3D model of photograph data B. This allows for mapping of the high-precision photograph onto the low-precision 3D model, achieving disease identification. Unlike existing technologies, this application does not require 3D modeling of high-precision photos to identify diseases. Instead, it only requires 3D modeling of lower-precision photos and image matching between high-precision and lower-precision photos. This allows the high-precision photos to be mapped onto the lower-precision 3D model, thereby enabling accurate disease identification on the model.

[0014] In one possible implementation, when photographic data A is captured, adjacent photographs have a set overlap rate. In this application, for the same object, photographic data B has a lower resolution and a wider coverage area, while photographic data A has a higher resolution and a smaller coverage area. Therefore, during image matching, multiple images in photographic data A must be matched with one photograph in photographic data B. To ensure the continuity of the mapping of photographic data A on the 3D model, the overlap rate of each photograph in photographic data A must be guaranteed when capturing photographic data A.

[0015] In one possible implementation, when correcting distortion of photo data A and photo data B, photo data A and photo data B are separately entered into Photoshop software for distortion correction. Distortion correction is performed according to the data type and distortion type of photo data A or photo data B. When photographing an object, the original photo data A and photo data B captured by the camera may be distorted due to lens distortion, perspective distortion, and other reasons. Therefore, in this application, photo data A and photo data B are entered into Photoshop software for distortion correction. Different methods are used for distortion correction depending on the distortion type and photo data format. For example, if the photo distortion is caused by lens distortion and the photo is in RAW format, the photo is opened with Photoshop's ACR or LR, and correction is performed by enabling lens profiles. During correction, the corresponding camera and lens model are selected for automatic distortion correction. Alternatively, manual correction can be performed using the manual correction function in ACR. It should be noted that if the photo is in JPG format, only manual correction can be used in ACR to correct distortion.

[0016] In one possible implementation, feature matching or template matching methods are used to perform image matching between distortion-free photo A and distortion-free photo B. To ensure that distortion-free photo A can be accurately mapped onto the 3D model, in this application, before replacing distortion-free photo B with distortion-free photo A, image matching is performed between distortion-free photo A and distortion-free photo B. Image matching methods include feature matching and template matching methods. Image matching methods are existing technologies known in the art and will not be elaborated upon here.

[0017] In one possible implementation, before constructing the camera perspective projection model, the photo data B is processed to obtain camera intrinsic parameters. In this application, before constructing the camera perspective projection model, the camera intrinsic parameters are obtained first, and the camera perspective projection model is constructed based on the camera intrinsic parameters.

[0018] Secondly, this application discloses an image matching system that applies a matching method for three-dimensional model images used for disease identification as described above, including an image capture module, a data processing module, a three-dimensional model construction module, a perspective projection model construction module, a depth map generation module, and an image replacement module.

[0019] The shooting module is configured to acquire two sets of photo data of the same object, the two sets of photo data being denoted as photo data A and photo data B, respectively, wherein the resolution of photo data A is higher than the resolution of photo data B;

[0020] The data processing module is configured to perform distortion correction on photo data A and photo data B, and to perform image matching on the distortion-corrected undistorted photo A and undistorted photo B.

[0021] The 3D model building module is configured to build a 3D model of photo data B based on photo data B;

[0022] The perspective projection model building module is configured to build a camera perspective projection model of photo data B based on the photo data B;

[0023] The depth map generation module is configured to construct a depth map based on the 3D model and camera perspective projection model of the photo data B;

[0024] The image replacement module is configured to replace the content of the distortion-free photo B with the distortion-free photo A based on the image matching data of the distortion-free photo A and the distortion-free photo B, and hide the content of the distortion-free photo B, thereby completing the mapping of photo data A on the three-dimensional model constructed from photo data B.

[0025] Furthermore, the data processing module includes Photoshop software, and the distortion correction of photo data A and photo data B is achieved through Photoshop software.

[0026] In summary, the present invention has the following advantages compared with the prior art: the present invention performs 3D modeling using ordinary photos and maps high-quality photos onto the 3D model, thereby reducing the amount of data and modeling time when performing 3D modeling using high-quality photos. At the same time, the model established has high accuracy in disease identification. Attached Figure Description

[0027] The accompanying drawings, which are included to provide a further understanding of embodiments of the invention and form part of this application, do not constitute a limitation thereof. In the drawings:

[0028] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the embodiments and accompanying drawings. The illustrative embodiments and descriptions of the present invention are only used to explain the present invention and are not intended to limit the present invention.

[0030] Example 1:

[0031] like Figure 1 As shown, a method for matching 3D model images for disease identification includes the following steps:

[0032] Two sets of photographic data of the same object are acquired and denoted as photographic data A and photographic data B, respectively. The resolution of photographic data A is higher than that of photographic data B.

[0033] Distortion correction is performed on the photo data A and photo data B, and they are respectively denoted as distortion-free photo A and distortion-free photo B;

[0034] Perform image matching between distortion-free image A and distortion-free image B;

[0035] Based on the photo data B, construct a 3D model and a camera perspective projection model;

[0036] A depth map is constructed based on the 3D model and camera perspective projection model of the photo data B;

[0037] The distortion-free photo B is mapped onto the model texture using a camera perspective projection model and a depth map.

[0038] Based on the image matching data of distortion-free photo A and distortion-free photo B, the content of distortion-free photo B is replaced with distortion-free photo A and the content of distortion-free photo B is hidden, thus completing the mapping of photo data A onto the 3D model constructed from photo data B.

[0039] In existing technologies, 3D modeling using high-precision photos requires a large amount of resources, involves a large amount of data, and takes a long time. On the other hand, 3D modeling using ordinary photos cannot meet the requirements for disease identification. Therefore, how to achieve high-precision modeling with limited resources is a problem that needs to be solved.

[0040] In this embodiment, for high-precision (resolution) photo data A and low-precision (resolution) photo data B relative to photo data A, after image distortion correction of photo data A and photo data B, image matching is performed on the distortion-corrected undistorted photo A and undistorted photo B. Then, 3D modeling and camera perspective projection modeling are performed using the photo data, and a depth map is constructed using the 3D model and camera perspective projection model of photo data B. Thus, the undistorted photo data B is mapped onto the 3D model of photo data B based on the camera perspective projection model and depth map of photo data B. Since image matching was performed on undistorted photo data A and undistorted photo data B before modeling, the content of undistorted photo B can be directly replaced with the content of undistorted photo A on the 3D model of photo data B. This allows mapping of high-precision photos to be completed on a low-precision 3D model, achieving disease identification.

[0041] Unlike existing technologies, this application does not require direct 3D modeling of high-precision photos to identify diseases. Instead, it only requires 3D modeling of lower-precision photos and image matching between high-precision and lower-precision photos. This allows the high-precision photos to be mapped onto the lower-precision 3D model, thereby enabling accurate disease identification on the model.

[0042] More specifically, the core idea of ​​the above technical solution is to achieve high-precision 3D modeling based on data decoupling and mapping replacement, ultimately obtaining a high-precision 3D model, and solving the problems of massive data volume, high computing power requirements, and time-consuming processing and rendering in the high-precision 3D modeling process. In the specific solution, the data decoupling means separating the data used for modeling (photo data B) from the data used to improve the accuracy of the 3D model (photo data A). The mapping replacement means mapping high-precision distortion-free photo A and distortion-free photo B through image matching data, and using this mapping to replace photos on the 3D model based on the lightweight 3D model obtained based on photo data B, ultimately obtaining a 3D model loaded with high-precision disease information.

[0043] More detailed:

[0044] Regarding the acquisition of two sets of photographic data of the same object, the following method can be adopted: For the same target object (e.g., a highway bridge or dam), two imaging systems are used to acquire photographic data A and photographic data B by synchronous or distributed shooting. Considering the efficiency of acquiring the original aerial photographs (photo data), a preferred approach is to equip the aircraft with two imaging systems for acquiring photographic data A and photographic data B respectively. Different imaging systems have different lens parameters and image sensor parameters. Photo data A and photographic data B are acquired synchronously under the planned flight path. When shooting photographic data A and photographic data B, the aerial photographs contained in each type of photographic data have a set overlap rate (overlap rate set to 30%~80%). Since the data accuracy requirement for photographic data B is low, in order to reduce the amount of photographic data, the number of photographic data A photos is set to be greater than the number of photographic data B photos.

[0045] Regarding the distortion correction methods for each photo data, the following can be adopted: For the camera equipped in each imaging system, before the camera performs the shooting task, use a standard calibration board to obtain the camera's distortion parameters. The distortion parameters include the lens radial distortion parameters and tangential distortion parameters. For photo data A, when using this data in later stages, more attention should be paid to the texture details of the defects it can reflect. The focus of distortion correction includes visual effects. The specific method can be to import the aerial shooting data into Photoshop (Adobe Photoshop) software after completion, and use the distortion parameters as the parameters to guide the manual distortion correction, or as the configuration file parameters for distortion correction in Photoshop software, to eliminate the corresponding barrel distortion and / or pincushion distortion. For photo data B, which is used for 3D model construction, this data can be directly imported into 3D construction software (such as ContextCapture, DJI Terra), and the software can be used to perform SFM (Structure from Motion) processing associated with the distortion parameters to finally obtain distortion-free photo B. The degree of distortion correction should aim to achieve geometric consistency between the two photo data after correction.

[0046] Regarding the above image matching, the specific approach is as follows: The system extracts feature points that meet the minimum number of feature points from both distortion-free photo A and distortion-free photo B, respectively, based on the set minimum number of feature points. Subsequently, based on the feature matcher, the image matching relationship between the two distortion-free photos is established by using a feature point-based feature matching method.

[0047] The construction of the 3D model and camera perspective projection model based on the aforementioned photo data B, as well as the completion of the depth map construction, are all common techniques in aerial 3D modeling. For those skilled in the art, a specific approach can be taken using ContextCapture 3D modeling software. After denoising and color balancing the photo data B, feature matching (stitching) and triangulation (positioning) methods can be used to construct the lightweight 3D model. For those skilled in the art, based on existing 3D modeling software, the software will automatically perform feature extraction, matching, triangulation, and multi-view stereo calculations, simultaneously outputting two results: one is a triangular mesh 3D model of the object (a 3D digital representation of the object's geometry); the other is the camera perspective corresponding to each photo data B. The projection model (containing a mathematical matrix of camera intrinsic and extrinsic parameters during shooting, used to define the rules for projecting 3D spatial points onto the 2D plane of the photograph) further involves determining the viewing angle for each photograph based on its camera extrinsic parameters, then calculating the distance (i.e., depth value) from each visible triangular facet sampling point on the 3D model to the camera optical center based on that viewing angle, and rendering these depth values ​​onto a 2D image of the same size as the photograph according to the projection relationship, thereby generating a depth map of that viewing angle. The grayscale value of each pixel in the depth map represents the depth information of the object surface observed by the corresponding pixel in the photograph. In this scheme, since the input photograph data B has a low resolution, the constructed 3D model has a small data volume and fast processing speed, achieving a 3D model output that meets the requirements of lightweight design.

[0048] Regarding the mapping of the distortion-free photo B onto the model texture, the specific process is the texture mapping process in the 3D modeling process. Specifically, the 3D modeling software uses the camera perspective projection model and depth map to calculate which photo(s) pixel regions should fill each triangular facet, ultimately generating a complete 3D model with the texture of the distortion-free photo B.

[0049] The mapping of photo data A onto the 3D model constructed from photo data B is as follows: Distortion-free photo B is mapped onto the model texture to obtain a lightweight 3D model (baseline 3D model). For the 3D model obtained based on distortion-free photo B, any texture on this 3D model has a specific correspondence with the pixel region of distortion-free photo B. After obtaining the pixel region of distortion-free photo B corresponding to the texture on the 3D model, the image matching data obtained through image matching is used to query the specific pixel region in distortion-free photo A for that pixel region in distortion-free photo B. The pixel regions in distortion-free photo A are then used to replace the pixel regions in distortion-free photo B in the 3D model. By traversing all texture regions on the 3D model contributed by distortion-free photo B, the original lightweight 3D model's texture map is replaced with a texture map capable of accurately displaying the details of object defects, ultimately resulting in a high-precision 3D model.

[0050] As is easily understood, the core concept behind the above 3D model image matching method is not how to construct a 3D model, but rather, from the perspective of heterogeneous photo data and its uses, it provides a method for obtaining a high-precision 3D model that clearly reflects the characteristics of an object's defects based on a lightweight 3D model.

[0051] Taking the direct construction of a 3D model with object defect features using high-resolution aerial photographs as an example, the aerial photographs used have a resolution of 40 megapixels. Furthermore, to meet the overlap requirements, multiple aerial photographs need to be taken. During the 3D modeling process, feature extraction and model matching, sparse point cloud and camera pose calculation, 3D spatial point depth and color calculation, meshing, and texture mapping are performed from these aerial photographs. This requires high computing power and a long time; even on a high-performance workstation, the time required can reach tens of hours or even days. However, using this solution, if photo data B uses a resolution such as 5 megapixels (and photo data A uses a resolution such as 40 megapixels or...), the computational power required is high and the time is long. Higher resolution (because photo data B has a lower resolution, the field of view can be larger when shooting, so fewer photo data Bs are needed to meet the modeling requirements. In the lightweight 3D modeling process, the total amount of data that needs to be transmitted and loaded is reduced. At the same time, the above feature extraction and modeling matching, sparse point cloud and camera pose calculation, 3D spatial point depth and color calculation, meshing and texture mapping are all based on photo data B. Since the total number of feature points is less, the feature point cloud is smaller, the resolution and number of images are greatly reduced in the dense reconstruction process, and the final geometric model file size is significantly reduced, the total modeling time can be greatly shortened, and the requirements for data processing hardware are reduced.

[0052] Example 2:

[0053] This embodiment is a further refinement of embodiment 1:

[0054] When photographic data A is captured, adjacent photographs have a set overlap rate. In this application, for the same object, photographic data B has a lower resolution and a wider coverage area, while photographic data A has a higher resolution and a smaller coverage area. Therefore, when performing image matching, multiple images in photographic data A can be matched with one photograph in photographic data B. To ensure the continuity of the mapping of photographic data A on the 3D model, the overlap rate of each photograph in photographic data A must be guaranteed when capturing photographic data A.

[0055] Example 3:

[0056] This embodiment is a further refinement based on embodiment 1 or 2:

[0057] When performing distortion correction on photo data A and photo data B, both photo data A and photo data B are separately entered into Photoshop for distortion correction. Distortion correction is performed according to the data type and distortion type of photo data A or photo data B. When photographing an object, the original photo data A and photo data B captured by the camera may be distorted due to lens distortion, perspective distortion, and other reasons. Therefore, in this application, photo data A and photo data B are entered into Photoshop for distortion correction. Different distortion correction methods are used for different distortion types and photo data formats. For example, if the photo distortion is caused by lens distortion and the photo is in RAW format, the photo is opened with Photoshop's ACR or LR, and correction is performed by enabling lens profiles. During correction, the corresponding camera and lens model are selected for automatic distortion correction. Alternatively, manual correction can be performed using the manual correction function in ACR. It should be noted that if the photo is in JPG format, only manual correction can be used in ACR to correct distortion.

[0058] Example 4:

[0059] This embodiment is a further refinement based on any of the above embodiments:

[0060] Image matching is performed on distortion-free photograph A and distortion-free photograph B using feature matching or template matching methods. To ensure that distortion-free photograph A can be accurately mapped onto the 3D model, in this application, image matching is performed on distortion-free photograph A and distortion-free photograph B before replacing them with distortion-free photograph A. Image matching methods include feature matching and template matching methods. These image matching methods are well-known prior art and will not be elaborated upon here.

[0061] Example 5:

[0062] This embodiment is a further refinement based on any of the above embodiments:

[0063] Before constructing the camera perspective projection model, the photo data B is processed to obtain camera intrinsic parameters. In this application, before constructing the camera perspective projection model, the camera intrinsic parameters are obtained first, and the camera perspective projection model is constructed based on the camera intrinsic parameters.

[0064] Example 6:

[0065] Based on any of the above embodiments, this embodiment provides a matching system for three-dimensional model images for disease identification. The system is used to implement the three-dimensional model image matching method described in any of the above embodiments. The matching system includes an image capture module, a data processing module, a three-dimensional model construction module, a perspective projection model construction module, a depth map generation module, and an image replacement module.

[0066] The shooting module is configured to acquire two sets of photo data of the same object, the two sets of photo data being denoted as photo data A and photo data B, respectively, wherein the resolution of photo data A is higher than the resolution of photo data B;

[0067] The data processing module is configured to perform distortion correction on photo data A and photo data B, and to perform image matching on the distortion-corrected undistorted photo A and undistorted photo B.

[0068] The 3D model building module is configured to build a 3D model of photo data B based on photo data B;

[0069] The perspective projection model building module is configured to build a camera perspective projection model of photo data B based on the photo data B;

[0070] The depth map generation module is configured to construct a depth map based on the 3D model and camera perspective projection model of the photo data B;

[0071] The image replacement module is configured to replace the content of the distortion-free photo B with the distortion-free photo A based on the image matching data of the distortion-free photo A and the distortion-free photo B, and hide the content of the distortion-free photo B, thereby completing the mapping of photo data A on the three-dimensional model constructed from photo data B.

[0072] Example 7:

[0073] This embodiment is a further refinement based on any of the above embodiments:

[0074] The data processing module includes Photoshop software, and the distortion correction of photo data A and photo data B is achieved through Photoshop software.

[0075] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A three-dimensional model image matching method for disease identification, characterized in that, Includes the following steps: Two sets of photographic data of the same object are acquired and denoted as photographic data A and photographic data B, respectively. The resolution of photographic data A is higher than that of photographic data B. Distortion correction is performed on the photo data A and photo data B, and they are respectively denoted as distortion-free photo A and distortion-free photo B; Perform image matching between distortion-free image A and distortion-free image B; Based on the photo data B, construct a 3D model and a camera perspective projection model; A depth map is constructed based on the 3D model and camera perspective projection model of the photo data B; The distortion-free photo B is mapped onto the model texture using the camera perspective projection model and depth map. Specifically, the 3D modeling software calculates which photo(s) should fill each triangular facet using the camera perspective projection model and depth map, ultimately generating a complete 3D model with the texture of the distortion-free photo B. Based on image matching data from distortion-free photo A and distortion-free photo B, the content of distortion-free photo B is replaced with the content of distortion-free photo A, and the content of distortion-free photo B is hidden. This completes the mapping of photo data A onto the 3D model constructed from photo data B. Specifically, distortion-free photo B is mapped onto the model texture to obtain a lightweight 3D model. For the 3D model obtained based on distortion-free photo B, any texture on the 3D model has a specific correspondence with the pixel region of distortion-free photo B. After obtaining the pixel region of distortion-free photo B corresponding to the texture on the 3D model, the image matching data obtained through image matching is used to query the specific pixel region of the pixel region in distortion-free photo A. The pixel region in distortion-free photo A is used to replace the pixel region of distortion-free photo B in the 3D model. By traversing all texture regions on the 3D model contributed by distortion-free photo B, the texture map of the original lightweight 3D model is replaced with a texture map that can accurately display the details of the object's defects, ultimately obtaining a high-precision 3D model.

2. The three-dimensional model image matching method for disease identification according to claim 1, characterized in that, When taking photo data A, there is a set overlap rate between adjacent photos.

3. The three-dimensional model image matching method for disease identification according to claim 1, characterized in that, When performing distortion correction on photo data A and photo data B, photo data A and photo data B are entered into Photoshop software separately for distortion correction. Distortion correction is performed according to the data type and distortion type of photo data A or photo data B.

4. The three-dimensional model image matching method for disease identification according to claim 1, characterized in that, Image matching is performed on distortion-free photo A and distortion-free photo B using feature matching or template matching methods.

5. The three-dimensional model image matching method for disease identification according to claim 1, characterized in that, Before constructing the camera perspective projection model, the photo data B is processed and the camera intrinsic parameters are obtained.

6. A three-dimensional model image matching system for disease identification, applied to implement the three-dimensional model image matching method according to any one of claims 1 to 5, characterized in that, It includes a shooting module, a data processing module, a 3D model building module, a perspective projection model building module, a depth map generation module, and an image replacement module; The shooting module is configured to acquire two sets of photo data of the same object, the two sets of photo data being denoted as photo data A and photo data B, respectively, wherein the resolution of photo data A is higher than the resolution of photo data B; The data processing module is configured to perform distortion correction on photo data A and photo data B, and to perform image matching on the distortion-corrected undistorted photo A and undistorted photo B. The 3D model building module is configured to build a 3D model of photo data B based on photo data B; The perspective projection model building module is configured to build a camera perspective projection model of photo data B based on the photo data B; The depth map generation module is configured to construct a depth map based on the 3D model and camera perspective projection model of the photo data B; The distortion-free photo B is mapped onto the model texture using the camera perspective projection model and depth map. Specifically, the 3D modeling software calculates which photo(s) should fill each triangular facet using the camera perspective projection model and depth map, ultimately generating a complete 3D model with the texture of the distortion-free photo B. The image replacement module is configured to replace the content of distortion-free photo B with the content of distortion-free photo A based on the image matching data of distortion-free photo A and distortion-free photo B, and hide the content of distortion-free photo B, thus completing the mapping of photo data A onto the 3D model constructed from photo data B. Specifically, distortion-free photo B is mapped onto the model texture to obtain a lightweight 3D model. For the 3D model obtained based on distortion-free photo B, any texture on the 3D model has a specific correspondence with the pixel region of distortion-free photo B. After obtaining the pixel region of distortion-free photo B corresponding to the texture on the 3D model, the specific pixel region of the pixel region in distortion-free photo A is queried through the image matching data obtained by the image matching. The pixel region in distortion-free photo A is used to replace the pixel region of distortion-free photo B in the 3D model, and all texture regions contributed by distortion-free photo B on the 3D model are traversed. This replaces the original lightweight 3D model's texture map with a texture map that can accurately display the details of the object's defects, ultimately obtaining a high-precision 3D model.

7. The three-dimensional model image matching system for disease identification according to claim 6, characterized in that, The data processing module includes Photoshop software, and the distortion correction of photo data A and photo data B is achieved through Photoshop software.

Images