An automatic relocation and matching method, apparatus, equipment, medium, and product for panoramic images of cross-period inspections.

Abstract

This application discloses an automatic relocation and matching method, apparatus, equipment, medium, and product for panoramic images during cross-period inspections, relating to the field of engineering inspection technology. Using the target frame with the best match to the defect frame as the origin, this application employs a bidirectional local mapping mechanism. This not only significantly reduces the ineffective computational overhead of redundant videos but also fundamentally eliminates scale drift caused by long-sequence extrapolation, ensuring extremely high 3D relocation accuracy for the core areas of interest. Simultaneously, by combining spherical feature extraction and the ZNCC local depth matching algorithm, it effectively overcomes severe stretching distortion of panoramic images and environmental interference from complex lighting fluctuations across different periods. This application achieves highly robust, automated, and accurate mapping of defect locations, providing a foundation for quantitative analysis and digital closed-loop management of defects in building engineering.

Description

Technical Field

[0001] This application relates to the field of engineering inspection technology, and in particular to an automatic repositioning and matching method, device, equipment, medium and product for panoramic images of cross-period inspections. Background Technology

[0002] In engineering construction management, quality and safety inspections are a core component. Currently, the mainstream inspection and recording methods in the industry are still primarily manual. Inspectors typically carry ordinary cameras or smartphones to take two-dimensional perspective photos on-site and manually mark (e.g., circle) problem areas on the images as a record of quality and safety issues. In the next round of inspections, workers need to compare the marked problem areas with the previous image records to confirm whether the rectification has been completed, thereby achieving problem feedback and management. During this process, achieving accurate relocation of the same location between inspection images from different periods and batches becomes a tedious and error-prone critical step.

[0003] In recent years, some solutions have begun to incorporate computer vision-based 3D relocalization technologies (such as Simultaneous Localization and Mapping (SLAM) or Visual Odometry (VO)). 3D relocalization technology can reconstruct the camera's motion trajectory and the sparse 3D point cloud of the scene by analyzing a series of continuously captured images. Theoretically, applying this to inspections can mount discrete inspection photos onto a unified 3D coordinate system, enabling location tracking during the inspection process. This assigns spatial attributes to images of potential hazards, assisting supervisors in tracing the location across different time periods. However, directly applying existing conventional 3D relocalization technologies to large-scale, multi-period panoramic inspection scenarios still faces several technical bottlenecks. These include severe scale drift caused by long-sequence relocalization simulations; matching failures due to differences in ambient lighting over time; and severe spherical distortion of panoramic images affecting matching results. Summary of the Invention

[0004] The purpose of this application is to provide an automatic repositioning and matching method, device, equipment, medium, and product for cross-period inspection panoramic images, so as to improve the accuracy of automatic repositioning and matching of cross-period inspection panoramic images.

[0005] To achieve the above objectives, this application provides the following solution.

[0006] Firstly, this application provides an automatic relocation and matching method for panoramic images from cross-period inspections, including: Obtain the inspection video after rectification; In the inspection video, a panoramic image of the target frame that matches the panoramic image of the defect frame is selected; the panoramic image of the defect frame is a panoramic image with defect annotation boxes before rectification. Centered on the target frame, panoramic images of the predicted number of frames before and after the target frame are selected in the inspection video to form a panoramic image sequence. Based on the panoramic view of the target frame and the panoramic view of the secondary key frame, perform the initialization operation of 3D mapping, and construct a global coordinate system and a global sparse point cloud library; the secondary key frame is the frame following the target frame. Based on the global sparse point cloud library, the panoramic images of all auxiliary frames in the panoramic image sequence are matched and triangulated, and the global 3D map points constructed based on the panoramic images of each auxiliary frame are added to the global sparse point cloud library; the auxiliary frames are the frames in the panoramic image sequence other than the target frame and the secondary key frame; Based on the global sparse point cloud library, the panoramic image of the defect frame is matched and solved to obtain the pose transformation matrix of the defect frame in the global coordinate system. Based on the pose transformation matrix of the defect frame in the global coordinate system, the ZNCC (Zero-mean Normalized Cross-Correlation) algorithm is used to map the center pixel of the defect annotation box in the panoramic image of the defect frame to the panoramic image of the target frame, thereby obtaining the corresponding position of the center pixel of the defect annotation box in the panoramic image of the target frame.

[0007] Optionally, a spherical feature extraction algorithm is used to select a panoramic image of a target frame that matches the panoramic image of the defect frame in the inspection video, specifically including: Using a spherical feature extraction algorithm, spherical feature points are extracted from the panoramic view of each frame in the inspection video and the panoramic view of the defect frame, respectively. Based on the spherical feature points of the panoramic images of each frame in the inspection video and the panoramic images of the defect frames, the KNN matching algorithm is used to match feature points, and the number of matching point pairs between the panoramic images of the defect frames and the panoramic images of each frame in the inspection video is calculated respectively. The panoramic image of the frame with the largest number of matching point pairs in the inspection video is selected as the panoramic image of the target frame.

[0008] Optionally, based on the panoramic images of the target frame and the sub-keyframes, initialization operations for 3D mapping are performed to construct a global coordinate system and a global sparse point cloud library, specifically including: Convert the camera coordinate system of the target frame to the global coordinate system; Spherical features are extracted and matched from the panoramic images of the target frame and the secondary key frame to obtain multiple first matching point pairs; The two-dimensional pixel coordinates of each matching point in each first matching point pair are projected onto the unit spheres of the target frame and the secondary key frame, respectively, to obtain the 3D unit direction ray of each matching point in each first matching point pair; The ACRANSAC (AContrario Random Sample Consensus) algorithm is used to solve the 3D unit direction ray of each matching point in each first matching point pair to obtain the essential matrix between the target frame and the secondary key frame. Singular value decomposition is performed on the essential matrix to obtain multiple sets of candidate transformation matrices; each set of candidate transformation matrices includes a rotation matrix and a translation vector of the secondary keyframe relative to the target frame. Candidate transformation matrices that satisfy the positive depth verification constraint are selected as relative transformation matrices; Triangulation is performed on each first matching point pair based on the relative transformation matrix to obtain multiple global 3D map points constructed based on the initialization operation; A global sparse point cloud is constructed based on multiple global 3D map points built from initialization operations.

[0009] Optionally, based on the global sparse point cloud library, the panoramic images of all auxiliary frames in the panoramic image sequence are matched and triangulated, and the global 3D map points constructed based on the panoramic image of each auxiliary frame are added to the global sparse point cloud library, specifically including: Spherical features are extracted and matched between the panoramic images of the current frame and the panoramic images of adjacent frames to obtain multiple second matching point pairs; the current frame is the auxiliary frame currently participating in the localization, and the adjacent frames are frames that have been localized and are adjacent to the current frame; for auxiliary frames before the target frame, the current frame is obtained in order of increasing time distance from the target frame, and for auxiliary frames after the target frame, the current frame is obtained in order of increasing time distance from the secondary key frame; Based on the second matching point pair that exists in the global sparse point cloud library and has a corresponding global 3D map point, the PnP (Perspective-n-Point) algorithm is used to solve the problem and reversely determine the rotation matrix and translation vector of the current frame relative to the global coordinate system. Based on the rotation matrix and translation vector of the current frame relative to the global coordinate system, the second matching point pair that does not have a corresponding global 3D map point in the global sparse point cloud library is triangulated to obtain global 3D map points constructed based on the panoramic image of the current frame, and added to the global sparse point cloud library in an incremental manner.

[0010] Optionally, based on a global sparse point cloud library, the panoramic image of the defect frame is matched and solved to obtain the pose transformation matrix of the defect frame in the global coordinate system, specifically including: The KNN (K-Nearest Neighbors) algorithm is used to extract and match spherical features between the panoramic image of the defective frame and the panoramic images of each frame in the panoramic image sequence, thereby obtaining multiple third matching point pairs. The third matching point pair that contains global 3D map points in the global sparse point cloud database is selected as the fourth matching point pair; Based on multiple fourth matching point pairs, the PnP algorithm is used to solve the problem and reverse the position transformation matrix of the defect frame in the global coordinate system.

[0011] Optionally, based on the pose transformation matrix of the defect frame in the global coordinate system, the ZNCC algorithm is used to map the center pixel of the defect annotation box in the panoramic image of the defect frame to the panoramic image of the target frame, thereby obtaining the corresponding position of the center pixel of the defect annotation box in the panoramic image of the target frame. Specifically, this includes: Generate multiple candidate depth values ​​for the center pixel of the defect annotation box; For each candidate depth value, the simulated 3D coordinates of the center pixel of the defect annotation box in the global coordinate system are calculated based on the pose transformation matrix of the defect frame in the global coordinate system. The simulated 3D coordinates corresponding to each candidate depth value are projected onto the panoramic image of each frame in the panoramic image sequence, and the weighted average of the ZNCC score corresponding to each candidate depth value is calculated. The candidate depth value corresponding to the optimal weighted average value is selected as the optimal depth value; The simulated 3D coordinates corresponding to the optimal depth value are projected onto the panoramic view of the target frame and used as the corresponding position of the center pixel of the defect annotation box in the panoramic view of the target frame.

[0012] Secondly, this application provides an automatic relocation and matching device for cross-period inspection panoramic images. The automatic relocation and matching device for cross-period inspection panoramic images is applied to the aforementioned automatic relocation and matching method for cross-period inspection panoramic images. The automatic relocation and matching device for cross-period inspection panoramic images includes: The inspection video acquisition module is used to acquire inspection videos after rectification. The target frame selection module is used to select a panoramic image of a target frame that matches the panoramic image of the defect frame in the inspection video; the panoramic image of the defect frame is a panoramic image with defect annotation boxes before rectification. The panoramic image sequence construction module is used to select panoramic images of a predicted number of frames before and after the target frame in the inspection video, with the target frame as the center, and form a panoramic image sequence. The initialization operation module is used to perform 3D mapping initialization operations based on the panoramic image of the target frame and the panoramic image of the secondary key frame, and to construct a global coordinate system and a global sparse point cloud library; the secondary key frame is the frame following the target frame. The global map dynamic update module is used to match and triangulate the panoramic images of all auxiliary frames in the panoramic image sequence based on the global sparse point cloud library, and add the global 3D map points constructed based on the panoramic images of each auxiliary frame to the global sparse point cloud library; the auxiliary frames are the frames in the panoramic image sequence other than the target frame and the secondary key frame. The defect frame matching module is used to match and solve the panoramic image of the defect frame based on the global sparse point cloud library, and obtain the pose transformation matrix of the defect frame in the global coordinate system. The mapping module is used to map the center pixel of the defect annotation box in the panoramic image of the defect frame to the panoramic image of the target frame using the ZNCC algorithm, based on the pose transformation matrix of the defect frame in the global coordinate system, and to obtain the corresponding position of the center pixel of the defect annotation box in the panoramic image of the target frame.

[0013] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described automatic relocation and matching method for cross-period inspection panoramic images.

[0014] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described automatic relocation and matching method for panoramic images of cross-period inspections.

[0015] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described automatic relocation and matching method for panoramic images during inter-period inspections.

[0016] According to the specific embodiments provided in this application, this application has the following technical effects.

[0017] This application provides an automatic relocation and matching method, apparatus, equipment, medium, and product for panoramic images from cross-period inspections. Using the target frame with the best match to the defect frame as the origin, this application employs a bidirectional local mapping mechanism. This not only significantly reduces the ineffective computational overhead of redundant videos but also fundamentally eliminates scale drift caused by long-sequence extrapolation, ensuring extremely high 3D relocation accuracy for the core areas of interest. Simultaneously, by combining spherical feature extraction and the ZNCC local depth matching algorithm, it effectively overcomes severe stretching distortion of panoramic images and environmental interference from complex lighting fluctuations across periods. This application achieves highly robust, automated, and accurate mapping of defect locations, providing a foundation for quantitative analysis and digital closed-loop management of defects in building engineering. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating an automatic relocation and matching method for panoramic images during cross-period inspections, provided as an embodiment of this application.

[0020] Figure 2 This is a schematic diagram illustrating the principle of an automatic relocation and matching method for panoramic images during cross-period inspections, provided in one embodiment of this application.

[0021] Figure 3 This is a schematic diagram of the key point feature matching result between the defective frame and the target frame provided in an embodiment of this application.

[0022] Figure 4 This is a schematic diagram of a global coordinate system provided in an embodiment of this application.

[0023] Figure 5 A schematic diagram illustrating the principle of the PnP algorithm for estimating the optical center pose of the current frame according to an embodiment of this application.

[0024] Figure 6 This is a schematic diagram of pose estimation for a local temporal frame provided in an embodiment of this application.

[0025] Figure 7 A schematic diagram illustrating the depth recovery principle of the center pixel of the defect annotation box provided in an embodiment of this application.

[0026] Figure 8 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0029] In traditional manual inspections for problem detection and closed-loop management, the process primarily relies on inspectors taking two-dimensional perspective photographs on-site, followed by manual screening and labeling of potential hazards by designers or managers. Several months later, during the follow-up inspection phase, it is necessary to manually compare the photos from different periods to confirm the rectification status. This traditional process has the following significant drawbacks.

[0030] Spatial information gaps and location difficulties: Local two-dimensional perspective photographs have limited field of view and lack a global spatial coordinate system for reference. When tracing back from a massive number of discrete photographs, designers rely heavily on their personal spatial imagination and memory, making it easy to miss or misjudge the location of potential hazards.

[0031] Data matching across different time periods is time-consuming and inefficient: When conducting re-inspections at different points in time, the shooting location, angle, and ambient lighting often vary greatly. Due to the lack of an automated GPS (Global Positioning System) registration mechanism, personnel must manually search and match the same target location for potential hazards among a massive number of two-dimensional photos taken across different time periods. This process is extremely time-consuming, and if the re-inspection fails to align with the original perspective, it directly leads to the inability to close the loop on rectification status.

[0032] Lack of quantitative basis and collaborative foundation: Blurry image quality or partial occlusion further reduces the accuracy of judgment, leading to errors in the judgment of potential hazards by inspection personnel and making it difficult to quantitatively analyze the rectification of defects.

[0033] This application provides an automatic relocation and matching method, device, equipment, medium, and product for cross-period inspection panoramic images. The aim is to achieve automated tracking and annotation of quality issues in cross-period inspection images. Specifically, after receiving the initial panoramic video images from the construction site, the supervision unit can interact with the system and select existing quality and safety hazards, generating rectification instructions. After the construction party completes the rectification, the latest (cross-period) panoramic video with a similar trajectory is entered. The system will automatically compare the two videos, accurately output cross-period target images containing the original hazard locations, and automatically identify the corresponding rectification areas.

[0034] In one exemplary embodiment, an automatic relocation and matching method for panoramic images from cross-period inspections is provided, such as... Figure 1 and Figure 2 As shown, it includes the following steps 101-107.

[0035] Step 101: Obtain the inspection video after rectification; Step 102: Using a spherical feature extraction algorithm, select a panoramic image of a target frame that matches the panoramic image of the defect frame in the inspection video; the panoramic image of the defect frame is a panoramic image with defect annotation boxes before rectification. Step 103: Taking the target frame as the center, select panoramic images of the predicted number of frames before and after the target frame in the inspection video to form a panoramic image sequence. Step 104: Based on the panoramic view of the target frame and the panoramic view of the secondary key frame, perform the initialization operation of 3D mapping, and construct a global coordinate system and a global sparse point cloud library; the secondary key frame is the frame following the target frame. Step 105: Based on the global sparse point cloud library, match and triangulate the panoramic images of all auxiliary frames in the panoramic image sequence, and add the global 3D map points constructed based on the panoramic images of each auxiliary frame to the global sparse point cloud library; the auxiliary frames are the frames in the panoramic image sequence other than the target frame and the secondary key frame; in this step 105, based on the global sparse point cloud library, match and triangulate the panoramic images of all auxiliary frames in the panoramic image sequence, calculate the pose relationship of each auxiliary frame relative to the global coordinate system, and construct the global 3D map points of each auxiliary frame; Step 106: Based on the global sparse point cloud library, match and solve the panoramic image of the defect frame to obtain the pose transformation matrix of the defect frame in the global coordinate system. Step 107: Based on the pose transformation matrix of the defect frame in the global coordinate system, the ZNCC algorithm is used to map the center pixel of the defect annotation box in the panoramic image of the defect frame to the panoramic image of the target frame, and obtain the corresponding position of the center pixel of the defect annotation box in the panoramic image of the target frame.

[0036] Implementing steps 101-107 above can achieve the following technical effects.

[0037] This application uses the target frame with the best match to the defect frame as the origin and employs a bidirectional local mapping mechanism. This not only significantly reduces the ineffective computational overhead of redundant video but also fundamentally eliminates scale drift caused by long-sequence inference, ensuring extremely high 3D repositioning accuracy for the core areas of interest. Simultaneously, by combining spherical feature extraction and the ZNCC local depth matching algorithm, it effectively overcomes severe stretching distortion of panoramic images and environmental interference from complex intertemporal lighting fluctuations. This application achieves highly robust, automated, and accurate mapping of defect locations, providing a foundation for quantitative analysis and digital closed-loop management of defects in building engineering.

[0038] In another exemplary embodiment, the process of acquiring the panoramic view of the defect frame described above is a process of defining and extracting the spatial information of the potential hazard location, as detailed below: The initial panoramic video from the inspection is input into the system (here, the system refers to a device capable of interacting with the user and performing image processing, analysis, and viewing, such as a computer). The user can browse the video in spherical panoramic mode and drag the viewpoint to expand a portion into a distortion-free perspective projection image. The video is paused at keyframes containing defects, and the user selects the affected area using the interactive interface. The system automatically records the timestamp of the "defect frame," the panoramic image data, and the two-dimensional pixel coordinates of the defect bounding box in the partial perspective view, storing these in the system database as a matching benchmark.

[0039] Compared to traditional methods that use conventional perspective images as inspection evidence, spherical panoramic video provides users with a wider field of view, allowing them to more intuitively locate the current shooting position, showcasing defect features and their surrounding environment from multiple angles, and also helping to quickly determine the cause of defects in remote locations.

[0040] In another exemplary embodiment, step 102 above is the process of feature extraction and nearest neighbor frame retrieval of cross-period images. In this process, the inspection video taken along a similar path after rectification is input into the system, and panoramic images of each frame in the inspection process are extracted according to preset rules (such as setting a fixed time interval of 0.5 seconds or a specified frame rate).

[0041] Subsequently, the SPHORB (Spherical Oriented FAST and Rotated BRIEF) algorithm was used to calculate the spherical image features of the panoramic images of the defect frame and the panoramic images of different frames in the inspection video. Furthermore, the KNN (K-Nearest Neighbors) matching algorithm was employed to calculate the number of matching point pairs between the defect frame and each frame in the inspection video. The frame with the most matching point pairs was selected, thus identifying it as the target frame with the highest visual overlap and the closest spatial pose to the defect frame. Figure 3 As shown. Subsequently, based on this target frame and its temporally adjacent frames, a local 3D mapping process will be initiated to locate the defect area.

[0042] In this embodiment, step 102 can be implemented using steps 201-203.

[0043] Step 201: Using the spherical feature extraction algorithm, feature points are extracted from the panoramic view of each frame and the panoramic view of the defect frame in the inspection video. Step 202: Based on the spherical feature points of the panoramic image of each frame in the inspection video and the panoramic image of the defect frame, use the KNN matching algorithm to perform feature point matching, and calculate the number of matching point pairs between the panoramic image of the defect frame and the panoramic image of each frame in the inspection video. Step 203: Select the panoramic image of the frame with the largest number of matching point pairs in the inspection video as the panoramic image of the target frame.

[0044] The SPHORB algorithm used in this embodiment has the following advantages compared to the traditional SIFT (Scale-Invariant Feature Transform) and ORB (Oriented Fast and Rotated BRIEF) algorithms: The SPHORB algorithm directly constructs feature descriptors in the spherical domain, fundamentally overcoming the limitations of panoramic imaging. Figure 2 Geometric distortion interference during dimensional unfolding significantly improves the matching accuracy between 'defective frames' and 'target frames' across time periods.

[0045] The feature matching and solving processes involved in the subsequent process are all based on the spherical features extracted by the SPHORB algorithm, and will not be described in detail in the subsequent description of the technical solution of this application.

[0046] In another exemplary embodiment, step 104 above is a process of relative pose estimation and establishing a global coordinate system based on the panoramic images of adjacent frames. This process uses the target frame (i.e., the optimal frame matching the defective frame) and its temporally adjacent subsequent frames (e.g., the next frame extracted at a preset time interval) as the basis for initializing the 3D mapping operation, such as... Figure 4 As shown. The specific method is as follows: First, key point feature matching is performed on the two panoramic images to obtain the first matching point pair. The two-dimensional pixel coordinates of the matching points on the panoramic image are mapped onto a unit sphere with a radius of 1 according to the spherical projection model to obtain the corresponding three-dimensional normalized spherical coordinates, that is, the "3D unit direction ray" of the corresponding points in the two frames.

[0047] Secondly, based on the corresponding points on the spheres of two frames, the ACRANSAC algorithm is used to robustly calculate the essential matrix E between the two frames. Then, Singular Value Decomposition (SVD) is performed on the essential matrix E to calculate the rotation matrix R and translation vector T of multiple cameras. Further, for the candidate transformation matrices generated by the decomposition, triangulation is used to calculate the spatial depth of the matching points. Using a positive depth value (i.e., a positive depth check constraint), the unique correct relative transformation matrix is ​​selected. This matrix accurately represents the relative spatial pose change between two consecutive frames (i.e., the target frame and the secondary keyframe).

[0048] Then, the matching points in the two unit spheres are triangulated using a defined relative transformation matrix. The depth of each matching point is calculated, with the relative translation distance between the two frames as the unit scale. This yields the relative 3D coordinates of each matching point without an absolute scale, i.e., the global 3D map points. Subsequent 3D reconstruction and localization processes are based on the global coordinate system generated from these two frames and the determined relative 3D coordinates, progressively deriving the shooting pose for each subsequent frame.

[0049] In this embodiment, step 104 can be implemented using steps 301-308.

[0050] Step 301: Convert the camera coordinate system of the target frame to the global coordinate system; Step 302: Extract and match spherical features from the panoramic image of the target frame and the panoramic image of the secondary key frame to obtain multiple first matching point pairs; Step 303: Project the two-dimensional pixel coordinates of each matching point in each first matching point pair onto the unit sphere of the target frame and the sub-keyframe respectively to obtain the 3D unit direction ray of each matching point in each first matching point pair. Step 304: The ACRANSAC algorithm is used to solve the 3D unit direction ray of each matching point in each first matching point pair to obtain the essential matrix between the target frame and the secondary key frame. Step 305: Perform singular value decomposition on the essential matrix to obtain multiple sets of candidate transformation matrices; each set of candidate transformation matrices includes the rotation matrix and translation vector of the secondary keyframe relative to the target frame. Step 306: Select the candidate transformation matrix that satisfies the positive depth verification constraint as the relative transformation matrix; Step 307: Triangulate each first matching point pair according to the relative transformation matrix to obtain multiple global 3D map points constructed based on the initialization operation; Step 308: Construct a global sparse point cloud library based on multiple global 3D map points built based on the initialization operation.

[0051] Compared to traditional triangulation based on 2D pixel coordinates of planar perspective images, this embodiment, taking into account the equidistant cylindrical projection characteristics of panoramic images, abandons the traditional pinhole camera intrinsic parameter model. It reprojects the 2D feature points of the panoramic image onto a unit sphere, generating 3D unit direction rays and performing triangulation. This effectively eliminates geometric stretching distortion in the polar regions of the panoramic 2D image, ensuring higher accuracy in defect spatial localization.

[0052] In another exemplary embodiment, step 105 above is a process of pose estimation and dynamic construction of sparse 3D point cloud for local temporal frames. This process performs local localization and 3D reconstruction on the target frame and its pre-set number of preceding and following frames (e.g., 10 frames before and after, for a total of 21 frames) to construct a locally refined sparse point cloud, such as... Figure 5 and Figure 6 As shown, the process includes the following steps.

[0053] A1. Cross-frame feature association and 2D-3D constraint establishment: Whenever a new sequence frame is added to the calculation, its spherical feature points are extracted and matched with the local keyframe set (adjacent frames that have been localized). The system searches whether the matched point pairs contain global 3D map points that have been triangulated. If so, the correspondence between the current frame's spherical observation ray (BearingVector) and the global 3D map points is established.

[0054] A2. Robust Pose Estimation (PnP): Based on the generated 2D-3D correspondence, the PnP (Perspective-n-Point) pose estimation algorithm combined with the RANSAC (Random Sample Consistency) kernel is used to calculate the rotation matrix R and translation vector T of the current frame relative to the initial coordinate system (i.e., the global coordinate system mentioned above). During this process, the magnitude of the translation vector is strictly normalized based on the scale of the target frame and the secondary keyframe to ensure scale consistency throughout the reconstructed sequence. This is based on known global 3D map points. Local spherical unit rays that match one-to-one The relative pose of the camera optical center in the current frame is recovered by solving the PnP algorithm, such as... Figure 5 As shown, where, , and Let n be the number of second matching point pairs that exist in the global sparse point cloud database and have corresponding global 3D map points. These second matching point pairs are obtained by extracting and matching spherical features from the panoramic images of the current frame and adjacent frames. , , They are respectively with , and Matching local spherical unit rays.

[0055] A3. Spatial Point Cloud Expansion and Map Maintenance: For feature point pairs that match in the current frame and the local keyframe set but have not yet been triangulated, perform multi-view... Figure 3An angled matrix algorithm is used to calculate the 3D spatial coordinates of the points in the global coordinate system. The newly generated global 3D map points are incrementally added to the global sparse point cloud library as a matching reference for localization of subsequent frames.

[0056] In this embodiment, step 105 can be implemented using steps 401-403.

[0057] Step 401: Extract and match spherical features from the panoramic image of the current frame and the panoramic images of adjacent frames to obtain multiple second matching point pairs; the current frame is the auxiliary frame currently participating in localization, and the adjacent frames are frames that have been localized and are adjacent to the current frame; for auxiliary frames before the target frame, the current frame is obtained sequentially in order of increasing time distance from the target frame, and for auxiliary frames after the target frame, the current frame is obtained sequentially in order of increasing time distance from the secondary key frame; Step 402: Based on the second matching point pair that has a corresponding global 3D map point in the global sparse point cloud library, the PnP algorithm is used to solve the problem and reverse the rotation matrix and translation vector of the current frame relative to the global coordinate system. Step 403: Based on the rotation matrix and translation vector of the current frame relative to the global coordinate system, the second matching point pair that does not have a corresponding global 3D map point in the global sparse point cloud library is triangulated to obtain global 3D map points constructed based on the panoramic image of the current frame, and added to the global sparse point cloud library in an incremental manner.

[0058] In this embodiment, the advantage of bidirectional local extrapolation and mapping centered on the target frame is that a matching relationship exists between the current frame and the set of local keyframes. This local localization algorithm abandons the conventional mode of visual odometry (VO) that sequentially extrapolates from the first frame of the video, and innovatively adopts a PnP localization and triangulation strategy that uses the "optimal matching frame" as an anchor point and expands bidirectionally frame by frame along the video time axis (i.e., past and future directions). The aforementioned PnP solution method is performed directly in the unit spherical coordinate system, minimizing the angular error between the normalized spherical ray and the projected ray of the spatial point, rather than the image plane pixel residual in the traditional PnP algorithm. This effectively avoids the decrease in localization accuracy caused by edge distortion of panoramic images.

[0059] This method has the following significant technical advantages and effects: B1. Fundamentally Suppressing Scale Drift and Pose Distortion: Traditional monocular localization algorithms often accumulate significant translation and rotation errors over time during long-distance sequence processing. In this scenario, if calculations are performed from the starting frame of the video, the accumulated scale drift often leads to a severe decrease in the accuracy of the localization of the defect target frame and the global 3D points when the calculations reach the distant defect target frame, ultimately resulting in the inability to accurately delineate the defect location. Unlike traditional visual SLAM technology, which aims for global trajectory reconstruction, the core task of this invention is to achieve local identification of defect locations across different time periods without performing global 3D reconstruction on the entire lengthy inspection video. Therefore, based on this task orientation, this invention sets the nearest "optimal matching frame" of the defect as the zero point of the basic coordinate system, ensuring that the accumulated calculation error within the core area of ​​interest of the defect is minimized (nearly zero), thus maximizing the accuracy of 3D relocalization of the defect image.

[0060] B2. It only locates the defect in adjacent frames (e.g., extracting 5-10 frames before and after each area), eliminating the invalid 3D mapping calculations for a large number of irrelevant video frames. This greatly reduces the system's memory usage and processing time, meeting the real-time requirements for rapid defect comparison in engineering sites.

[0061] In another exemplary embodiment, step 106 above is a process of defect frame pose relocalization based on local 3D point cloud. After completing the local 3D mapping of the rectified video sequence, a panoramic image of the defect frame with defect bounding boxes is introduced, and cross-period spatial coordinate system alignment is performed. The specific method is as follows: First, the SPHORB features of the panoramic image of the defective frame are extracted, and the KNN (K-Nearest Neighbors) matching algorithm is used to match these features with the set of local keyframes. The matching results are then used to retrieve and determine which feature points in the panoramic image of the defective frame have been triangulated, thus obtaining the known global 3D point coordinates in the current local coordinate system. This establishes a one-to-one correspondence between the local spherical observation ray (2D) of the panoramic image of the defective frame and the local sparse point cloud (3D) in the global sparse point cloud library.

[0062] Furthermore, a PnP algorithm based on minimizing angular error (PnP) combined with the RANSAC algorithm is used to robustly solve the aforementioned 2D-3D correspondence. Finally, the pose transformation matrix (including rotation and translation vectors) of the trans-period defect frame in the newly established global coordinate system is accurately estimated, thus successfully unifying the spatial pose of the initial inspection and subsequent re-inspection to the same coordinate reference.

[0063] In this embodiment, step 106 can be implemented using steps 501-503.

[0064] Step 501: Using the KNN algorithm, spherical features are extracted and matched between the panoramic image of the defect frame and the panoramic images of each frame in the panoramic image sequence to obtain multiple third matching point pairs. Step 502: Select the third matching point pair that contains global 3D map points in the global sparse point cloud library as the fourth matching point pair; Step 503: Based on multiple fourth matching point pairs, the PnP algorithm is used to solve the problem and reverse the pose transformation matrix of the defect frame in the global coordinate system.

[0065] In another exemplary embodiment, step 107 above is a process of depth recovery of the defect center pixel based on multi-view photometric consistency constraint (ZNCC). This process, for user-defined defect annotation boxes, accurately maps the center pixel to the panoramic image of the rectified target frame through the following steps, such as... Figure 7 As shown.

[0066] C1. Construction of candidate observation rays: Based on the spherical projection model, calculate the unit observation ray (Bearing Vector) corresponding to the center pixel of the defect annotation box.

[0067] C2. Depth Space Discretization Sampling: Since the center pixel typically does not belong to the triangulated feature point cloud set and lacks depth information, it is impossible to find the corresponding point solely through the pose transformation matrix. Therefore, within a preset physical depth range (e.g., 0.5 meters to 50 meters), a series of candidate depth values ​​are constructed according to a preset step size (e.g., equal inverse interval sampling). ,in, , , For different candidate depth values, I is the number of candidate depth values.

[0068] C3. Multi-view tangent plane resampling across periods: For each candidate depth value Calculate the simulated three-dimensional coordinates of the ray endpoint in the global coordinate system. , Let be the three-dimensional coordinates of the camera optical center in the global coordinate system for the defective frame. The unit observation ray corresponds to the center pixel of the defect annotation box. Then, using the pose transformation matrix of each frame in the panoramic image sequence, the... The image is projected onto the panoramic image of each frame in the panoramic image sequence (i.e., onto the panoramic 2D plane). To eliminate spherical distortion interference, a pre-defined tangent patch (e.g., 30×30 pixels) is extracted from the 2D plane of each frame, centered on the projection point. This process ensures that the image patches from different viewpoints are free from visual distortion in geometry and are comparable.

[0069] C4. Similarity Measurement and Depth Optimization: The Zero-Mean Normalized Cross-Correlation (ZNCC) algorithm is used to match the original sectional image blocks of the defective frame with the sectional resampled image blocks extracted at each candidate depth. This ZNCC algorithm uses grayscale images, which can effectively suppress interference caused by illumination fluctuations in cross-period inspections. The system calculates the similarity measurement and depth optimization at each depth. The weighted average of ZNCC scores across all frames. Based on the consistency principle of multi-view geometry, when the average score reaches its peak, the corresponding candidate depth... This is the actual physical depth of the center pixel.

[0070] C5. Pixel-level position mapping: Select the optimal depth value with the highest weighted average. And combined with the specific keyframe (optimal view) or target frame with the highest ZNCC score in the corresponding frame, the result will be determined by... The determined global 3D coordinates are back-projected onto the panoramic image of that frame, thereby completing the automated and accurate identification of defective parts in the rectified image.

[0071] In another exemplary embodiment, step 107 above may be implemented using steps 601-605.

[0072] Step 601: Generate multiple candidate depth values ​​for the center pixel of the defect annotation box; Step 602: For each candidate depth value, calculate the simulated 3D coordinates of the center pixel of the defect annotation box in the global coordinate system based on the pose transformation matrix of the defect frame in the global coordinate system. Step 603: Project the simulated 3D coordinates corresponding to each candidate depth value onto the panoramic image of each frame in the panoramic image sequence, and calculate the weighted average of the ZNCC scores corresponding to each candidate depth value. Step 604: Select the candidate depth value corresponding to the optimal weighted average value as the optimal depth value; Step 605: Obtain the projection position of the simulated 3D coordinates corresponding to the optimal depth value onto the panoramic view of the target frame as the corresponding position of the center pixel of the defect annotation box in the panoramic view of the target frame.

[0073] In another exemplary embodiment, the target frame in step 605 above can also be replaced with a specific key frame, which is the frame in the panoramic image sequence with the highest ZNCC score calculated at the optimal depth value.

[0074] Based on the same inventive concept, this application also provides an automatic relocation and matching device for cross-period inspection panoramic images to implement the automatic relocation and matching method for cross-period inspection panoramic images described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the automatic relocation and matching device for cross-period inspection panoramic images provided below can be found in the limitations of the automatic relocation and matching method for cross-period inspection panoramic images described above, and will not be repeated here.

[0075] In one exemplary embodiment, an automatic relocation and matching device for cross-period inspection panoramic images is provided, comprising: The inspection video acquisition module is used to acquire inspection videos after rectification. The target frame selection module is used to select a panoramic image of a target frame that matches the panoramic image of the defect frame in the inspection video; the panoramic image of the defect frame is a panoramic image with defect annotation boxes before rectification. The panoramic image sequence construction module is used to select panoramic images of a predicted number of frames before and after the target frame in the inspection video, with the target frame as the center, and form a panoramic image sequence. The initialization operation module is used to perform 3D mapping initialization operations based on the panoramic image of the target frame and the panoramic image of the secondary key frame, and to construct a global coordinate system and a global sparse point cloud library; the secondary key frame is the frame following the target frame. The global map dynamic update module is used to match and triangulate the panoramic images of all auxiliary frames in the panoramic image sequence based on the global sparse point cloud library, and add the global 3D map points constructed based on the panoramic images of each auxiliary frame to the global sparse point cloud library; the auxiliary frames are the frames in the panoramic image sequence other than the target frame and the secondary key frame. The defect frame matching module is used to match and solve the panoramic image of the defect frame based on the global sparse point cloud library, and obtain the pose transformation matrix of the defect frame in the global coordinate system. The mapping module is used to map the center pixel of the defect annotation box in the panoramic image of the defect frame to the panoramic image of the target frame using the ZNCC algorithm, based on the pose transformation matrix of the defect frame in the global coordinate system, and to obtain the corresponding position of the center pixel of the defect annotation box in the panoramic image of the target frame.

[0076] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 8As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements an automatic relocation and matching method for cross-period inspection panoramic images.

[0077] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0078] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0079] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0080] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0081] In an exemplary embodiment, to verify the effectiveness of the technical solution of this application, this embodiment selects a panoramic inspection video of a large reservoir project taken over multiple periods for verification. First, the "defect frame" with the original defect annotation is input into the rectified video stream. Using the SPHORB feature and KNN matching strategy, the single panoramic image with the highest visual overlap is automatically retrieved as the "optimal matching frame" (e.g., ...). Figure 3 As shown in the figure, the large number of identical feature points extracted between the two objectively verifies the high consistency of their spatial poses. Subsequently, the system uses this optimal matching frame as the temporal anchor point to start a bidirectional local mapping algorithm (such as...) for adjacent frames before and after. Figure 6As shown in the figure, the camera optical center pose of local keyframes was successfully calculated frame by frame. This mechanism effectively eliminates the cumulative scale drift error caused by long global sequence calculations, ensuring the absolute accuracy of the spatial coordinate system of the defect core region from the bottom layer.

[0082] After completing the local high-precision relocation, the system performs depth traversal and precise reprojection based on multi-view constraints for the center point of the original defect region. The system discretizes the depth along the spherical observation ray corresponding to the center point, maps candidate 3D points backward to the local target frame set, and extracts tangent plane image patches for ZNCC photometric similarity matching to determine the optimal depth. The results of the automated mapping are as follows: Figure 7 As shown: Figure 7 The yellow dot set distributed along the epipolar line represents candidate search locations that strictly adhere to polar geometry constraints; the white box area indicates the perspective-corrected tangent image block for ZNCC score matching; and the red circle precisely identifies the final matching point calculated from the highest ZNCC score. These results intuitively and objectively demonstrate that this application can automatically and accurately trace and identify the spatial source of defects in complex inter-period scenarios without manual intervention, completely solving the pain point of difficult inter-period matching in traditional inspections.

[0083] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0084] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0085] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0086] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An automatic relocation and matching method for panoramic images from cross-period inspections, characterized in that, include: Obtain the inspection video after rectification; Using a spherical feature extraction algorithm, a panoramic image of a target frame that matches the panoramic image of the defect frame is selected from the inspection video; the panoramic image of the defect frame is a panoramic image with defect annotation boxes before rectification. Centered on the target frame, panoramic images of the predicted number of frames before and after the target frame are selected in the inspection video to form a panoramic image sequence. Based on the panoramic view of the target frame and the panoramic view of the secondary key frame, perform the initialization operation of 3D mapping, and construct a global coordinate system and a global sparse point cloud library; the secondary key frame is the frame following the target frame. Based on the global sparse point cloud library, the panoramic images of all auxiliary frames in the panoramic image sequence are matched and triangulated, and the global 3D map points constructed based on the panoramic images of each auxiliary frame are added to the global sparse point cloud library. Auxiliary frames are the frames in the panoramic sequence other than the target frame and the secondary key frame; Based on the global sparse point cloud library, the panoramic image of the defect frame is matched and solved to obtain the pose transformation matrix of the defect frame in the global coordinate system. Based on the pose transformation matrix of the defect frame in the global coordinate system, the ZNCC algorithm is used to map the center pixel of the defect annotation box in the panoramic image of the defect frame to the panoramic image of the target frame, thereby obtaining the corresponding position of the center pixel of the defect annotation box in the panoramic image of the target frame.

2. The automatic relocation and matching method for panoramic images of cross-period inspections according to claim 1, characterized in that, Using a spherical feature extraction algorithm, a panoramic image of a target frame that matches the panoramic image of the defect frame is selected from the inspection video. Specifically, this includes: Using a spherical feature extraction algorithm, spherical feature points are extracted from the panoramic view of each frame in the inspection video and the panoramic view of the defect frame, respectively. Based on the spherical feature points of the panoramic images of each frame in the inspection video and the panoramic images of the defect frames, the KNN matching algorithm is used to match feature points, and the number of matching point pairs between the panoramic images of the defect frames and the panoramic images of each frame in the inspection video is calculated respectively. The panoramic image of the frame with the largest number of matching point pairs in the inspection video is selected as the panoramic image of the target frame.

3. The automatic relocation and matching method for panoramic images of cross-period inspections according to claim 1, characterized in that, Based on the panoramic images of the target frame and the secondary keyframes, perform 3D mapping initialization operations, constructing a global coordinate system and a global sparse point cloud library, specifically including: Convert the camera coordinate system of the target frame to the global coordinate system; Spherical features are extracted and matched from the panoramic images of the target frame and the secondary key frame to obtain multiple first matching point pairs; The two-dimensional pixel coordinates of each matching point in each first matching point pair are projected onto the unit spheres of the target frame and the secondary key frame, respectively, to obtain the 3D unit direction ray of each matching point in each first matching point pair; The ACRANSAC algorithm is used to solve the essential matrix between the target frame and the secondary key frame by using the 3D unit direction ray of each matching point in each first matching point pair. Singular value decomposition is performed on the essential matrix to obtain multiple sets of candidate transformation matrices; each set of candidate transformation matrices includes a rotation matrix and a translation vector of the secondary keyframe relative to the target frame. Candidate transformation matrices that satisfy the positive depth verification constraint are selected as relative transformation matrices; Triangulation is performed on each first matching point pair based on the relative transformation matrix to obtain multiple global 3D map points constructed based on the initialization operation; A global sparse point cloud is constructed based on multiple global 3D map points built from initialization operations.

4. The automatic relocation and matching method for panoramic images of cross-period inspections according to claim 1, characterized in that, Based on the global sparse point cloud library, the panoramic images of all auxiliary frames in the panoramic image sequence are matched and triangulated. The global 3D map points constructed based on the panoramic image of each auxiliary frame are then added to the global sparse point cloud library. Specifically, this includes: Spherical features are extracted and matched between the panoramic images of the current frame and the panoramic images of adjacent frames to obtain multiple second matching point pairs; the current frame is the auxiliary frame currently participating in the localization, and the adjacent frames are frames that have been localized and are adjacent to the current frame; for auxiliary frames before the target frame, the current frame is obtained in order of increasing time distance from the target frame, and for auxiliary frames after the target frame, the current frame is obtained in order of increasing time distance from the secondary key frame; Based on the second matching point pair that has a corresponding global 3D map point in the global sparse point cloud library, the PnP algorithm is used to solve the problem and reverse the rotation matrix and translation vector of the current frame relative to the global coordinate system. Based on the rotation matrix and translation vector of the current frame relative to the global coordinate system, the second matching point pair that does not have a corresponding global 3D map point in the global sparse point cloud library is triangulated to obtain global 3D map points constructed based on the panoramic image of the current frame, and added to the global sparse point cloud library in an incremental manner.

5. The automatic relocation and matching method for panoramic images of cross-period inspections according to claim 1, characterized in that, Based on the global sparse point cloud library, the panoramic image of the defect frame is matched and solved to obtain the pose transformation matrix of the defect frame in the global coordinate system, specifically including: The KNN algorithm is used to extract and match spherical features of the panoramic image of the defect frame and the panoramic images of each frame in the panoramic image sequence to obtain multiple third matching point pairs. The third matching point pair that contains global 3D map points in the global sparse point cloud database is selected as the fourth matching point pair; Based on multiple fourth matching point pairs, the PnP algorithm is used to solve the problem and reverse the position transformation matrix of the defect frame in the global coordinate system.

6. The automatic relocation and matching method for panoramic images of cross-period inspections according to claim 1, characterized in that, Based on the pose transformation matrix of the defect frame in the global coordinate system, the ZNCC algorithm is used to map the center pixel of the defect annotation box in the panoramic image of the defect frame to the panoramic image of the target frame, obtaining the corresponding position of the center pixel of the defect annotation box in the panoramic image of the target frame. Specifically, this includes: Generate multiple candidate depth values ​​for the center pixel of the defect annotation box; For each candidate depth value, the simulated 3D coordinates of the center pixel of the defect annotation box in the global coordinate system are calculated based on the pose transformation matrix of the defect frame in the global coordinate system. The simulated 3D coordinates corresponding to each candidate depth value are projected onto the panoramic image of each frame in the panoramic image sequence, and the weighted average of the ZNCC score corresponding to each candidate depth value is calculated. The candidate depth value corresponding to the optimal weighted average value is selected as the optimal depth value; The simulated 3D coordinates corresponding to the optimal depth value are projected onto the panoramic view of the target frame and used as the corresponding position of the center pixel of the defect annotation box in the panoramic view of the target frame.

7. An automatic repositioning and matching device for panoramic images of cross-period inspections, characterized in that, The automatic relocation and matching device for the cross-period inspection panoramic image is applied to the automatic relocation and matching method for the cross-period inspection panoramic image according to any one of claims 1-6, wherein the automatic relocation and matching device for the cross-period inspection panoramic image comprises: The inspection video acquisition module is used to acquire inspection videos after rectification. The target frame selection module is used to select a panoramic image of a target frame that matches the panoramic image of the defect frame in the inspection video; the panoramic image of the defect frame is a panoramic image with defect annotation boxes before rectification. The panoramic image sequence construction module is used to select panoramic images of a predicted number of frames before and after the target frame in the inspection video, with the target frame as the center, and form a panoramic image sequence. The initialization operation module is used to perform 3D mapping initialization operations based on the panoramic image of the target frame and the panoramic image of the secondary key frame, and to construct a global coordinate system and a global sparse point cloud library; the secondary key frame is the frame following the target frame. The global map dynamic update module is used to match and triangulate the panoramic images of all auxiliary frames in the panoramic image sequence based on the global sparse point cloud library, and add the global 3D map points constructed based on the panoramic images of each auxiliary frame to the global sparse point cloud library; the auxiliary frames are the frames in the panoramic image sequence other than the target frame and the secondary key frame. The defect frame matching module is used to match and solve the panoramic image of the defect frame based on the global sparse point cloud library, and obtain the pose transformation matrix of the defect frame in the global coordinate system. The mapping module is used to map the center pixel of the defect annotation box in the panoramic image of the defect frame to the panoramic image of the target frame using the ZNCC algorithm, based on the pose transformation matrix of the defect frame in the global coordinate system, and to obtain the corresponding position of the center pixel of the defect annotation box in the panoramic image of the target frame.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the automatic relocation and matching method for panoramic images of cross-period inspections as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the automatic relocation and matching method for panoramic images of cross-period inspections as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the automatic relocation and matching method for panoramic images of cross-period inspections as described in any one of claims 1-6.

Images