A method and system for outdoor engineering detection and supervision based on panoramic images and outdoor detection trajectories

Abstract

The present application belongs to the field of engineering detection technology, and specifically relates to a method and system for outdoor engineering detection and supervision based on panoramic images and outdoor detection trajectories, which automatically records 360-degree visual information of the detection scene by using a safety helmet equipped with a panoramic camera, and synchronously tracks and saves the advancing route and position information, realizes full-process visual management of the detection process, and guarantees the full-process traceability of the construction engineering site quality detection activities. Meanwhile, the recorded position information is used to restore the detection site view, and an interactive 360-degree viewing function is provided, so that the user can observe the detection site in detail from any viewing angle. The solution provided by the present application significantly improves the efficiency and accuracy of outdoor engineering detection, guarantees the authenticity and fairness of the site detection data, provides a more intuitive and complete site detection experience for the user, and simplifies the data management and analysis process, and improves the decision support effect.

Description

Technical Field

[0001] This invention belongs to the field of engineering inspection technology, specifically relating to a method and system for outdoor engineering inspection and supervision based on panoramic imaging and outdoor inspection trajectories. Background Technology

[0002] In the current field of outdoor engineering inspection, traditional inspection methods typically rely on manual inspections and static image recording. Inspectors record the site conditions by taking photos and videos with handheld cameras or mobile devices, and manually record their routes and location information. While this method provides some visual documentation, it has many limitations in practical applications.

[0003] First, traditional manual inspection methods struggle to achieve 360-degree panoramic coverage of the entire inspection area, potentially overlooking crucial details and affecting the comprehensiveness and accuracy of the results. Second, the lack of effective path tracking methods makes it difficult to accurately reproduce the movement of inspection personnel, complicating subsequent data analysis and problem localization. Furthermore, the labeling and explanation of key areas are typically done retrospectively, which is not only inefficient but also prone to information bias.

[0004] Furthermore, existing technologies are inadequate in data processing and decision support. Because there is no systematic platform to manage and analyze the collected information, decision-makers often need to spend a significant amount of time organizing and understanding this scattered data, thus reducing work efficiency.

[0005] The main problem with the aforementioned existing technology is that it cannot efficiently integrate panoramic images with inspection path information, resulting in incomplete and inaccurate data recording during the inspection process, which affects the quality of subsequent data analysis and the effectiveness of decision-making. Summary of the Invention

[0006] The purpose of this invention is to provide a method and system for outdoor engineering inspection and supervision based on panoramic images and outdoor inspection trajectories, which significantly improves the efficiency and accuracy of outdoor engineering inspection, provides users with a more intuitive and complete on-site inspection experience, simplifies data management and analysis processes, and improves the effectiveness of decision support, thereby solving the problems mentioned in the background art.

[0007] To achieve the above objectives, this invention proposes a method for outdoor engineering inspection and monitoring based on panoramic imagery and outdoor detection trajectories, comprising the following steps:

[0008] Step 1: Collect panoramic images of the site. Use a safety helmet equipped with a panoramic camera to automatically record 360-degree visual information of the outdoor inspection scene and record the inspection path. Track and save the route and location information synchronously as the inspection personnel move. At the same time, use a mobile APP to attach high-definition photos of the inspection location and content to the outdoor inspection trajectory and panoramic images.

[0009] Step 2: Based on the recorded location information, reconstruct the scene and construct a realistic and complete view of the detection site using panoramic images. The resulting view allows for 360-degree viewing, enabling users to rotate, zoom in, or zoom out to observe from any angle. Clicking on a high-resolution photo allows viewing the corresponding detection location and on-site detection content.

[0010] Step 3: Based on the perspective, mark key areas, annotate the areas that need attention or have questions in the image, and use the marked key areas for interactive operations. Users can select the marked points to obtain detection information or additional explanations.

[0011] Step 4: Based on the interaction results of the interactive operation, analyze the detection data, process the collected data information to support decision-making, and store and manage the information after data processing.

[0012] Preferably, the acquisition of panoramic images of the site involves automatically recording 360-degree visual information of the outdoor inspection scene using a safety helmet equipped with a panoramic camera, including:

[0013] Based on the acquisition of panoramic images of the site, image quality is optimized. The panoramic camera built into the safety helmet automatically adjusts the exposure and color balance to ensure the accuracy of visual information. The adjusted image is segmented and each segment is assigned a unique identifier IDi according to its position in the 360-degree view.

[0014] Based on the unique identifier IDi, perform coordinate transformation on each image segment using the formula: new_Xv=Xv*cos(θ)-Yv*sin(θ), new_Yv=Xv*sin(θ)+Yv*cos(θ), where Xv,Yv represent the original view coordinate points, new_Xv,new_Yv are the new view coordinate points after transformation, and θ represents the rotation angle;

[0015] After completing the coordinate transformation, edge fusion is performed between adjacent image segments. The average pixel value is calculated using the overlapping area and used as the boundary value of the new image segment.

[0016] Based on the edge blending, a complete panoramic image file is generated.

[0017] Preferably, the recording of the inspection path, which synchronously tracks and saves the travel route and location information as the inspection personnel move, includes:

[0018] Based on the recorded inspection path, the location information is made more accurate. As the inspection personnel move, the current location coordinates (Xp, Yp) are captured by the built-in positioning device and associated with the timestamp T.

[0019] Calculate the distance Dp between two adjacent points based on the location coordinates (Xp, Yp) and timestamp T using the formula: Dp = sqrt((Xp2-Xp1)^2+(Yp2-Yp1)^2), where (Xp1, Yp1) and (Xp2, Yp2) represent the location coordinates of two consecutive captures;

[0020] After completing the distance Dp calculation, the accumulated distance values ​​are used to construct the inspection path map. For each newly captured location coordinate (Xp, Yp), the path map is updated to reflect the latest travel direction, and the location is added to the existing path sequence.

[0021] After the path graph is updated, path optimization is implemented. For duplicate or unnecessary backtracking segments on the path, a simplified algorithm is used, and a threshold Vp is set. When the shortest distance between two paths is less than Vp, the two paths are merged.

[0022] Preferably, the step of reconstructing the scene based on the recorded location information and constructing a realistic and complete view of the detection site using panoramic images includes:

[0023] Based on the location coordinates (Xp, Yp) and timestamp T, the corresponding panoramic image file is located, and the location information is matched with the unique identifier IDi in the panoramic image.

[0024] After acquiring the corresponding panoramic image, apply the coordinate transformation formula:

[0025] new_Xv = Xv*cos(θ) - Yv*sin(θ), new_Yv = Xv*sin(θ) + Yv*cos(θ), converting the current position coordinates (Xp, Yp) into new coordinate points (new_Xv, new_Yv) relative to the center of the panoramic image, thereby determining the viewing direction, where θ represents the rotation angle;

[0026] After determining the viewing direction, integrate adjacent panoramic images. For two consecutive position coordinates (Xp1,Yp1) and (Xp2,Yp2), calculate the distance Dp between them using the formula: Dp=sqrt((Xp2-Xp1)^2+(Yp2-Yp1)^2), and adjust the transition smoothness between adjacent images according to Dp.

[0027] Based on the completion of panoramic image integration, a multi-view interactive 3D view is generated. For each newly added panoramic image segment, the dataset of the 3D view is updated, and the view layout is adjusted according to the optimized inspection path. A threshold Vp is set to control the degree of path simplification. When the shortest distance between two paths is less than Vp, the two paths are merged.

[0028] Preferably, the step of achieving 360-degree viewing based on the formed detection site view, allowing users to rotate, zoom in, or zoom out the image to observe from any angle, includes:

[0029] After generating a multi-view interactive 3D view, a user input interface is established to receive user operation commands. When the user selects to rotate the view, the system adjusts the view according to the direction and angle α specified by the user, using the coordinate transformation formula: new_Xv=Xv*cos(α)-Yv*sin(α), new_Yv=Xv*sin(α)+Yv*cos(α), where (Xv,Yv) represents the original coordinate point, new_Xv and new_Yv are the new coordinate points after transformation, and α is the rotation angle specified by the user.

[0030] After the view rotation is completed, if the user wants to zoom in on a specific area, the system will recalculate the position of all pixels in that area based on the coordinates of the center point (Xc, Yc) selected by the user and the zoom factor F, and apply the scaling formula: scaled_Xv=(Xv-Xc)*F+Xc, scaled_Yv=(Yv-Yc)*F+Yc;

[0031] After the user completes the zoom-in or zoom-out operation, the system automatically adjusts the smoothness of the transition between adjacent image segments. For two consecutive position coordinates (Xp1,Yp1) and (Xp2,Yp2), the system calculates the distance Dp between them and dynamically adjusts the overlapping area between adjacent images based on Dp.

[0032] Based on all operations, users can set a marker at any point of interest. The system records the offset (ΔXv, ΔYv) of the marker relative to the current view center point. When the user revisits the view or shares it with other users, the system can accurately reproduce the marker position, allowing the user to directly navigate to the area of ​​interest.

[0033] Preferably, marking key areas based on the viewpoint, and annotating areas in the image that require attention or raise questions, includes:

[0034] The user first selects the key area to be marked, and the system records the center coordinates (Xm, Ym) and the radius R of the area. (Xm, Ym) represents the center position of the marked area, and R is the size range of the marked area defined by the user.

[0035] After selecting the marking area, in order to ensure that the marking is not accidentally covered or removed, the system calculates a unique identifier IDm. This IDm is composed of the center coordinates (Xm,Ym), radius R and current timestamp T: IDm = hash(Xm+Ym+R+T), where hash represents a hash function used to generate a fixed-length string as a unique identifier;

[0036] The system uses IDm as an index to embed visual markers in the image and associates the position of the markers with the coordinates in the original view. When the user performs zoom in or zoom out, the system automatically adjusts the display position of the markers according to the scaling formulas: scaled_Xm=(Xm-Xc)*F+Xc, scaled_Ym=(Ym-Yc)*F+Yc, to ensure that the markers are always in the correct area. (Xc,Yc) are the coordinates of the center point of the view, and F is the magnification factor.

[0037] After all operations are completed, if the user wants to add text descriptions, they can specify the starting coordinates (Xt, Yt) of a text box near the marked position. The system will check whether the position overlaps with other marks and calculate the minimum distance between the new text box and the nearest mark using the distance formula Dm=sqrt((Xt-Xm)^2+(Yt-Ym)^2). If Dm is less than the preset safety distance Sm, the user will be prompted to reselect the position.

[0038] Preferably, the interactive operation using marked key areas allows users to select marked points to obtain detection information or additional instructions, including:

[0039] When a user clicks on any marker, the system retrieves the associated detailed information based on the marker's unique identifier IDm. IDm is composed of the center coordinates (Xm, Ym), radius R, and timestamp T: IDm = hash(Xm + Ym + R + T), ensuring that each marker is linked to specific detection information.

[0040] Once a marker point is selected, the system searches for and loads the pre-stored relevant data records based on the point's IDm. At the same time, it calculates the marker point's display position (scaled_Xm, scaled_Ym) in the current view using the formulas: scaled_Xm = (Xm - Xc) * F + Xc, scaled_Ym = (Ym - Yc) * F + Yc, where (Xc, Yc) are the coordinates of the view's center point, and F is the magnification factor.

[0041] After loading the detailed information, the system creates a temporary interactive window that allows users to browse or scroll through additional information;

[0042] After a user completes viewing the detailed information of a certain marker, the system records the user's access behavior and updates the access log. For each interaction, the system generates a new timestamp Tn and saves it in association with IDm.

[0043] Preferably, the interaction results based on interactive operations, analyzing detection data, and processing the collected data information to support decision-making include:

[0044] Collect access logs for all marked points. The system retrieves the corresponding access record based on the unique identifier IDm of each marked point. IDm is composed of the center coordinates (Xm, Ym), radius R, and timestamp T: IDm = hash(Xm + Ym + R + T). The system also calculates the number of visits N and the average dwell time A under each IDm, using the formula: N = sum(number of visits), A = total_time / N, where total_time is the total time spent visiting the marked point.

[0045] The importance of the marker is assessed based on the number of visits N and the average dwell time A. The importance score Im is defined as: Im = w1*N + w2*A, where w1 and w2 are weighting coefficients, representing the proportion of the number of visits and the dwell time in the importance score, respectively.

[0046] For markers with high importance scores, extract related details and then apply content analysis rules to identify potential problem types or actions that need to be taken.

[0047] The analysis results are compiled into a report format and provided to decision-makers for reference. The report not only lists the location of high-scoring markers, related issues and their severity, but also suggests corresponding solutions.

[0048] Preferably, the storage and management of information after data processing includes:

[0049] For each marker point and its associated access logs, details, and importance score Im, the system creates a comprehensive data packet containing the marker point IDm, location coordinates (Xm, Ym), radius R, timestamp T, number of visits N, average dwell time A, and importance score Im.

[0050] The system calculates a checksum Cm based on the content of the comprehensive data packet using the formula: Cm = checksum(IDm + Xm + Ym + R + T + N + A + Im), where checksum represents a hash function used to generate a fixed-length string as a unique identifier for the data packet;

[0051] After completing packet creation and checksum calculation, the system selects a suitable data storage location and organizes the data according to predefined classification rules. For each newly created packet, its category K is determined, and a specific storage path P is allocated based on category K.

[0052] Based on ensuring the secure storage of all data, an indexing mechanism is established to support efficient retrieval. The system generates an index entry for each data packet, including but not limited to IDm, location coordinates (Xm, Ym), category K, storage path P, and tracking number Zm. When a user needs to query specific information, they can find the corresponding index entry by inputting relevant parameters, and then directly access the target data.

[0053] On the other hand, this invention proposes a system for outdoor engineering inspection and monitoring based on panoramic imagery and outdoor detection trajectories, comprising:

[0054] The panoramic imaging and path recording module is used to collect panoramic images of the site. It uses a safety helmet equipped with a panoramic camera to automatically record 360-degree visual information of the outdoor inspection scene and record the inspection path. It tracks and saves the route and location information synchronously as the inspection personnel move.

[0055] The panoramic view restoration and interaction module is used to restore the scene based on the recorded location information, construct a real and complete view of the detection site using panoramic images, and realize 360-degree viewing based on the formed detection site view, allowing users to rotate, zoom in or zoom out of the image to observe any angle.

[0056] The key area marking and interaction module is used to mark key areas based on the viewpoint, mark places that need attention or have questions in the image, and perform interactive operations using the marked key areas. Users can select the marked points to obtain detection information or additional explanations.

[0057] The data analysis and decision support module is used to analyze and detect data based on the interactive results of interactive operations, process the collected data information to support decision-making, and store and manage information after data processing.

[0058] Technical effects and advantages of the present invention: The method and system for outdoor engineering inspection and monitoring based on panoramic imagery and outdoor detection trajectories proposed in this invention have the following advantages compared with the prior art:

[0059] This invention automatically records 360-degree visual information of the inspection scene using a safety helmet equipped with a panoramic camera, and simultaneously tracks and saves the travel route and location information, achieving full-process visual management of the inspection. Simultaneously, it reconstructs the inspection site view using the recorded location information and provides an interactive 360-degree viewing function, ensuring users can observe the inspection site in detail from any angle. Furthermore, by marking and annotating key areas, users can easily obtain relevant inspection information or additional explanations, enhancing the readability and usability of the data. Finally, by analyzing the results of the interactive operation, this method can effectively support decision-making and securely store and manage all information, providing users with a structured and efficient workflow. In summary, the solution provided by this invention significantly improves the efficiency and accuracy of outdoor engineering inspection, provides users with a more intuitive and complete inspection site experience, simplifies data management and analysis processes, and enhances the effectiveness of decision support. Attached Figure Description

[0060] Figure 1 This is a flowchart of the outdoor engineering inspection method based on panoramic images and outdoor inspection trajectories according to the present invention;

[0061] Figure 2 This is a block diagram of the outdoor engineering inspection system based on panoramic imaging and outdoor inspection trajectory of the present invention. Detailed Implementation

[0062] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The specific embodiments described herein are merely used to explain the present invention and are not intended to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0063] This invention provides, for example Figure 1 The method for outdoor engineering inspection and supervision based on panoramic imagery and outdoor inspection trajectories, as shown, includes the following steps:

[0064] Step 1: Acquire panoramic images of the site. A safety helmet equipped with a panoramic camera automatically records 360-degree visual information of the outdoor inspection scene and tracks the inspection path. It moves with the inspection personnel, synchronously recording and saving the route and location information. Simultaneously, a mobile app overlays high-resolution photos of the inspection location and content onto the outdoor inspection track and panoramic imagery. Specifically, this includes:

[0065] Based on the acquired panoramic imagery, image quality was optimized. The panoramic camera built into the safety helmet automatically adjusted exposure and color balance to ensure the accuracy of visual information. This not only improved image quality but also reduced misjudgments caused by changes in lighting conditions or color distortion. The adjusted image was then segmented, with each segment assigned a unique identifier IDi based on its position in the 360-degree view; this ensured precise identification and management of each segment. A coordinate transformation formula was used to ensure seamless stitching of the image segments, enhancing the overall coherence of the panoramic imagery.

[0066] Based on the unique identifier IDi, perform coordinate transformation on each image segment using the formula: new_Xv=Xv*cos(θ)-Yv*sin(θ), new_Yv=Xv*sin(θ)+Yv*cos(θ), where Xv,Yv represent the original view coordinate points, new_Xv,new_Yv are the new view coordinate points after transformation, and θ represents the rotation angle;

[0067] After coordinate transformation, edge fusion is performed between adjacent image segments. The average pixel value is calculated using the overlapping area and used as the boundary value of the new image segment. This process further improves the consistency and realism of the overall image and eliminates visible seams between different segments.

[0068] Based on the edge blending, a complete panoramic image file is generated. The final generated panoramic image file integrates all processed image segments, maintaining the coherence and integrity of the original scene, and providing users with a realistic and complete view of the inspection site.

[0069] Based on the recorded inspection path, the system achieves precise location information. As the inspection personnel move, the system captures the current location coordinates (Xp, Yp) through the built-in positioning device and associates them with the timestamp T. This enables the synchronous tracking and storage of the travel route and location information, ensuring that each step has accurate time and space markings.

[0070] Based on the location coordinates (Xp, Yp) and timestamp T, the distance Dp between two adjacent points is calculated using the formula: Dp = sqrt((Xp2 - Xp1)^2 + (Yp2 - Yp1)^2), where (Xp1, Yp1) and (Xp2, Yp2) represent the location coordinates captured in two consecutive captures. This provides the basic data for subsequent route drawing. The accumulated distance values ​​are used to construct an inspection path map, dynamically reflecting the latest direction of travel, and new locations are added to the path sequence to ensure real-time updates of the path map.

[0071] After completing the distance Dp calculation, the accumulated distance values ​​are used to construct the inspection path map. For each newly captured location coordinate (Xp, Yp), the path map is updated to reflect the latest travel direction, and the location is added to the existing path sequence.

[0072] After updating the path map, path optimization is implemented. For duplicate or unnecessary backtracking segments on the path, a simplification algorithm is used, with a threshold Vp set. When the shortest distance between two path segments is less than Vp, these two path segments are merged. The optimized path is more concise and clear, effectively showing the actual patrol range of the inspection personnel, while reducing path redundancy and improving the efficiency of data analysis.

[0073] By applying the aforementioned technical means, this invention effectively solves the problems of incomplete image coverage and inaccurate path recording in traditional detection methods, providing a highly efficient, accurate, and intuitive outdoor engineering inspection solution. It not only improves the efficiency and accuracy of inspection work but also lays a solid foundation for subsequent data analysis and decision support. Users can easily obtain high-quality panoramic images and precise inspection path information, thereby better understanding and handling on-site conditions and making more informed decisions.

[0074] Step Two: Based on the recorded location information, reconstruct the scene and construct a realistic and complete view of the detection site using panoramic imagery. This view allows for 360-degree viewing, enabling users to rotate, zoom in, or zoom out to observe from any angle. Clicking on a high-resolution photo allows viewing the corresponding detection location and on-site detection content. Specifically, this includes:

[0075] Based on the location coordinates (Xp, Yp) and timestamp T, the corresponding panoramic image file is located. By matching the location information with the unique identifier IDi in the panoramic image, it is ensured that the selected image is a true record taken by the inspector at that location, thus enhancing the accuracy and reliability of the data.

[0076] After acquiring the corresponding panoramic image, apply the coordinate transformation formula:

[0077] new_Xv = Xv*cos(θ) - Yv*sin(θ), new_Yv = Xv*sin(θ) + Yv*cos(θ) converts the current position coordinates (Xp, Yp) into new coordinate points (new_Xv, new_Yv) relative to the center of the panoramic image, thereby determining the viewing direction. θ represents the rotation angle. This process ensures the correct orientation of the view, enabling users to observe the inspection site from the correct angle and improving the realism of the user experience.

[0078] After determining the viewing direction, adjacent panoramic images are integrated. For two consecutive position coordinates (Xp1, Yp1) and (Xp2, Yp2), the distance Dp between them is calculated using the formula: Dp = sqrt((Xp2-Xp1)^2 + (Yp2-Yp1)^2). The smoothness of the transition between adjacent images is then adjusted based on Dp. This step ensures seamless stitching even between images taken from different positions, reducing visual discontinuities and improving the consistency and coherence of the overall image.

[0079] Based on the integrated panoramic images, a multi-view interactive 3D view is generated. For each newly added panoramic image segment, the dataset of the 3D view is updated, and the view layout is adjusted according to the optimized inspection path. A threshold Vp is set to control the degree of path simplification; when the shortest distance between two path segments is less than Vp, these two path segments are merged. Setting the threshold Vp to control the degree of path simplification further optimizes the view structure, making the display of the entire inspection site more concise and clear.

[0080] After generating a multi-view interactive 3D view, a user input interface is established to receive user operation commands. When the user selects to rotate the view, the system adjusts the view according to the user-specified direction and angle α, using the coordinate transformation formulas: new_Xv=Xv*cos(α)-Yv*sin(α), new_Yv=Xv*sin(α)+Yv*cos(α), where (Xv,Yv) represents the original coordinate point, new_Xv and new_Yv are the new coordinate points after transformation, and α is the rotation angle specified by the user. This real-time feedback mechanism greatly enhances the user's interactive experience, allowing users to flexibly explore every detail of the inspection site.

[0081] After the view rotation is completed, if the user wants to zoom in on a specific area, the system recalculates the position of all pixels in that area based on the coordinates of the center point (Xc, Yc) selected by the user and the zoom factor F, and applies the scaling formulas: scaled_Xv=(Xv-Xc)*F+Xc, scaled_Yv=(Yv-Yc)*F+Yc; ensuring that the view remains seamless even when zoomed in or out at high magnification, avoiding image fragmentation or overlap issues.

[0082] After the user completes the zoom-in or zoom-out operation, the system automatically adjusts the smoothness of the transition between adjacent image segments. For two consecutive position coordinates (Xp1,Yp1) and (Xp2,Yp2), the system calculates the distance Dp between them and dynamically adjusts the overlapping area between adjacent images based on Dp.

[0083] Based on all operations, users can set a marker at any point of interest. The system records the offset (ΔXv, ΔYv) of this marker relative to the current view center point. When the user revisits the view or shares it with other users, the system can accurately reproduce the marker's position, allowing the user to directly navigate to the area of ​​interest. This not only increases the system's usability but also facilitates subsequent analysis and discussion. Through the application of the above-mentioned technical means, this invention significantly improves the accuracy of outdoor engineering inspection scene reconstruction and the user interaction experience. It not only provides high-quality panoramic images and accurate location information but also achieves smooth user operation and powerful marking functions.

[0084] This also includes: attaching high-resolution photos. As inspection personnel move, a mobile app can attach high-resolution photos of the inspection location and content to the outdoor inspection trajectory and panoramic imagery. Users can take and upload high-resolution photos at any time and associate them with their current geographical location information, thus providing additional visual reference for the location. These high-resolution photos can be embedded into the panoramic imagery, allowing users to access the high-resolution photos and obtain more detailed local observations when viewing the panoramic imagery of the corresponding location.

[0085] Step 3: Based on the viewpoint, mark key areas. Annotate areas in the image that require attention or raise questions. Use these marked key areas for interactive operations; users can select marked points to obtain detection information or additional explanations. Specifically, this includes:

[0086] The user first selects the key area to be marked. The system records the center coordinates (Xm, Ym) and radius R of the area. (Xm, Ym) represents the center position of the marked area, and R is the size range of the marked area defined by the user. This not only allows users to accurately point out areas that require attention or have questions, but also provides accurate geographic reference for subsequent information association.

[0087] After selecting the marking area, to ensure that the markings are not accidentally overwritten or removed, the system calculates a unique identifier IDm. This IDm is composed of the center coordinates (Xm, Ym), radius R, and current timestamp T: IDm = hash(Xm + Ym + R + T), where hash represents a hash function used to generate a fixed-length string as a unique identifier. This ensures that markings created at different times in the same location can be distinguished, improving data integrity and reliability.

[0088] The system uses IDm as an index to embed visual markers in the image and associates the position of these markers with their coordinates in the original view. When the user zooms in or out, the system automatically adjusts the display position of the markers according to the scaling formulas: scaled_Xm = (Xm - Xc) * F + Xc, scaled_Ym = (Ym - Yc) * F + Yc, ensuring that the markers are always in the correct area. (Xc, Yc) are the coordinates of the view's center point, and F is the magnification factor. When the user wants to add text descriptions, the system automatically checks whether the new text box overlaps with other markers by calculating the minimum distance Dm and comparing it with the preset safety spacing Sm. This process effectively prevents confusion between different markers, ensuring that each marker and its description are clearly visible and improving the effectiveness of information delivery.

[0089] After all operations are completed, if the user wants to add text descriptions, they can specify the starting coordinates (Xt, Yt) of a text box near the marked position. The system will check whether the position overlaps with other marks and calculate the minimum distance between the new text box and the nearest mark using the distance formula Dm=sqrt((Xt-Xm)^2+(Yt-Ym)^2). If Dm is less than the preset safety distance Sm, the user will be prompted to reselect the position.

[0090] The interactive operation utilizes marked key areas, allowing users to select marked points to obtain detection information or additional explanations, including:

[0091] When a user clicks on any marker, the system retrieves the associated detailed information based on the marker's unique identifier IDm. IDm is composed of the center coordinates (Xm, Ym), radius R, and timestamp T: IDm = hash(Xm + Ym + R + T), ensuring that each marker is linked to specific detection information. When a user clicks on any marker, the system can quickly retrieve the associated detailed information based on the marker's unique identifier IDm, ensuring that each interaction corresponds to specific detection information, thus achieving accurate data query and service response.

[0092] Once a marker point is selected, the system searches for and loads the pre-stored relevant data records based on the point's IDm. Simultaneously, it calculates the marker point's display position (scaled_Xm, scaled_Ym) in the current view using the formulas: scaled_Xm = (Xm - Xc) * F + Xc, scaled_Ym = (Ym - Yc) * F + Yc, where (Xc, Yc) are the coordinates of the view's center point, and F is the magnification factor. This instant loading and display method improves the speed and convenience of information retrieval while avoiding interference with the main interface.

[0093] After loading the detailed information, the system creates a temporary interactive window, allowing users to browse or scroll through additional information. Once the user has viewed the details of a specific marker, the system records the user's access behavior, updates the access log, and generates a new timestamp Tn for each interaction, which is then associated with and saved to IDm. This helps track user activity patterns, provide personalized service support, and facilitates subsequent data analysis and auditing.

[0094] In summary, this step significantly enhances the system's application value in outdoor engineering inspection scenarios. It not only allows users to more easily annotate and manage important information, but also greatly improves the user experience through a series of intelligent designs, such as dynamically adjusted markers and a mechanism for adding text descriptions that avoids overlap.

[0095] Step 4: Based on the interaction results, analyze the detection data, process the collected data to support decision-making, and store and manage the information after data processing; specifically including:

[0096] The system collects access logs for all marked points. It retrieves the corresponding access record based on the unique identifier IDm of each marked point. IDm is composed of the center coordinates (Xm, Ym), radius R, and timestamp T: IDm = hash(Xm + Ym + R + T). The system also calculates the number of visits N and the average dwell time A for each IDm using the formulas: N = sum(number of visits), A = total_time / N, where total_time is the total time spent visiting that marked point. The system comprehensively collects access logs for all marked points and retrieves the corresponding access record based on the unique identifier IDm. This ensures a complete record of the interaction behavior at each marked point, providing a detailed data foundation for subsequent analysis.

[0097] The importance of a marker is assessed based on the number of visits N and the average dwell time A. The importance score Im is defined as: Im = w1*N + w2*A, where w1 and w2 are weighting coefficients, representing the proportion of visits and dwell time in the importance score, respectively. This statistical data not only reflects the user's interests but also provides an objective basis for assessing the importance of the marker.

[0098] This scoring mechanism can scientifically assess the importance of each marker point, helping decision-makers quickly identify key areas or potential problems.

[0099] For markers with high importance scores, the system extracts related detailed information and then applies content analysis rules to determine potential problem types or necessary actions. For markers with high scores, the system extracts related detailed information and applies content analysis rules, such as image pattern recognition or text keyword search, to determine potential problem types or necessary actions. This intelligent analysis method improves the accuracy and relevance of problem diagnosis.

[0100] The analysis results are compiled into a report format and provided to decision-makers. The report not only lists the locations of high-scoring markers, related issues and their severity, but also suggests corresponding solutions. This approach enables decision-makers to quickly grasp the situation on-site and formulate effective response strategies.

[0101] For each marker point and its associated access logs, detailed information, and importance score Im, the system creates a comprehensive data packet. The data packet contains the marker point IDm, location coordinates (Xm, Ym), radius R, timestamp T, number of visits N, average dwell time A, and importance score Im. This data is structured for easy management and retrieval.

[0102] The system calculates a checksum Cm based on the content of the aggregated data packet using the formula: Cm = checksum(IDm + Xm + Ym + R + T + N + A + Im), where checksum represents a hash function used to generate a fixed-length string as a unique identifier for the data packet. Calculating the checksum Cm ensures the consistency and integrity of the data packet. Using a hash function to generate a fixed-length string as a unique identifier prevents errors during data transmission or storage, thus guaranteeing data security and reliability.

[0103] After completing packet creation and checksum calculation, the system selects a suitable data storage location and organizes the data according to predefined classification rules. For each newly created packet, its category K is determined, and a specific storage path P is allocated based on category K.

[0104] Based on ensuring the secure storage of all data, an indexing mechanism is established to support efficient retrieval. The system generates an index entry for each data packet, including but not limited to IDm, location coordinates (Xm, Ym), category K, storage path P, and tracking number Zm. When a user needs to query specific information, they can find the corresponding index entry by inputting relevant parameters, and then directly access the target data.

[0105] The system selects appropriate data storage locations based on predefined classification rules and assigns a specific storage path P to each data packet. Simultaneously, it establishes an efficient indexing mechanism, allowing users to quickly find target data by inputting relevant parameters, greatly improving the speed and accuracy of data retrieval.

[0106] In summary, this step significantly enhances the system's data analysis capabilities and decision support functions. It not only achieves accurate recording and statistics of user interaction behavior but also helps decision-makers quickly identify and resolve key issues through intelligent scoring and content analysis methods. Furthermore, structured data packet creation, secure and efficient storage management, and a powerful indexing mechanism ensure the secure preservation and convenient access of all information.

[0107] On the other hand, this invention proposes a system for outdoor engineering inspection and monitoring based on panoramic imagery and outdoor detection trajectories, such as... Figure 2 As shown, it includes: a panoramic image and path recording module, a panoramic view restoration and interaction module, a key area marking and interaction module, and a data analysis and decision support module.

[0108] The panoramic imaging and path recording module is used to collect panoramic images of the site. It uses a safety helmet equipped with a panoramic camera to automatically record 360-degree visual information of the outdoor inspection scene and record the inspection path. It tracks and saves the route and location information synchronously as the inspection personnel move.

[0109] The panoramic view restoration and interaction module is used to restore the scene based on the recorded location information, construct a real and complete view of the detection site using panoramic images, and realize 360-degree viewing based on the formed detection site view, allowing users to rotate, zoom in or zoom out of the image to observe any angle.

[0110] The key area marking and interaction module is used to mark key areas based on the viewpoint, mark places that need attention or have questions in the image, and perform interactive operations using the marked key areas. Users can select the marked points to obtain detection information or additional explanations.

[0111] The data analysis and decision support module is used to analyze and detect data based on the interactive results of interactive operations, process the collected data information to support decision-making, and store and manage information after data processing.

[0112] In addition, the aforementioned panoramic image and path recording module, panoramic view restoration and interaction module, key area marking and interaction module, and data analysis and decision support module are also used to implement other steps of the outdoor engineering inspection method based on panoramic images and outdoor inspection trajectories, which will not be elaborated here.

[0113] In summary, this invention achieves full-process visual management of the detection process by using a safety helmet equipped with a panoramic camera to automatically record 360-degree visual information of the detection scene and simultaneously track and save the travel route and location information.

[0114] Meanwhile, the system reconstructs the inspection site view using the recorded location information and provides an interactive 360-degree viewing function, ensuring users can observe the inspection site in detail from any angle. Furthermore, by marking and annotating key areas, users can easily obtain relevant inspection information or additional explanations, enhancing the readability and usability of the data.

[0115] Finally, by analyzing the results of the interactive operations, this method effectively supports decision-making and securely stores and manages all information, providing users with a structured and efficient workflow. The solution provided by this invention significantly improves the efficiency and accuracy of outdoor engineering inspection, offering users a more intuitive and complete on-site inspection experience, while also simplifying data management and analysis processes and enhancing the effectiveness of decision support.

[0116] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for outdoor engineering inspection and monitoring based on panoramic imagery and outdoor inspection trajectories, characterized in that, Includes the following steps: Step 1: Collect panoramic images of the site. Use a safety helmet equipped with a panoramic camera to automatically record 360-degree visual information of the outdoor inspection scene and record the inspection path. Track and save the route and location information synchronously as the inspection personnel move. At the same time, use a mobile APP to attach high-definition photos of the inspection location and content to the outdoor inspection trajectory and panoramic images. Step 2: Based on the recorded location information, reconstruct the scene and construct a realistic and complete view of the detection site using panoramic images. Based on the formed view of the detection site, 360-degree viewing is achieved, allowing users to rotate, zoom in or out of the image to observe any angle. Click on the high-resolution photo to view the corresponding detection location and on-site detection details; Step 3: Based on the perspective, mark key areas, annotate the areas that need attention or have questions in the image, and use the marked key areas for interactive operations. Users can select the marked points to obtain detection information or additional explanations. Step 4: Based on the interaction results of the interactive operation, analyze the detection data, process the collected data information to support decision-making, and store and manage the information after data processing. The acquisition of panoramic images of the site involves using a safety helmet equipped with a panoramic camera to automatically record 360-degree visual information of the outdoor inspection scene, including: Based on the acquisition of panoramic images of the scene, image quality is optimized. The panoramic camera built into the safety helmet automatically adjusts the exposure and color balance to ensure the accuracy of visual information. The adjusted image is segmented and each segment is assigned a unique identifier according to its position in the 360-degree view. Based on a unique identifier, coordinate transformation is performed on each image segment. After the coordinate transformation is completed, edge fusion is performed between adjacent image segments. The average pixel value is calculated using the overlapping area and used as the boundary value of the new image segment. Based on the edge fusion, a complete panoramic image file is generated. High-definition photos can be attached, and as the inspection personnel move, high-definition photos of the inspection location and content can be attached to the outdoor inspection trajectory and panoramic inspection image via a mobile APP. This allows users to take and upload high-definition photos at any time and associate them with the current geographical location information, thereby providing additional visual reference for the location. The recorded inspection path, which tracks and saves the travel route and location information synchronously as the inspection personnel move, includes: Based on the recorded inspection path, the location information is made more accurate. As the inspection personnel move, the current location coordinates are captured by the built-in positioning device and associated with the timestamp. Based on the location coordinates and timestamp, the distance between two adjacent points is calculated. After the distance calculation is completed, the inspection path map is constructed using the accumulated distance values. For each newly captured location coordinate, the path map is updated to reflect the latest direction of travel, and the location is added to the existing path sequence. After the path map is updated, path optimization is implemented. For duplicate or unnecessary backtracking segments on the path, a simplified algorithm is used and a threshold is set. When the shortest distance between two paths is less than the threshold, the two paths are merged. The step of reconstructing the scene based on the recorded location information and constructing a realistic and complete view of the detection site using panoramic images includes: Based on the location coordinates and timestamp, the corresponding panoramic image file is located, and the location information is matched with the unique identifier in the panoramic image. After acquiring the corresponding panoramic image, integrate the adjacent panoramic images, and calculate the distance between two consecutive position coordinates; Based on the completion of panoramic image integration, a multi-view interactive 3D view is generated. For each newly added panoramic image segment, the dataset of the 3D view is updated, and the view layout is adjusted according to the optimized inspection path. A threshold is set to control the degree of path simplification. When the shortest distance between two paths is less than the threshold, the two paths are merged. The step of achieving 360-degree viewing based on the formed detection site view, allowing users to rotate, zoom in, or zoom out the image to observe from any angle, includes: After generating a multi-view interactive 3D view, a user input interface is established to receive user operation commands. When the user selects to rotate the view, the system adjusts the view according to the direction and angle specified by the user. After the view rotation is completed, if the user wants to zoom in on a specific area, the system will recalculate the position of all pixels in that area based on the coordinates of the center point selected by the user and the zoom level. After the user completes the zoom-in or zoom-out operation, the system automatically adjusts the smoothness of the transition between adjacent image segments. For two consecutive position coordinates, it calculates the distance between them and dynamically adjusts the overlapping area between adjacent images based on the distance. Based on all operations, users can set a marker at any point of interest. The system records the offset of the marker relative to the current view center point. When the user revisits the view or shares it with other users, the system can accurately reproduce the marker position, allowing the user to directly navigate to the area of ​​interest.

2. The method for outdoor engineering inspection and supervision based on panoramic imagery and outdoor detection trajectory according to claim 1, characterized in that, Based on the viewpoint, marking key areas and annotating areas in the image that require attention or raise questions includes: The user first selects the key area to be marked, and the system records the center coordinates and radius of the area; After selecting the marking area, in order to ensure that the marking is not accidentally covered or removed, the system calculates a unique identifier. The system uses an identifier as an index to embed visual markers in the image and associates the position of the markers with the coordinates in the original view. This information is displayed when the user zooms in or out. After all operations are completed, if the user wants to add text descriptions, they can specify the starting coordinates of a text box near the marked position, and the system will check whether the position overlaps with other marks.

3. The method for outdoor engineering inspection and supervision based on panoramic imagery and outdoor detection trajectory according to claim 2, characterized in that, The interactive operation utilizes marked key areas, allowing users to select marked points to obtain detection information or additional explanations, including: When a user clicks on any marker, the system retrieves the associated detailed information based on the marker's unique identifier. Once a marker point is selected, the system searches for and loads the pre-stored relevant data records based on the point's unique identifier, and simultaneously calculates the marker point's display position in the current view; After loading the detailed information, the system creates a temporary interactive window that allows users to browse or scroll through additional information; After a user completes viewing the detailed information of a certain marker, the system records the user's access behavior and updates the access log. For each interaction, the system generates a new timestamp and associates it with a unique identifier.

4. The method for outdoor engineering inspection and supervision based on panoramic imagery and outdoor detection trajectory according to claim 3, characterized in that, The interaction results based on the interactive operation are analyzed, and the collected data information is processed to support decision-making, including: Collect access logs for all marked points, and the system retrieves the corresponding access records based on the unique identifier of each marked point; Assess the importance of markers based on the number of visits and average dwell time; For markers with high importance scores, extract related details and then apply content analysis rules to identify potential problem types or actions that need to be taken. The analysis results are compiled into a report format and provided to decision-makers for reference. The report not only lists the location of high-scoring markers, related issues and their severity, but also suggests corresponding solutions.

5. The method for outdoor engineering inspection and supervision based on panoramic imagery and outdoor detection trajectory according to claim 4, characterized in that, The storage and management of information after data processing includes: For each marker and its associated access logs, details, and importance score, the system creates a comprehensive data packet containing the marker, location coordinates, radius, timestamp, number of visits, average dwell time, and importance score. The system calculates a checksum based on the content of the composite data packet; After completing packet creation and checksum calculation, the system selects a suitable data storage location and organizes the data according to predefined classification rules. For each newly created packet, its category is determined and a specific storage path is allocated based on the category. Based on ensuring the secure storage of all data, an indexing mechanism is established to support efficient retrieval, and the system generates an index entry for each data packet.

6. A system for outdoor engineering inspection and monitoring based on panoramic imagery and outdoor inspection trajectory for implementing the method described in any one of claims 1-5, characterized in that, include: The panoramic imaging and path recording module is used to collect panoramic images of the site. It uses a safety helmet equipped with a panoramic camera to automatically record 360-degree visual information of the outdoor inspection scene and record the inspection path. It tracks and saves the route and location information synchronously as the inspection personnel move. The panoramic view restoration and interaction module is used to restore the scene based on the recorded location information, construct a real and complete view of the detection site using panoramic images, and realize 360-degree viewing based on the formed detection site view, allowing users to rotate, zoom in or zoom out of the image to observe any angle. The key area marking and interaction module is used to mark key areas based on the viewpoint, mark places that need attention or have questions in the image, and perform interactive operations using the marked key areas. Users can select the marked points to obtain detection information or additional explanations. The data analysis and decision support module is used to analyze and detect data based on the interactive results of interactive operations, process the collected data information to support decision-making, and store and manage information after data processing.

Images