Panoramic image-based engineering inspection track generation method and system

Abstract

The invention belongs to the technical field of engineering inspection, and particularly relates to an engineering inspection track generation method and system based on a panoramic image, automatic inspection is performed through a panoramic camera device along a path dynamically generated by a preset or intelligent planning algorithm, the requirement of manual operation is reduced, and through a spatial position and planar graph hooking technology, the engineering inspection track generation method and system based on the panoramic image are realized. A panoramic video data set with geographic reference is formed, each photo and each frame of image are ensured to have accurate space positioning, subsequent detailed analysis and visual display are facilitated, digitally acquired data are transmitted to a central processing system in real time, and the data are acquired in real time by combining the moving track and timestamp information of inspection personnel. The method provided by the invention not only improves the automation degree and the data accuracy of engineering inspection, but also provides powerful support for data analysis and decision-making after inspection, thereby promoting higher-level intelligent and automatic development of engineering management and monitoring.

Description

Technical Field

[0001] The present invention belongs to the technical field of engineering inspection, and in particular relates to a method and system for generating an engineering inspection trajectory based on panoramic images. Background Art

[0002] In the current field of engineering inspection, traditional inspection methods mainly rely on manual on-site inspections. This method usually involves inspectors carrying paper record sheets or portable devices to manually record information, take photos, and mark key areas at the engineering site. Although this method can meet inspection needs to a certain extent, it has obvious limitations:

[0003] Inefficiency: Manual inspections require a lot of time and human resources, are slow, and are difficult to cover large or complex engineering sites.

[0004] Susceptible to human factors: The quality and accuracy of inspection results are highly dependent on the professional level and personal status of the inspectors, which may lead to omissions or misjudgments.

[0005] Lack of comprehensive coverage: Due to manpower limitations, certain hard-to-reach locations or hidden corners may not be fully inspected, resulting in incomplete inspections.

[0006] In summary, the most significant problem in the existing technology is the lack of efficiency and accuracy in data collection and analysis. The traditional manual inspection method is not only time-consuming and laborious, but also difficult to ensure that all inspection points can be thoroughly inspected, especially for large or complex engineering projects. In addition, the manually recorded data often lacks accurate spatial positioning and timestamp information, which makes subsequent data processing and analysis difficult, affecting the effectiveness and timeliness of decision-making. Summary of the invention

[0007] The purpose of the present invention is to provide a method and system for generating engineering inspection trajectories based on panoramic images, which realizes the comprehensive automation and digitization of engineering inspections through a series of innovative technical steps to solve the problems raised in the above-mentioned background technology.

[0008] To achieve the above object, the present invention provides a method for generating an engineering inspection trajectory based on panoramic images, comprising the following steps:

[0009] Deploy at least one panoramic camera device at the construction site, wherein the panoramic camera device is equipped with multiple cameras for capturing 360-degree environmental images without blind spots, and use the panoramic camera device to perform automatic inspections along a preset path or a path dynamically generated by an intelligent planning algorithm, and collect panoramic image data in real time;

[0010] During the automatic inspection process, mobile terminal devices are used to take photos of key and problematic parts of the project site, and the photos are automatically attached to the inspection track plan of the corresponding position. Based on the spatial position information of the photos and panoramic images, a panoramic video data set with geographic reference is formed through the spatial position and plan attachment technology.

[0011] The obtained panoramic video data set is digitally collected, the original images and photos are converted into digital format, and the movement trajectory and timestamp information of the inspection personnel are simultaneously collected. The digitally collected data and movement trajectory information are transmitted to the central processing system in real time through the AI ​​workstation;

[0012] Combining digitally collected data and mobile trajectory information, the inspection trajectory is displayed in the form of lines or marks on the floor plan, and panoramic images and high-definition photos are fused with the inspection path data for display. Based on the fusion display, users can explore different perspectives and areas through an interactive interface.

[0013] Preferably, the deploying of at least one panoramic camera device at the construction site includes:

[0014] Determine the locations of multiple observation points at the project site. Each observation point corresponds to one or more panoramic camera installation positions. Install panoramic camera equipment according to the selected positions. For any two adjacent devices, the overlap ratio of their field of view is at least P%, where P represents the ratio of the intersection of the fields of view between the two devices to the total field of view of a single device.

[0015] Based on the adjusted device settings, the panoramic camera device is started to collect images. The device shoots at a fixed frame rate F, where F represents the number of images captured per second. At the same time, the geographic location coordinates (X, Y, Z) of each device are recorded for spatial positioning of subsequent images.

[0016] During the image acquisition process, the device posture data corresponding to each frame of the image is recorded synchronously, including the rotation angle θ, the tilt angle φ and the yaw angle ψ, and the direction matrix M = R(θ)*T(φ)*P(ψ) of each device relative to the reference coordinate system is calculated, where R, T, and P represent the rotation operations around the three axes, respectively.

[0017] Preferably, the use of the panoramic camera device to automatically inspect along a preset path or a path dynamically generated by an intelligent planning algorithm and collect panoramic image data in real time includes:

[0018] Set the inspection starting point and target end point, and plan at least one inspection route based on the map information of the project site. This route consists of continuous coordinate points (X_i, Y_i, Z_i), where i represents the i-th coordinate point on the path;

[0019] Start the panoramic camera device to move along the defined route and start capturing images at a fixed frame rate F. For each coordinate point (X_i, Y_i, Z_i), calculate the time required for the device to reach that point T_i = D_i / V, where D_i is the distance from the previous point to the current point and V is the moving speed of the device.

[0020] During the movement, when the device approaches the next coordinate point, adjust the device's attitude parameters (θ_i, φ_i, ψ_i) so that the device's field of view is aligned with the direction of the next coordinate point. The attitude adjustment is based on the formula M_i=R(θ_i)*T(φ_i)*P(ψ_i), where R, T, and P represent the operation of rotating around three axes respectively;

[0021] After completing the posture adjustment, start recording the panoramic image data of the location immediately, and save the data together with the current position coordinates (X_i, Y_i, Z_i), timestamp T_i and posture parameters (θ_i, φ_i, ψ_i), and then continue to move to the next coordinate point until the inspection route ends.

[0022] Preferably, the step of automatically attaching the photo to the inspection track plan of the corresponding position includes:

[0023] When the inspection device reaches the preset key or problem area, it triggers the shooting command of the mobile terminal device to determine the coordinates (X_j, Y_j, Z_j) of each shooting position P_j;

[0024] At the specified position P_j, the mobile terminal device captures a high-definition photo I_j with the highest resolution, and records the timestamp T_j of the shooting time and the device posture parameters (θ_j, φ_j, ψ_j). For each photo I_j, calculate its spatial distance D_j = sqrt((X_j-X_i)^2+(Y_j-Y_i)^2+(Z_j-Z_i)^2) from the most recent panoramic image acquisition point, where (X_i, Y_i, Z_i) are the coordinates of the most recent panoramic image acquisition point;

[0025] Based on the timestamp T_j and spatial distance D_j, associate the high-definition photo I_j with the corresponding inspection track position, and use the formula W_j=exp(-α*D_j^2) to calculate the weight W_j, where α is an adjustment parameter;

[0026] The high-definition photos I_j and their associated weights W_j, timestamps T_j, coordinates (X_j, Y_j, Z_j) and posture parameters (θ_j, φ_j, ψ_j) are integrated into the dataset of the inspection trajectory plan. This dataset allows users to browse photos by clicking or dragging the interactive interface and intuitively see the relationship between the photos and the actual location.

[0027] Preferably, the forming of a panoramic video data set with a geographical reference comprises:

[0028] Collect all high-definition photos I_j and their corresponding spatial position coordinates (X_j, Y_j, Z_j), timestamps T_j and posture parameters (θ_j, φ_j, ψ_j). At the same time, obtain the continuous image frames F_i recorded by the panoramic camera device in the same time period and their corresponding geographic location coordinates (X_i, Y_i, Z_i) and timestamps T_i;

[0029] For each high-definition photo I_j, find the panoramic image frame F_i closest to its timestamp T_j, calculate the time difference ΔT = |T_j-T_i| between the two to ensure the time synchronization between the photo and the image frame, and select the photo and image frame pair that meets the threshold of ΔT < τ, where τ is a preset time tolerance;

[0030] According to the selected photo and image frame pair, the spatial distance D_i j = sqrt((X_j-X_i)^2+(Y_j-Y_i)^2+(Z_j-Z_i)^2) is calculated using the spatial position coordinates (X_j, Y_j, Z_j) and (X_i, Y_i, Z_i), and the association weight W_i j is calculated by the formula W_i j = exp(-β*D_i j^2), where β is the adjustment parameter;

[0031] The identified photo and image frame pairs, together with their weights Wij, timestamps Tj / Ti, coordinates (Xj, Yj, Zj) / (Xi, Yi, Zi) and pose parameters (θj, φj, ψj), are integrated into a unified dataset.

[0032] Preferably, the method of digitally collecting the obtained panoramic video data set, converting the original images and photos into digital format, and synchronously collecting the movement trajectory and timestamp information of the inspection personnel includes:

[0033] Receive a panoramic video dataset with georeferences, and for each pair of high-definition photos I_j and panoramic image frames F_i, extract the binary data streams B_j and B_i of the original image and photo, and at the same time, obtain the associated timestamps T_j / T_i, spatial position coordinates (X_j, Y_j, Z_j) / (X_i, Y_i, Z_i) and posture parameters (θ_j, φ_j, ψ_j);

[0034] Convert the binary data streams B_j and B_i into compressed formats C_j and C_i using the formula C_x = Compress(B_x, r), where x represents j or i, Compress represents the compression operation, and r is the compression rate. At the same time, keep the original precision of the timestamp T_j / T_i, coordinates (X_j, Y_j, Z_j) / (X_i, Y_i, Z_i) and attitude parameters (θ_j, φ_j, ψ_j) unchanged;

[0035] During the data processing, the movement trajectory of the patrol personnel is recorded synchronously. The position coordinates (X_p, Y_p, Z_p) and timestamp T_p of the patrol personnel are sampled regularly through the positioning device integrated in the patrol equipment. The time interval ΔT_p = T_p-T_(p-1) between two adjacent samples is calculated to ensure the continuity of the trajectory and time synchronization. For each timestamp T_p, the corresponding posture parameters (θ_p, φ_p, ψ_p) of the patrol personnel are recorded to form a complete movement trajectory record.

[0036] The compressed image data C_j / C_i, timestamp T_j / T_i, spatial position coordinates (X_j, Y_j, Z_j) / (X_i, Y_i, Z_i), posture parameters (θ_j, φ_j, ψ_j) and the inspection personnel’s movement trajectory (X_p, Y_p, Z_p), timestamp T_p and posture parameters (θ_p, φ_p, ψ_p) are integrated into one data set.

[0037] Preferably, the digitally collected data and movement trajectory information are transmitted to the central processing system in real time through the AI ​​workstation, including:

[0038] Extract the compressed image data C_j / C_i, timestamp T_j / T_i, spatial position coordinates (X_j, Y_j, Z_j) / (X_i, Y_i, Z_i), posture parameters (θ_j, φ_j, ψ_j), and the movement trajectory (X_p, Y_p, Z_p), timestamp T_p and posture parameters (θ_p, φ_p, ψ_p) of the inspector from the integrated data set;

[0039] Pack the data into transmission packets D. Each transmission packet D contains one or more image frames and their associated information, and is accompanied by a checksum (D) = Hash (Summary (D)), where Summary (D) represents a summary of the contents of the transmission packet, and Hash is a hash function;

[0040] The generated transmission packet D is sent to the central processing system in real time. After each transmission, the sending time and the receiving confirmation time are recorded, and the transmission delay ΔT_transmit = T_receive-T_send is calculated. For each transmission packet D, its corresponding transmission delay ΔT_transmit is recorded to form a transmission log; T_send is the timestamp of the data packet starting to be transmitted from the sending end, and T_receive is the timestamp of the data packet arriving at the receiving end and being successfully received;

[0041] After the central processing system receives the transmission package D, it unpacks and verifies the checksum (D). Then, according to the timestamps T_j / T_i and T_p, the image data C_j / C_i and the inspection personnel's movement trajectory (X_p, Y_p, Z_p) are synchronized and integrated into the database. The timestamp T_x is adjusted by the formula Sync(T_x,ΔT_transmit)=Adjust(T_x+ΔT_transmit), where x represents j / i or p, and T_x represents the original timestamp.

[0042] Preferably, the fusion display of the panoramic image and high-definition photo with the inspection path data includes:

[0043] Obtain all the data transmitted and integrated from the central processing system, and draw the inspection route in the form of lines on the plan of the project site based on the inspection personnel's movement trajectory (X_p, Y_p, Z_p). For each trajectory point P_p, use the formula P_line=Line(P_(p-1),P_p), where Line represents the line segment connecting two consecutive trajectory points, forming a coherent inspection route map. At the same time, record the timestamp T_p of each trajectory point;

[0044] According to the inspection route map, the panoramic image frame F_i and the high-definition photo I_j are associated with the corresponding trajectory point P_p, using the spatial position coordinates (X_j, Y_j, Z_j) / (X_i, Y_i, Z_i) and timestamp T_j / T_i;

[0045] The closest trajectory point P_p is found by the formula D_associate=min(Distance(P_p,(X_x,Y_x,Z_x))), where x represents j or i, and Distance represents the distance calculation between two points;

[0046] Marks are added to the corresponding positions on the inspection path map to display the panoramic image frame F_i and the high-definition photo I_j. For each mark M_x, the timestamp T_x of the shooting time and the posture parameters (θ_x, φ_x, ψ_x) are displayed to provide users with interactive browsing functions. The formula Overlay(M_x, P_line) = Merge(M_x, Line(P_(p-1), P_p)) is used to merge the mark with the inspection path segment to form a comprehensive view.

[0047] Preferably, the fusion display enables users to explore different perspectives and areas through an interactive interface, including:

[0048] Construct a map view based on the integration of patrol route map and image data. In this map view, each patrol track point P_p and its associated panoramic image frame F_i and high-definition photo I_j are marked. For each mark M_x, its visible range V_x is defined as a sphere with a center at (X_x, Y_x, Z_x) and a radius of R, expressed as V_x = Sphere((X_x, Y_x, Z_x), R);

[0049] Based on the established map view, zoom and pan operations are provided to allow users to focus on the area of ​​interest. When the user selects a specific marker M_x, the system calculates the distance D_view between the marker and the user's current viewpoint and uses the formula ZoomLevel = f(D_view), where f is a mapping function to determine the appropriate zoom level.

[0050] According to the user operation, the multi-view browsing mode is activated. For the selected marker M_x, its posture parameters (θ_x, φ_x, ψ_x) are used to construct the transformation matrix T_x to adjust the angle of image display;

[0051] Introduce an instant information prompt box. When the user hovers or clicks a marker M_x, relevant information is immediately displayed, including but not limited to the timestamp T_x, location coordinates (X_x, Y_x, Z_x), and a brief description.

[0052] On the other hand, the present invention proposes a system for generating engineering inspection trajectories based on panoramic images, comprising:

[0053] A panoramic camera equipment deployment module, used to deploy at least one panoramic camera equipment at the construction site, wherein the panoramic camera equipment is equipped with multiple cameras for capturing environmental images at 360 degrees without blind spots, and the panoramic camera equipment is used to perform automatic inspections along a preset path or a path dynamically generated by an intelligent planning algorithm, and collect panoramic image data in real time;

[0054] The high-definition photo shooting and mounting module is used to take photos of key parts and problem parts of the project site using mobile terminal devices during the automatic inspection process, and automatically mount the photos to the inspection track plan of the corresponding position. Based on the spatial position information of the photos and panoramic images, a panoramic video data set with geographical reference is formed through the spatial position and plan mounting technology;

[0055] The digital acquisition and synchronous recording module is used to digitally acquire the acquired panoramic video data set, convert the original images and photos into digital format, and synchronously acquire the movement trajectory and timestamp information of the inspection personnel. The digital acquisition data and movement trajectory information are transmitted to the central processing system in real time through the AI ​​workstation;

[0056] The inspection track visualization display module is used to combine digital collection data and movement track information to display the inspection track in the form of lines or marks on the floor plan, and to fuse panoramic images and high-definition photos with the inspection path data for display. Based on the fusion display, the interactive interface enables users to explore different perspectives and areas.

[0057] Technical effects and advantages of the present invention: Compared with the prior art, the present invention provides a method and system for generating engineering inspection trajectories based on panoramic images, which has the following advantages:

[0058] The present invention integrates panoramic camera equipment, high-definition camera and positioning system, and the method realizes all-round and high-definition automatic inspection of the engineering site. The panoramic camera equipment performs automatic inspection along the path preset or dynamically generated by the intelligent planning algorithm, which reduces the need for manual operation, ensures the comprehensiveness and consistency of inspection coverage, and improves inspection efficiency. Through the spatial position and plane map hanging technology, a panoramic video data set with geographical reference is formed to ensure that each photo and each frame of the image has accurate spatial positioning, which is convenient for subsequent detailed analysis and visual display. The digitally collected data is transmitted to the central processing system in real time, and combined with the movement trajectory and timestamp information of the inspection personnel, a comprehensive and interactive interface is provided. Users can quickly browse information from different perspectives and regions through this interface, support rapid analysis and decision-making, as well as identification and response of complex problems. The method of the present invention not only improves the automation level and data accuracy of engineering inspection, but also provides strong support for data analysis and decision-making after inspection, thereby promoting engineering management and monitoring to a higher level of intelligence and automation. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 It is a flow chart of a method for generating an engineering inspection trajectory based on panoramic images according to the present invention;

[0060] Figure 2The present invention is a block diagram of a system for generating engineering inspection trajectories based on panoramic images. DETAILED DESCRIPTION

[0061] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. The specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0062] The present invention provides a method for generating an engineering inspection trajectory based on panoramic images, such as Figure 1 As shown, the following steps are included:

[0063] Step 1: deploy at least one panoramic camera device at the construction site, wherein the panoramic camera device is equipped with multiple cameras for capturing 360-degree environmental images without blind spots; further comprising:

[0064] Determine the locations of multiple observation points at the project site. Each observation point corresponds to one or more panoramic camera installation positions. Panoramic camera equipment is installed according to the selected positions. For any two adjacent devices, the overlap ratio of their field of view is at least P%, where P represents the ratio of the intersection of the fields of view between the two devices to the total field of view of a single device. This setting not only avoids visual blind spots, but also provides redundant data, enhancing the accuracy of image stitching and 3D reconstruction.

[0065] Based on the adjusted device settings, the panoramic camera device is started to collect images. The device shoots at a fixed frame rate F, where F represents the number of images captured per second. At the same time, the geographic location coordinates (X, Y, Z) of each device are recorded for the spatial positioning of subsequent images; this provides an accurate geographic reference for the subsequent spatial positioning of images, so that each frame of the image can be accurately located at the actual position.

[0066] During the image acquisition process, the device posture data corresponding to each frame of the image is recorded synchronously, including the rotation angle θ, the tilt angle φ, and the yaw angle ψ, and the direction matrix M = R(θ)*T(φ)*P(ψ) of each device relative to the reference coordinate system is calculated, where R, T, and P represent the rotation operations around the three axes. By calculating the direction matrix M = R(θ)*T(φ)*P(ψ) of each device relative to the reference coordinate system, the posture information of the device at the time of shooting can be accurately restored. This step is crucial for subsequent data analysis because it ensures that images and photos can be displayed in the correct geospatial context, supporting detailed analysis of complex structures and environments.

[0067] Accurate geographic location coordinates and device posture data provide a solid foundation for subsequent data processing and visualization. These data can help build a panoramic video dataset with georeferences, support the application of virtual reality technology, and provide users with an immersive inspection environment experience. In addition, it also facilitates advanced applications such as map updates, space planning, and asset management.

[0068] Through the careful deployment and parameter setting of panoramic camera equipment, the entire system has been significantly improved in terms of inspection efficiency, data quality and user experience. The need for manual operation has been reduced, the risk of human error has been reduced, and the reliability and repeatability of inspection results have been improved.

[0069] Step 2: Use the panoramic camera device to automatically patrol along a preset path or a path dynamically generated by an intelligent planning algorithm, and collect panoramic image data in real time; further comprising:

[0070] Set the inspection starting point and target end point, and plan at least one inspection route based on the map information of the project site. This route consists of continuous coordinate points (X_i, Y_i, Z_i), where i represents the i-th coordinate point on the path; this path planning based on map information ensures comprehensive inspection coverage, avoids missing important areas, and can flexibly respond to complex terrain and structures. The panoramic camera equipment moves along the defined route, ensuring the automation and standardization of the inspection process, reducing the need for manual intervention and improving inspection efficiency.

[0071] Start the panoramic camera device to move along the defined route, and start capturing images at a fixed frame rate F. For each coordinate point (X_i, Y_i, Z_i), calculate the time required for the device to reach that point T_i = D_i / V, where D_i is the distance from the previous point to the current point, and V is the moving speed of the device; this step ensures that the device can accurately control the moving speed and time during the inspection process, ensuring the continuity and time synchronization of image capture. Each time a new coordinate point is reached, the current position coordinates (X_i, Y_i, Z_i) and timestamp T_i are recorded, providing accurate time and space references for subsequent data processing, enhancing the reliability and availability of the data.

[0072] During the movement, when the device approaches the next coordinate point, the device's posture parameters (θ_i, φ_i, ψ_i) are adjusted to align the center of the device's field of view with the direction of the next coordinate point. The posture adjustment is based on the formula M_i = R (θ_i) * T (φ_i) * P (ψ_i), where R, T, and P represent the operation of rotating around three axes respectively; ensuring that the device always faces the predetermined direction, improving the quality and accuracy of image capture. Through precise posture adjustment, the device can capture the most relevant and valuable images at each coordinate point, reducing unnecessary repeated shooting and improving inspection efficiency.

[0073] After completing the posture adjustment, it immediately starts recording the panoramic image data of the location, and saves the data together with the current position coordinates (X_i, Y_i, Z_i), timestamp T_i and posture parameters (θ_i, φ_i, ψ_i), and then continues to move to the next coordinate point until the inspection route ends. This detailed metadata recording method ensures that each frame of the image has complete information support, which is convenient for subsequent data analysis and visualization. The device captures images at a fixed frame rate F, which ensures the continuity and clarity of the image, while ensuring the data integrity during the entire inspection process.

[0074] The entire inspection process is highly automated, from path planning to image acquisition to data recording, which reduces the need for manual operation, reduces the risk of human error, and improves the reliability of inspection results. Detailed data recording and precise spatial positioning provide users with a detailed and intuitive inspection report, support rapid analysis and decision-making, and enhance user experience and system practicality.

[0075] Step 3: During the automatic inspection process, use a mobile terminal device to take photos of key parts and problem parts of the project site, and automatically attach the photos to the inspection track plan of the corresponding position; further including:

[0076] When the inspection equipment reaches the preset key or problem area, the shooting command of the mobile terminal device is triggered to determine the coordinates (X_j, Y_j, Z_j) of each shooting position P_j; this process ensures attention to important locations, reduces unnecessary shooting, and improves work efficiency. At the specified location P_j, the mobile terminal device captures high-definition photos I_j at the highest resolution, providing detailed local details, which is helpful for subsequent analysis and decision-making.

[0077] At the specified position P_j, the mobile terminal device captures a high-definition photo I_j at the highest resolution, and records the timestamp T_j of the shooting time and the device posture parameters (θ_j, φ_j, ψ_j). For each photo I_j, the spatial distance between it and the most recent panoramic image acquisition point is calculated D_j = sqrt((X_j-X_i)^2+(Y_j-Y_i)^2+(Z_j-Z_i)^2), where (X_i, Y_i, Z_i) are the coordinates of the most recent panoramic image acquisition point; this step ensures that the spatial relationship between the photo and the panoramic image is clear and unambiguous.

[0078] Based on the timestamp T_j and spatial distance D_j, the high-definition photo I_j is associated with the corresponding inspection track position, and the weight W_j is calculated using the formula W_j=exp(-α*D_j^2), where α is an adjustment parameter; this weight calculation method takes into account time and space factors, so that photos can be reasonably sorted and displayed according to their relevance, improving the efficiency of users browsing and finding specific information.

[0079] The high-definition photos I_j and their associated weights W_j, timestamps T_j, coordinates (X_j, Y_j, Z_j) and posture parameters (θ_j, φ_j, ψ_j) are integrated into the dataset of the inspection trajectory plan. This comprehensive dataset not only contains rich information, but also allows users to browse photos by clicking or dragging the interactive interface, and intuitively see the relationship between the photos and the actual location. This dataset allows users to browse photos by clicking or dragging the interactive interface, and intuitively see the relationship between the photos and the actual location. Through the interactive interface, users can easily explore different perspectives and areas, obtain more detailed information, and thus have a more comprehensive understanding of the inspection situation, supporting rapid analysis and decision-making.

[0080] This method can help engineers and managers quickly identify and respond to complex problems, improve the quality of inspections and decision-making efficiency. By collecting high-definition photos of key parts and problem parts and accurately attaching them, it provides strong data support, promotes data-based decision-making, and enhances project management and monitoring capabilities.

[0081] Step 4: Based on the spatial location information of the photos and panoramic images, a panoramic video dataset with geographic reference is formed through spatial location and plan view connection technology; further including:

[0082] Collect all the high-definition photos I_j and their corresponding spatial position coordinates (X_j, Y_j, Z_j), timestamps T_j and posture parameters (θ_j, φ_j, ψ_j). At the same time, obtain the continuous image frames F_i recorded by the panoramic camera device in the same time period and their corresponding geographic location coordinates (X_i, Y_i, Z_i) and timestamps T_i; this step ensures that all relevant data are fully collected, providing a rich information basis for subsequent processing.

[0083] For each high-definition photo I_j, find the panoramic image frame F_i closest to its timestamp T_j, calculate the time difference ΔT = |T_j-T_i| between the two to ensure the time synchronization between the photo and the image frame, and select the photo and image frame pairs that meet the ΔT<τ threshold, where τ is a preset time tolerance; this time synchronization mechanism ensures the matching of images and photos in the time dimension, and improves the consistency and reliability of the data.

[0084] Based on the selected photo and image frame pair, the spatial distance between the two is calculated using the spatial position coordinates (X_j, Y_j, Z_j) and (X_i, Y_i, Z_i):

[0085] D_i j = sqrt((X_j-X_i)^2+(Y_j-Y_i)^2+(Z_j-Z_i)^2), and the association weight Wi j is calculated by the formula Wi j = exp(-β*D_i j^2), where β is the adjustment parameter; this step ensures the accurate association between images and photos in the spatial dimension and enhances the spatial consistency of the data. This weight calculation method comprehensively considers the influence of spatial distance, making the association between different images and photos more reasonable, which is helpful for subsequent data analysis and priority sorting during display.

[0086] The determined photo and image frame pairs, together with their weights Wij, timestamps Tj / Ti, coordinates (Xj, Yj, Zj) / (Xi, Yi, Zi) and posture parameters (θj, φj, ψj), are integrated into a unified dataset. This integration method not only contains rich metadata information, but also ensures the integrity and traceability of the data, which is convenient for subsequent query and analysis. The resulting panoramic video dataset with georeferences can accurately reflect the location and time information of each image and photo at the actual engineering site, and supports advanced spatial analysis and visualization applications, such as virtual reality inspection, 3D modeling, etc.

[0087] Through precise time synchronization and spatial association, as well as reasonable weight calculation, the accuracy and reliability of data are greatly improved, the need for manual intervention is reduced, and the risk of data errors is reduced. Users can browse these data through an interactive interface to view the specific location and time relationship of high-definition photos and panoramic images, supporting rapid analysis and decision-making. This intuitive data display method greatly improves the user experience and the practicality of the system, and promotes efficient management and decision-making based on data.

[0088] Step 5: Digitally collect the obtained panoramic video data set, convert the original images and photos into digital format, and simultaneously collect the movement trajectory and timestamp information of the inspection personnel; further including:

[0089] Receive a panoramic video dataset with geo-reference. For each pair of high-definition photo I_j and panoramic image frame F_i, extract the binary data streams B_j and B_i of the original image and photo, and at the same time, obtain the associated timestamp T_j / T_i, spatial position coordinates (X_j, Y_j, Z_j) / (X_i, Y_i, Z_i) and posture parameters (θ_j, φ_j, ψ_j); this step ensures that all relevant information is fully collected, providing a solid foundation for subsequent processing.

[0090] Convert binary data streams B_j and B_i into compressed formats C_j and C_i using the formula C_x = Compress(B_x, r), where x represents j or i, Compress represents the compression operation, and r is the compression rate. At the same time, keep the original precision of timestamp T_j / T_i, coordinates (X_j, Y_j, Z_j) / (X_i, Y_i, Z_i) and attitude parameters (θ_j, φ_j, ψ_j) unchanged; during the data compression process, keep the original precision of timestamp T_j / T_i, coordinates (X_j, Y_j, Z_j) / (X_i, Y_i, Z_i) and attitude parameters (θ_j, φ_j, ψ_j) unchanged. This processing method ensures the consistency and reliability of data before and after compression, and avoids information loss caused by compression. The compressed data not only reduces storage requirements, but also optimizes transmission efficiency, making the management and transmission of large-scale data more convenient.

[0091] During the data processing, the movement trajectory of the patrol personnel is recorded synchronously. Through the positioning device integrated in the patrol equipment, the position coordinates (X_p, Y_p, Z_p) and timestamp T_p of the patrol personnel are sampled regularly, and the time interval ΔT_p = T_p-T_(p-1) between two adjacent samples is calculated to ensure the continuity of the trajectory and time synchronization. This step ensures that the movement trajectory of the patrol personnel can accurately reflect the actual inspection process. For each timestamp T_p, the corresponding posture parameters (θ_p, φ_p, ψ_p) of the patrol personnel are recorded to form a complete movement trajectory record; these data not only enhance the richness of the trajectory information, but also provide support for subsequent spatial analysis and behavior pattern recognition.

[0092] The compressed image data C_j / C_i, timestamp T_j / T_i, spatial position coordinates (X_j, Y_j, Z_j) / (X_i, Y_i, Z_i), posture parameters (θ_j, φ_j, ψ_j) and the movement trajectory (X_p, Y_p, Z_p), timestamp T_p and posture parameters (θ_p, φ_p, ψ_p) of the inspector are integrated into one data set. This comprehensive data set not only contains rich metadata information, but also ensures the relevance and consistency between all data.

[0093] The comprehensive data set provides strong support for subsequent data analysis, visualization and decision support. Users can browse this data through an interactive interface, view images, photos and their relationship with the movement trajectory of patrol personnel, so as to have a more comprehensive understanding of the patrol situation and support rapid analysis and decision making.

[0094] Step 6: Utilize digital collection data and movement trajectory information and transmit it to the central processing system in real time through the AI ​​workstation; further including:

[0095] Extract the compressed image data C_j / C_i, timestamp T_j / T_i, spatial position coordinates (X_j, Y_j, Z_j) / (X_i, Y_i, Z_i), posture parameters (θ_j, φ_j, ψ_j) and the inspection personnel's movement trajectory (X_p, Y_p, Z_p), timestamp T_p and posture parameters (θ_p, φ_p, ψ_p) from the integrated data set; this step ensures that all relevant data are fully extracted and provides a complete data packet for subsequent transmission.

[0096] The data is packaged into transmission packages D, each of which contains one or more image frames and their associated information, and is accompanied by a check code Checksum(D) = Hash(Summary(D)), where Summary(D) represents a summary of the contents of the transmission package and Hash is a hash function. This packaging method not only improves the security of data transmission, but also ensures the integrity of the data through the check code.

[0097] The generated transmission packet D is sent to the central processing system in real time. After each transmission, the sending time and the receiving confirmation time are recorded, and the transmission delay ΔT_transmit = T_receive-T_send is calculated. For each transmission packet D, its corresponding transmission delay ΔT_transmit is recorded to form a transmission log. This step ensures the timeliness and reliability of data transmission and provides a detailed transmission log for subsequent analysis. T_send is the timestamp of the data packet starting from the sender, and T_receive is the timestamp of the data packet arriving at the receiver and being successfully received; by recording the transmission delay ΔT_transmit, the network status can be monitored, potential transmission problems can be identified, and the transmission path can be optimized to ensure that the data reaches the central processing system in the shortest time.

[0098] After the central processing system receives the transmission packet D, it unpacks and verifies the checksum (D). Then, according to the timestamps T_j / T_i and T_p, the image data C_j / C_i and the movement trajectory of the patrol personnel (X_p, Y_p, Z_p) are synchronized and integrated into the database. The timestamp T_x is adjusted by the formula Sync(T_x, ΔT_transmit) = Adjust(T_x+ΔT_transmit), where x represents j / i or p, and T_x represents the original timestamp. This verification mechanism enhances the security and reliability of data transmission and reduces the risk of data loss or damage. This step ensures the time consistency of all data and supports accurate time synchronization and event association.

[0099] The entire transmission process is highly automated, from data extraction, packaging to transmission and verification, reducing the need for manual intervention and improving processing efficiency and accuracy. Through real-time transmission and synchronous integration, the central processing system can obtain the latest inspection data in a timely manner, supporting rapid data analysis and decision-making. Users can browse these data through an interactive interface, view images, photos and their relationship with the movement trajectory of inspection personnel, so as to have a more comprehensive understanding of the inspection situation. Through checksum verification and transmission delay monitoring, the security and reliability of data transmission are ensured, the risk of data errors and loss is reduced, and the stability and credibility of the system are enhanced.

[0100] Step 7: combining the digitally collected data and the movement trajectory information, and displaying the inspection trajectory in the form of lines or marks on the plan; further comprising:

[0101] All data transmitted and integrated from the central processing system are obtained. Based on the inspection personnel's movement trajectory (X_p, Y_p, Z_p), the inspection path is drawn in the form of lines on the plan of the project site. For each trajectory point P_p, the formula P_line=Line(P_(p-1),P_p) is used, where Line represents the line segment connecting two consecutive trajectory points, forming a coherent inspection route map. At the same time, the timestamp T_p of each trajectory point is recorded; this step ensures the accurate drawing of the inspection path and provides clear visual guidance. At the same time, the timestamp T_p of each trajectory point is recorded, which provides a basis for subsequent time synchronization and event association.

[0102] According to the inspection route map, the panoramic image frame F_i and the high-definition photo I_j are associated with the corresponding trajectory point P_p using the spatial position coordinates (X_j, Y_j, Z_j) / (X_i, Y_i, Z_i) and timestamp T_j / T_i; this association method ensures that the images and photos can be displayed in the correct position, enhancing the spatial consistency of the data.

[0103] The closest trajectory point P_p is found through the formula D_associate=min(Distance(P_p,(X_x,Y_x,Z_x))), where x represents j or i, and Distance represents the distance calculation between two points. This method ensures that images and photos can be accurately attached to the nearest patrol path point, improving the accuracy of the display.

[0104] Marks are added to the corresponding positions on the patrol route map to display the panoramic image frame F_i and high-definition photo I_j. For each mark M_x, the timestamp T_x and posture parameters (θ_x, φ_x, ψ_x) of the shooting time are displayed, and the user is provided with the function of interactive browsing. This step allows users to easily view detailed information of specific locations and supports quick analysis and decision-making.

[0105] The formula Overlay(M_x,P_line)=Merge(M_x,Line(P_(p-1),P_p)) is used to merge the mark with the inspection path line segment to form a comprehensive view. This comprehensive view not only shows the inspection path, but also provides the specific location of the image and photo, which enhances the user's understanding and operation convenience.

[0106] By displaying the inspection track in the form of lines or marks on the floor plan, users can intuitively see the inspection path and photos of key parts, supporting quick analysis and decision-making. This intuitive display method greatly improves the user experience and the practicality of the system. The comprehensive view allows users to browse photos by clicking or dragging the interactive interface and view the relationship between photos and actual locations, supporting efficient data query and analysis. Users can quickly locate problem areas and conduct detailed inspections, improving work efficiency.

[0107] Step 8: Fusing the panoramic image and high-definition photos with the inspection route data for display, and based on the fusion display, enabling users to explore different perspectives and areas through an interactive interface; further including:

[0108] A map view based on the integration of patrol route map and image data is constructed. In this map view, each patrol trajectory point P_p and its associated panoramic image frame F_i and high-definition photo I_j are marked. For each marker M_x, its visible range V_x is defined as a sphere with a radius R and its center at (X_x, Y_x, Z_x), expressed as V_x = Sphere((X_x, Y_x, Z_x), R). This setting enables users to intuitively understand the coverage of each marker and enhances the information richness of the map view.

[0109] Based on the established map view, zoom and pan operations are provided to allow users to focus on areas of interest. When the user selects a specific marker M_x, the system calculates the distance D_view between the marker and the user's current viewpoint, and uses the formula ZoomLevel=f(D_view), where f is a mapping function, to determine the appropriate zoom level. This method ensures that users can get the most appropriate visual experience during the browsing process, whether it is a global overview or a detailed view.

[0110] According to user operations, the multi-view browsing mode is activated. For the selected marker M_x, its posture parameters (θ_x, φ_x, ψ_x) are used to construct the transformation matrix T_x to adjust the angle of image display; this allows users to freely change the perspective and observe the same location from different angles, enhancing the user's sense of immersion and the comprehensiveness of information acquisition.

[0111] Introducing an instant information prompt box, when the user hovers or clicks a mark M_x, relevant information is immediately displayed, including but not limited to the timestamp T_x, location coordinates (X_x, Y_x, Z_x), and a brief description. This instant feedback mechanism greatly improves the speed and accuracy of users' acquisition of information and supports rapid decision-making and problem analysis.

[0112] Through the above technical means, users can more intuitively explore different perspectives and locations of the patrol area on the interactive interface. This interactive method not only improves user participation, but also promotes a deeper understanding of patrol data.

[0113] Users can efficiently query and analyze data at specific locations through an interactive interface, supporting faster problem discovery and resolution processes, thereby improving work efficiency. By integrating displays and interactive interfaces, managers can remotely monitor the progress of inspections, keep abreast of on-site conditions, and make necessary adjustments, thereby improving management efficiency. The system's multi-view browsing mode and detailed information prompts are also suitable for training and education scenarios, helping new employees get started faster and understand workflows and technical requirements.

[0114] On the other hand, the present invention proposes a system for generating engineering inspection trajectories based on panoramic images, such as Figure 2 As shown, including:

[0115] A panoramic camera equipment deployment module, used to deploy at least one panoramic camera equipment at the construction site, wherein the panoramic camera equipment is equipped with multiple cameras for capturing environmental images at 360 degrees without blind spots, and the panoramic camera equipment is used to perform automatic inspections along a preset path or a path dynamically generated by an intelligent planning algorithm, and collect panoramic image data in real time;

[0116] The high-definition photo shooting and mounting module is used to take photos of key parts and problem parts of the project site using mobile terminal devices during the automatic inspection process, and automatically mount the photos to the inspection track plan of the corresponding position. Based on the spatial position information of the photos and panoramic images, a panoramic video data set with geographical reference is formed through the spatial position and plan mounting technology;

[0117] The digital acquisition and synchronous recording module is used to digitally acquire the acquired panoramic video data set, convert the original images and photos into digital format, and synchronously acquire the movement trajectory and timestamp information of the inspection personnel. The digital acquisition data and movement trajectory information are transmitted to the central processing system in real time through the AI ​​workstation;

[0118] The inspection track visualization display module is used to combine digital collection data and movement track information to display the inspection track in the form of lines or marks on the floor plan, and to fuse panoramic images and high-definition photos with the inspection path data for display. Based on the fusion display, the interactive interface enables users to explore different perspectives and areas.

[0119] In addition, the above-mentioned panoramic camera equipment deployment module, high-definition photo shooting and mounting module, digital acquisition and synchronous recording module, when executed, are also used to implement the other steps of the above-mentioned method for generating engineering inspection tracks based on panoramic images, as follows:

[0120] Automatic inspection with panoramic cameras: Automatic inspection with panoramic cameras is the key first step in the intelligent engineering inspection method. Panoramic cameras are equipped with multiple high-resolution cameras that can capture 360-degree environmental images without blind spots. These cameras are spherical fisheye lenses to provide a wide viewing angle. The inspection path can be preset or dynamically generated through intelligent planning algorithms to adapt to different inspection needs and environmental changes.

[0121] High-definition photo collection module: When inspecting key and problematic areas at the construction site, use the app on your phone to take photos and save them in high definition. The photos are automatically attached to the inspection track plan of the corresponding position at the moment, so as to obtain more detailed local information and accurately locate the problem location.

[0122] Spatial position and floor plan attachment: Spatial position and floor plan attachment is the process of combining the spatial position information of panoramic images with a two-dimensional floor plan. Form a panoramic video with spatial reference. Combine the collected panoramic video data with precise spatial position information to form a panoramic video data set with geographic reference. Use virtual reality technology to integrate video data with map data to achieve a visual display of spatial data. Through spatial position attachment, users can intuitively understand the correspondence between video content and actual location, improving the readability and practicality of data. This process is achieved through virtual reality mapping technology, which can restore, process and analyze geospatial data. In this step, the panoramic image position, orientation and other spatial information are used to accurately locate the image on the floor plan. This attachment not only provides the geographic context of the image, but can also be used for map updates, spatial planning and asset management. Attach the collected panoramic video data to the corresponding spatial position information.

[0123] Digital acquisition: Digital acquisition is the process of converting the raw image and photo data captured by the panoramic camera into a digital format. This step involves the compression, encoding and storage of images for subsequent analysis and processing. Digital acquisition includes not only image data, but may also include metadata related to the image, such as shooting time, camera position, environmental parameters, etc. The acquisition and processing of these data require efficient algorithms and powerful computing power to ensure the integrity and availability of the data. Synchronously collect the movement trajectory and timestamp information of the patrol personnel. Use virtual reality positioning technology to record the changes in the movement trajectory of the patrol personnel in real time. Combined with timestamp information, ensure the synchronization and accuracy of the data. Through the AI ​​workstation, the collected data is transmitted to the central processing system in real time to reduce the risk of data loss. Encryption technology is used to protect the data security during transmission and prevent unauthorized access.

[0124] Fusion of panoramic images and inspection tracks: The fusion of panoramic images and inspection tracks combines panoramic images and high-definition photos with inspection path data to provide comprehensive and intuitive inspection information. On the floor plan, the inspection track can be displayed in the form of lines or marks, superimposed with panoramic images and photos, so that users can clearly see every key point on the inspection path and related visual information. This fusion display can be achieved through an interactive interface, allowing users to explore different perspectives and areas by clicking or dragging, and obtain more detailed information.

[0125] The effect is as follows:

[0126] Automation: The automatic inspection of panoramic cameras not only reduces the need for manual operation, but also ensures the comprehensiveness and consistency of inspection coverage through preset intelligent path planning. This automated process reduces human errors and improves the reliability and repeatability of inspections.

[0127] Digital collection: Digital collection not only ensures the accuracy and storability of data, but also optimizes the transmission and storage efficiency of data through advanced data compression and encoding technology. In addition, the collection of metadata provides rich contextual information for data analysis, enhancing the usability of data and the depth of analysis.

[0128] Scene restoration: Scene restoration technology provides users with an immersive inspection environment experience by processing video analysis into high-definition images of each frame. This technology not only enhances users' understanding of the inspection environment, but also supports detailed analysis of complex structures and environments, providing strong visual support for engineering decision-making.

[0129] Precise spatial location linkage: The precise linkage of spatial location with the floor plan provides an accurate geographic reference framework by utilizing advanced virtual reality mapping technology. This not only helps to identify specific locations and paths during inspections, but also supports geospatial analysis and tracking.

[0130] Fusion of panoramic images and inspection tracks: The fusion display of panoramic images and inspection tracks provides an intuitive overview of inspection information through an integrated visualization platform. This fusion not only facilitates users to make quick analysis and decisions, but also supports the rapid identification and response of complex problems.

[0131] The intelligent engineering inspection method based on panoramic images, high-definition photos and track mounting of the present invention realizes the comprehensive automation and digitization of engineering inspection through a series of innovative technical steps. Automated panoramic camera inspection improves the efficiency and consistency of inspection, while digital acquisition ensures the accuracy and efficient management of data. Scene restoration technology provides users with an in-depth understanding of the inspection environment, while the precise mounting of spatial positions provides a solid foundation for geospatial analysis. The fusion display of panoramic images and inspection tracks not only enhances the visualization of information, but also promotes rapid decision-making. Overall, this method provides an innovative, efficient and user-friendly solution for the field of intelligent engineering inspection, with broad application prospects and far-reaching industry impact. With the continuous advancement of technology, this method is expected to play a greater role in future engineering management and monitoring, and promote engineering inspection to a higher level of intelligence and automation.

[0132] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for generating engineering inspection trajectories based on panoramic images, characterized in that: The following steps are involved: Deploy at least one panoramic camera device at the construction site, wherein the panoramic camera device is equipped with multiple cameras for capturing 360-degree environmental images without blind spots, and use the panoramic camera device to perform automatic inspections along a preset path or a path dynamically generated by an intelligent planning algorithm, and collect panoramic image data in real time; During the automatic inspection process, mobile terminal devices are used to take photos of key and problematic parts of the project site, and the photos are automatically attached to the inspection track plan of the corresponding position. Based on the spatial position information of the photos and panoramic images, a panoramic video data set with geographic reference is formed through the spatial position and plan attachment technology. The obtained panoramic video data set is digitally collected, the original images and photos are converted into digital format, and the movement trajectory and timestamp information of the inspection personnel are simultaneously collected. The digitally collected data and movement trajectory information are transmitted to the central processing system in real time through the AI ​​workstation; Combining digitally collected data and mobile trajectory information, the inspection trajectory is displayed in the form of lines or marks on the floor plan, and panoramic images and high-definition photos are fused with the inspection path data for display. Based on the fusion display, users can explore different perspectives and areas through an interactive interface.

2. The method for generating an engineering inspection trajectory based on panoramic images according to claim 1, characterized in that: The step of deploying at least one panoramic camera device at the construction site includes: Determine the locations of multiple observation points at the project site. Each observation point corresponds to one or more panoramic camera installation positions. Install panoramic camera equipment according to the selected positions. For any two adjacent devices, the overlap ratio of their field of view is at least P%, where P represents the ratio of the intersection of the fields of view between the two devices to the total field of view of a single device. Based on the adjusted device settings, the panoramic camera device is started to collect images. The device shoots at a fixed frame rate F, where F represents the number of images captured per second. At the same time, the geographic location coordinates (X, Y, Z) of each device are recorded for spatial positioning of subsequent images. During the image acquisition process, the device posture data corresponding to each frame of the image is recorded synchronously, including the rotation angle θ, the tilt angle φ and the yaw angle ψ, and the direction matrix M = R(θ)*T(φ)*P(ψ) of each device relative to the reference coordinate system is calculated, where R, T, and P represent the rotation operations around the three axes, respectively.

3. The method for generating an engineering inspection trajectory based on panoramic images according to claim 2, characterized in that: The method of using the panoramic camera device to automatically inspect along a preset path or a path dynamically generated by an intelligent planning algorithm and collecting panoramic image data in real time includes: Set the inspection starting point and target end point, and plan at least one inspection route based on the map information of the project site. This route consists of continuous coordinate points (X_i, Y_i, Z_i), where i represents the i-th coordinate point on the path; Start the panoramic camera device to move along the defined route and start capturing images at a fixed frame rate F. For each coordinate point (X_i, Y_i, Z_i), calculate the time required for the device to reach that point T_i = D_i / V, where D_i is the distance from the previous point to the current point and V is the moving speed of the device. During the movement, when the device approaches the next coordinate point, adjust the device's attitude parameters (θ_i, φ_i, ψ_i) so that the device's field of view is aligned with the direction of the next coordinate point. The attitude adjustment is based on the formula M_i=R(θ_i)*T(φ_i)*P(ψ_i), where R, T, and P represent the operation of rotating around three axes respectively; After completing the posture adjustment, start recording the panoramic image data of the location immediately, and save the data together with the current position coordinates (X_i, Y_i, Z_i), timestamp T_i and posture parameters (θ_i, φ_i, ψ_i), and then continue to move to the next coordinate point until the inspection route ends.

4. The method for generating an engineering inspection trajectory based on panoramic images according to claim 3, characterized in that: The method of automatically attaching the photo to the inspection track plan of the corresponding position includes: When the inspection device reaches the preset key or problem area, it triggers the shooting command of the mobile terminal device to determine the coordinates (X_j, Y_j, Z_j) of each shooting position P_j; At the specified position P_j, the mobile terminal device captures a high-definition photo I_j with the highest resolution, and records the timestamp T_j of the shooting time and the device posture parameters (θ_j, φ_j, ψ_j). For each photo I_j, calculate its spatial distance D_j = sqrt((X_j-X_i)^2+(Y_j-Y_i)^2+(Z_j-Z_i)^2) from the most recent panoramic image acquisition point, where (X_i, Y_i, Z_i) are the coordinates of the most recent panoramic image acquisition point; Based on the timestamp T_j and spatial distance D_j, associate the high-definition photo I_j with the corresponding inspection track position, and use the formula W_j=exp(-α*D_j^2) to calculate the weight W_j, where α is an adjustment parameter; The high-definition photos I_j and their associated weights W_j, timestamps T_j, coordinates (X_j, Y_j, Z_j) and posture parameters (θ_j, φ_j, ψ_j) are integrated into the dataset of the inspection trajectory plan. This dataset allows users to browse photos by clicking or dragging the interactive interface and intuitively see the relationship between the photos and the actual location.

5. The method for generating an engineering inspection trajectory based on panoramic images according to claim 4, characterized in that: The forming of a panoramic video dataset with a geographic reference comprises: Collect all high-definition photos I_j and their corresponding spatial position coordinates (X_j, Y_j, Z_j), timestamps T_j and posture parameters (θ_j, φ_j, ψ_j). At the same time, obtain the continuous image frames F_i recorded by the panoramic camera device in the same time period and their corresponding geographic location coordinates (X_i, Y_i, Z_i) and timestamps T_i; For each high-definition photo I_j, find the panoramic image frame F_i closest to its timestamp T_j, calculate the time difference ΔT = |T_j-T_i| between the two to ensure the time synchronization between the photo and the image frame, and select the photo and image frame pair that meets the threshold of ΔT < τ, where τ is a preset time tolerance; According to the selected photo and image frame pair, the spatial distance D_ij = sqrt((X_j-X_i)^2+(Y_j-Y_i)^2+(Z_j-Z_i)^2) is calculated using the spatial position coordinates (X_j, Y_j, Z_j) and (X_i, Y_i, Z_i), and the association weight W_ij is calculated by the formula W_ij = exp(-β*D_ij^2), where β is the adjustment parameter; The identified photo and image frame pairs, together with their weights Wij, timestamps Tj / Ti, coordinates (Xj, Yj, Zj) / (Xi, Yi, Zi) and pose parameters (θj, φj, ψj), are integrated into a unified dataset.

6. The method for generating an engineering inspection trajectory based on panoramic images according to claim 5, characterized in that: The panoramic video data set is digitally collected, the original images and photos are converted into digital format, and the movement trajectory and timestamp information of the inspection personnel are synchronously collected, including: Receive a panoramic video dataset with georeferences, and for each pair of high-definition photos I_j and panoramic image frames F_i, extract the binary data streams B_j and B_i of the original image and photo, and at the same time, obtain the associated timestamps T_j / T_i, spatial position coordinates (X_j, Y_j, Z_j) / (X_i, Y_i, Z_i) and posture parameters (θ_j, φ_j, ψ_j); Convert the binary data streams B_j and B_i into compressed formats C_j and C_i using the formula C_x = Compress(B_x, r), where x represents j or i, Compress represents the compression operation, and r is the compression rate. At the same time, keep the original precision of the timestamp T_j / T_i, coordinates (X_j, Y_j, Z_j) / (X_i, Y_i, Z_i) and attitude parameters (θ_j, φ_j, ψ_j) unchanged; During the data processing, the movement trajectory of the patrol personnel is recorded synchronously. The position coordinates (X_p, Y_p, Z_p) and timestamp T_p of the patrol personnel are sampled regularly through the positioning device integrated in the patrol equipment. The time interval ΔT_p = T_p-T_(p-1) between two adjacent samples is calculated to ensure the continuity of the trajectory and time synchronization. For each timestamp T_p, the corresponding posture parameters (θ_p, φ_p, ψ_p) of the patrol personnel are recorded to form a complete movement trajectory record. The compressed image data C_j / C_i, timestamp T_j / T_i, spatial position coordinates (X_j, Y_j, Z_j) / (X_i, Y_i, Z_i), posture parameters (θ_j, φ_j, ψ_j) and the inspection personnel’s movement trajectory (X_p, Y_p, Z_p), timestamp T_p and posture parameters (θ_p, φ_p, ψ_p) are integrated into one data set.

7. The method for generating an engineering inspection trajectory based on panoramic images according to claim 6, characterized in that: The digitally collected data and movement trajectory information are transmitted to the central processing system in real time through the AI ​​workstation, including: Extract the compressed image data C_j / C_i, timestamp T_j / T_i, spatial position coordinates (X_j, Y_j, Z_j) / (X_i, Y_i, Z_i), posture parameters (θ_j, φ_j, ψ_j), and the movement trajectory (X_p, Y_p, Z_p), timestamp T_p and posture parameters (θ_p, φ_p, ψ_p) of the inspector from the integrated data set; Pack the data into transmission packets D. Each transmission packet D contains one or more image frames and their associated information, and is accompanied by a checksum (D) = Hash (Summary (D)), where Summary (D) represents a summary of the contents of the transmission packet, and Hash is a hash function; The generated transmission packet D is sent to the central processing system in real time. After each transmission, the sending time and the receiving confirmation time are recorded, and the transmission delay ΔT_transmit = T_receive-T_send is calculated. For each transmission packet D, its corresponding transmission delay ΔT_transmit is recorded to form a transmission log; T_send is the timestamp of the data packet starting to be transmitted from the sending end, and T_receive is the timestamp of the data packet arriving at the receiving end and being successfully received; After the central processing system receives the transmission package D, it unpacks and verifies the checksum (D). Then, according to the timestamps T_j / T_i and T_p, the image data C_j / C_i and the inspection personnel's movement trajectory (X_p, Y_p, Z_p) are synchronized and integrated into the database. The timestamp T_x is adjusted by the formula Sync(T_x,ΔT_transmit)=Adjust(T_x+ΔT_transmit), where x represents j / i or p, and T_x represents the original timestamp.

8. The method for generating an engineering inspection trajectory based on panoramic images according to claim 7, characterized in that: The fusion display of panoramic images and high-definition photos with inspection path data includes: Obtain all the data transmitted and integrated from the central processing system, and draw the inspection route in the form of lines on the plan of the project site based on the inspection personnel's movement trajectory (X_p, Y_p, Z_p). For each trajectory point P_p, use the formula P_line=Line(P_(p-1),P_p), where Line represents the line segment connecting two consecutive trajectory points, forming a coherent inspection route map. At the same time, record the timestamp T_p of each trajectory point; According to the inspection route map, the panoramic image frame F_i and the high-definition photo I_j are associated with the corresponding trajectory point P_p, using the spatial position coordinates (X_j, Y_j, Z_j) / (X_i, Y_i, Z_i) and timestamp T_j / T_i; The closest trajectory point P_p is found by the formula D_associate=min(Distance(P_p,(X_x,Y_x,Z_x))), where x represents j or i, and Distance represents the distance calculation between two points; Marks are added to the corresponding positions on the inspection path map to display the panoramic image frame F_i and the high-definition photo I_j. For each mark M_x, the timestamp T_x of the shooting time and the posture parameters (θ_x, φ_x, ψ_x) are displayed to provide users with interactive browsing functions. The formula Overlay(M_x, P_line) = Merge(M_x, Line(P_(p-1), P_p)) is used to merge the mark with the inspection path segment to form a comprehensive view.

9. The method for generating an engineering inspection trajectory based on panoramic images according to claim 8, characterized in that: The above-mentioned basis fusion display enables users to explore different perspectives and areas through an interactive interface, including: Construct a map view based on the integration of patrol route map and image data. In this map view, each patrol track point P_p and its associated panoramic image frame F_i and high-definition photo I_j are marked. For each mark M_x, its visible range V_x is defined as a sphere with a center at (X_x, Y_x, Z_x) and a radius of R, expressed as V_x = Sphere((X_x, Y_x, Z_x), R); Based on the established map view, zoom and pan operations are provided to allow the user to focus on the area of ​​interest. When the user selects a specific marker M_x, the system calculates the distance D_view between the marker and the user's current viewpoint and uses the formula ZoomLevel = f(D_view), where f is a mapping function to determine the appropriate zoom level; According to the user operation, the multi-view browsing mode is activated. For the selected marker M_x, its posture parameters (θ_x, φ_x, ψ_x) are used to construct the transformation matrix T_x to adjust the angle of image display; Introduce an instant information prompt box. When the user hovers or clicks a marker M_x, relevant information is immediately displayed, including but not limited to the timestamp T_x, location coordinates (X_x, Y_x, Z_x), and a brief description.

10. A system for generating engineering inspection trajectories based on panoramic images, characterized in that: include: A panoramic camera equipment deployment module, used to deploy at least one panoramic camera equipment at the construction site, wherein the panoramic camera equipment is equipped with multiple cameras for capturing environmental images at 360 degrees without blind spots, and the panoramic camera equipment is used to perform automatic inspections along a preset path or a path dynamically generated by an intelligent planning algorithm, and collect panoramic image data in real time; The high-definition photo shooting and mounting module is used to take photos of key parts and problem parts of the project site using mobile terminal devices during the automatic inspection process, and automatically mount the photos to the inspection track plan of the corresponding position. Based on the spatial position information of the photos and panoramic images, a panoramic video data set with geographical reference is formed through the spatial position and plan mounting technology; The digital acquisition and synchronous recording module is used to digitally acquire the acquired panoramic video data set, convert the original images and photos into digital format, and synchronously acquire the movement trajectory and timestamp information of the inspection personnel. The digital acquisition data and movement trajectory information are transmitted to the central processing system in real time through the AI ​​workstation; The inspection track visualization display module is used to combine digital collection data and movement track information to display the inspection track in the form of lines or marks on the floor plan, and to fuse panoramic images and high-definition photos with the inspection path data for display. Based on the fusion display, the interactive interface enables users to explore different perspectives and areas.

Images