Machine dog autonomous trace detection system based on panoramic image reproduction

Abstract

The present application relates to a kind of machine dog autonomous trace detection systems based on panoramic image reproduction, belong to criminal investigation trace detection technical field.Wherein, the method includes: by multispectral camera, obtain surface condition image data, by laser radar, obtain three-dimensional space point cloud data;Image data and point cloud data of acquisition are combined with the position and attitude information of machine dog to generate associated data set;Surface condition image data is optimized, and the data after processing is associated to the three-dimensional point corresponding to spatial position to generate multispectral point cloud data set;Based on three-dimensional space point cloud data, generate surface condition geometric model, and color, texture information is mapped to model by texture mapping technology;Optimize output 3D model to visual representation, and 3D model is transmitted to remote monitoring terminal by wireless communication.It realizes the scene reproduction of trace detection site, improves the efficiency, accuracy and scene adaptability of trace detection work.

Description

Technical Field

[0001] This invention belongs to the field of criminal investigation trace evidence technology, specifically relating to an autonomous trace evidence detection system for a robot dog based on panoramic image reconstruction. Background Technology

[0002] The situations at trace evidence examination sites are complex and diverse, and environmental factors pose significant obstacles to the work. As the judicial system increasingly demands higher accuracy in evidence, the standards for detecting microscopic traces in trace evidence examination are becoming increasingly stringent. Traditional static microscopic equipment struggles to flexibly adjust its position and angle when faced with complex scenes, making it impossible to continuously and accurately photograph and analyze traces at different locations and angles. Furthermore, due to the uncertainty of the traces' location at the scene, traditional equipment cannot achieve comprehensive, all-around coverage of the traces while maintaining high-precision imaging.

[0003] In recent years, robot dog technology has developed rapidly, and its high mobility, flexibility, and environmental adaptability have brought new solutions to trace evidence inspection. Mounting a trace evidence inspection gimbal on a robot dog allows microscopic equipment to break free from the limitations of traditional fixed positions, enabling dynamic tracking and examination of traces at the scene. During trace evidence inspection, the surfaces of trace evidence are often uneven, and the robot dog experiences vibrations and posture changes during movement, requiring the microscopic equipment to have real-time dynamic focusing capabilities.

[0004] Traditional fixed-focusing methods cannot adapt to constantly changing exploration environments, easily leading to blurred images, loss of key details, and difficulty in achieving high-precision attitude adjustment and focus control, thus failing to meet the requirements of photomicrography for detailed observation of tiny objects. Furthermore, existing focusing systems are slow and inaccurate when facing dynamically changing scenes, and cannot track the depth changes of target objects in a timely and accurate manner. These defects result in poor quality of acquired trace evidence images, affecting the accuracy and reliability of trace evidence results. Summary of the Invention

[0005] To address the aforementioned problems in the existing technology, this invention provides an autonomous trace inspection system for robotic dogs based on panoramic image reconstruction. The objective of this invention can be achieved through the following technical solutions:

[0006] The autonomous trace inspection system for robot dogs based on panoramic image reconstruction includes:

[0007] The data acquisition module acquires surface condition image data of the trace inspection site in the spectral band through a multispectral camera, and collects spectral reflectance information of the ground objects at the trace inspection site; it also acquires three-dimensional spatial point cloud data of the trace inspection site by emitting laser beams and receiving echoes through a lidar.

[0008] The data association module acquires the robot dog's position and attitude information based on the BeiDou positioning system and attitude sensors. Under the control of a hardware synchronization controller, the multispectral camera and the lidar simultaneously acquire the 3D spatial point cloud data and the surface condition image data. A timestamp is added to the acquired 3D spatial point cloud data and surface condition image data based on a time system. Combined with the robot dog's position and attitude information, an associated dataset is generated. By performing spectral enhancement and data correction processing on the received surface condition image data, and based on the associated dataset, the spectral information of the processed surface condition image data is associated with the corresponding 3D points in space, generating a multispectral point cloud dataset containing spatial coordinates and spectral information.

[0009] The texture mapping module generates a geometric model of the surface condition of the trace inspection site based on the three-dimensional spatial point cloud data. Based on the multispectral point cloud dataset, it uses texture mapping technology to map the color and texture information in the surface condition image data onto the geometric model of the surface condition, and outputs a visual representation of the trace inspection site in three-dimensional space.

[0010] The terminal interaction module optimizes the visualization by removing holes, cracks, and defects, and smoothing the surface of the visualization, outputting a 3D model, and transmitting the 3D model to a remote monitoring terminal via wireless communication.

[0011] As a preferred embodiment of the present invention, the multispectral camera in the data acquisition module is fixed to the back of the robot dog by a shockproof bracket; based on the instructions of the hardware synchronization controller, the multispectral camera and the lidar are started simultaneously; based on the preset path of the robot dog, the multispectral camera captures images of the surface condition of the trace inspection site at a set frame rate and adds timestamps to the surface condition images; combined with the robot dog's posture information at the time of capture, a metadata file containing image data, spectral band information, timestamps, and camera intrinsic parameters is output.

[0012] Specifically, the method for acquiring the three-dimensional spatial point cloud data in the data acquisition module is as follows:

[0013] A laser emitter emits a laser of a specific wavelength and power, and a laser receiver receives the laser reflected from the surface of an object. Based on a pre-defined task plan and the distribution of evidence at the scene, the laser controller adjusts the emission angle and scanning mode of the laser emitter.

[0014] The data collection circuit is connected to the lidar receiver, converts the received analog signal into a digital signal, and outputs three-dimensional spatial point cloud data of the trace inspection site by performing intensity correction, distance calculation and data filtering on the digital signal.

[0015] Specifically, the data correction processing in the data association module includes geometric correction and time synchronization correction, and the correction method is as follows:

[0016] Based on the associated dataset and the intrinsic and extrinsic parameters of the multispectral camera, ground control points are deployed in the trace detection acquisition area. The surface condition image coordinates and 3D point cloud coordinates of the ground control points are extracted based on the surface condition image data and the 3D point cloud data. A geometric transformation matrix is ​​used to associate the surface condition image coordinates and 3D point cloud coordinates with the world coordinates of the ground control points, establishing a coordinate association architecture. Based on this coordinate association architecture, the surface condition image is resampled and projected using a geometric correction model to complete geometric correction.

[0017] Based on the common features in the surface condition image data and the three-dimensional spatial point cloud data, the collected surface condition image data and three-dimensional spatial point cloud data are time-synchronized and corrected by the nearest neighbor interpolation method. The time-synchronized and corrected surface condition image data and three-dimensional spatial point cloud data are then organized into a structured array containing the pixel coordinates of the surface condition image, spectral information, three-dimensional spatial point cloud coordinates, and corresponding timestamps.

[0018] Specifically, the generation of the multispectral point cloud dataset in the data association module includes:

[0019] By performing spectral enhancement and data correction processing on the received surface condition image data, a surface condition image containing reflectance information and spatial coordinates is output; the received three-dimensional spatial point cloud data is processed by denoising and filtering to retain the effective point cloud, and a continuous point cloud in the global coordinate system is generated based on the effective point cloud through registration.

[0020] Based on the corrected surface condition image, spectral reflectance information of the pixels in the surface condition image is extracted pixel by pixel. Based on the spectral reflectance information, the actual spatial ray corresponding to the pixel in the surface condition image is calculated using a projection model. The three-dimensional points in the three-dimensional spatial point cloud data are traversed, and the point closest to the actual spatial ray is marked as the matching point. The spectral information of the pixel in the surface condition image is assigned to the corresponding three-dimensional point in the three-dimensional spatial point cloud data. After point-by-point matching, a multispectral point cloud dataset containing spatial coordinates and spectral reflectance data of the three-dimensional points is formed.

[0021] Specifically, the surface condition geometric model in the texture mapping module is generated as follows:

[0022] The three-dimensional spatial point cloud data is optimized by removing redundant noise points and invalid data. Based on the optimized three-dimensional spatial point cloud data, global registration of the three-dimensional spatial point cloud is performed in conjunction with the associated dataset, unifying the point clouds from different perspectives into the global coordinate system.

[0023] Based on the density and distribution of 3D spatial point cloud data, the triangulation rules are determined by the triangulation method, and the 3D spatial point cloud data is transformed into a triangular mesh model. The size and shape of the triangles are adjusted based on the triangulation rules. The triangular mesh model is then topologically optimized by removing non-manifold edges and isolated faces, and a surface condition geometric model is output.

[0024] Specifically, the texture mapping technology in the texture mapping module includes:

[0025] The triangular mesh model of the surface condition geometric model is assigned horizontal and vertical coordinates, and the surface of the three-dimensional mesh model is unfolded into two-dimensional planar coordinates; based on the premise of minimizing stretching deformation, the triangular facets and regions on the two-dimensional horizontal and vertical planes are mapped and paired through a mapping unfolding algorithm;

[0026] Based on the associated dataset, a spatial coordinate transformation relationship between the multispectral camera and the lidar is established; based on the projection matrix of the multispectral camera, the three-dimensional spatial projection coordinates of the multispectral image pixels are calculated, and the corresponding geometric model mesh patches are determined based on the three-dimensional spatial projection coordinates; based on the horizontal-vertical coordinates of the geometric model mesh patches, the positions of pixel color and texture information on the two-dimensional horizontal-vertical plane are determined.

[0027] Based on the matching results of the geometric model mesh patches, pixel information of the corresponding region of the surface condition image is collected and assigned to the corresponding positions in the horizontal-vertical plane.

[0028] Specifically, the result of mapping using the texture mapping technique is optimized using the following method:

[0029] Feathering technology is used to similarize the texture edges of adjacent horizontal and vertical blocks. By adjusting the color values ​​of the pixels at the edges of the horizontal and vertical blocks, the color difference at the seams is reduced. A texture stretching compensation algorithm is used to correct the texture deformation in areas of abrupt curvature changes. Combined with the reflectivity information in the multispectral point cloud dataset, texture enhancement is performed on specific trace detection target areas.

[0030] Based on the accuracy verification procedure, a geometric model mesh patch is randomly selected, and the color deviation between the texture of the geometric model mesh patch and the texture of the actual surface condition image is compared. If the deviation exceeds the threshold, a correction procedure is triggered to adjust the horizontal-vertical coordinates and perform resampling and mapping.

[0031] Specifically, the method for removing holes and cracks from the visual representation in the terminal interaction module is as follows:

[0032] By traversing the triangular facets of the surface condition geometric model, the boundary edges of the triangular facets that are not shared by adjacent facets are detected. When the boundary edges form a closed loop, it is determined to be a hole region, and the coordinates of the boundary vertex, the outline shape, and the area of ​​the hole are recorded. The angle between the normal vectors of adjacent triangular facets is calculated. When the angle between the normal vectors exceeds a preset threshold and the edge distance is less than a preset distance, it is determined to be a crack region, and the start and end points and extension path of the crack are located.

[0033] Select the effective vertex closest to the boundary inside the hole boundary as the seed point, and generate new triangular patches gradually towards the hole region with the seed point as the center, based on the principle of triangulation.

[0034] Based on the detected crack region, the crack width is reduced to within a preset width by adjusting the position coordinates of the vertices on both sides of the crack; then, the boundary edges on both sides of the crack are merged into a shared edge by using an edge fusion algorithm to generate a new triangular patch to connect the crack region.

[0035] Specifically, the data transmission method in the terminal interaction module is as follows:

[0036] Structured datasets are generated from information obtained through trace evidence analysis;

[0037] After the robot dog establishes a connection with the remote monitoring terminal, it negotiates transmission parameters through a handshake protocol. The robot dog monitors the transmission progress, signal strength, and battery level in real time. If the transmission is interrupted, it automatically saves the transmission breakpoint information and resumes transmission after the connection is restored. If the battery level is lower than the threshold, it prioritizes the transmission of key trace information, pauses the transmission and records the status, and resumes transmission after charging.

[0038] The remote monitoring terminal receives data through dedicated client software and loads a 3D model into a 3D visualization engine; the structured dataset is associated with the spatial coordinates of the 3D model, and spectral curves, size parameters, and analysis conclusions are displayed in the trace area of ​​the 3D model.

[0039] The beneficial effects of this invention are as follows:

[0040] By deeply fusing the spectral reflectance of multispectral images with the three-dimensional spatial information of point cloud data, the generated multispectral point cloud dataset contains both the three-dimensional coordinates of ground objects and reflectance characteristics across 3-10 bands. This dual data support of spatial location and material composition enables trace evidence investigators not only to locate traces but also to accurately identify targets such as bloodstains, fibers, and tool marks through spectral comparison, solving the problem of traditional visual imaging's difficulty in identifying substances of similar colors.

[0041] By employing a hardware synchronization controller and timestamp technology, strict spatiotemporal binding of multispectral data, point cloud data, and the robot dog's posture and position information is achieved. Combined with geometric and temporal synchronization correction of ground control points, the matching error between image pixels and the 3D point cloud is ensured, avoiding analytical biases caused by spatiotemporal misalignment of multi-source data. This provides a high-precision data foundation for subsequent modeling and trace analysis.

[0042] The robot dog is capable of overcoming obstacles and adapting to narrow spaces and complex terrains. Its multispectral and point cloud acquisition devices can reach areas that are difficult for traditional manual or fixed equipment to access. With preset path planning and LiDAR scanning mode adjustment, it can achieve comprehensive data collection in complex indoor and outdoor scenes, reducing the problem of missing traces due to poor on-site accessibility.

[0043] Based on point cloud data, a surface condition geometric model is generated. Combined with texture mapping technology, it can completely restore the geometric shape and surface texture details of the scene. Through model optimization and cropping measurement functions, it can not only preserve the microscopic features of key traces, but also generate a precise 3D visualization scene that meets the needs of trace inspection, realizing the long-term retention and retrospective analysis of on-site digital twins.

[0044] Utilizing wireless communication technology, optimized 3D models and structured trace evidence information can be transmitted to remote terminals in real time. Remote experts can then use a 3D visualization engine to perform model rotation, sectioning, and interactive trace analysis, breaking through the time and space limitations of relying on local personnel for on-site investigation. This is particularly suitable for cross-regional collaborative analysis of major cases, improving trace evidence response speed and decision-making efficiency.

[0045] Holes are repaired using a region growing and filling algorithm, surface noise is smoothed using Gaussian filtering, and topology checks and accuracy verification ensure that the output model is free of geometric defects while retaining key trace features. The measurement function can directly obtain trace size parameters, avoiding the subjective errors of traditional manual measurements.

[0046] The multispectral camera supports extended visible and near-infrared bands, the lidar allows for adjustable scanning modes and wavelengths, and dynamic focusing control technology adapts to different magnification microscopy requirements. This modular architecture enables the system to flexibly upgrade hardware or algorithms according to the trace evidence inspection scenario, providing scalability to continuously adapt to the development of trace evidence inspection technology.

[0047] In summary, this invention significantly improves the accuracy, efficiency, and adaptability of trace evidence inspection through in-depth innovation in multi-source data fusion, intelligent acquisition, and precise modeling. It provides a systematic solution for trace identification, digital preservation, and remote collaborative analysis in complex scenarios, and has significant practical value and promotional significance. Attached Figure Description

[0048] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0049] Figure 1 This is a flowchart illustrating an autonomous trace inspection system for a robot dog based on panoramic image reconstruction according to the present invention.

[0050] Figure 2 This is a block diagram of the multispectral point cloud dataset structure of the present invention;

[0051] Figure 3 This is a schematic diagram illustrating the application process of the texture mapping technology of the present invention. Detailed Implementation

[0052] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the following detailed description of the specific implementation methods, structures, features and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided.

[0053] Please see Figure 1 An autonomous trace inspection system for robot dogs based on panoramic image reconstruction includes:

[0054] The data acquisition module acquires surface condition image data of the trace inspection site in the spectral band through a multispectral camera, and collects spectral reflectance information of the ground objects at the trace inspection site; it also acquires three-dimensional spatial point cloud data of the trace inspection site by emitting laser beams and receiving echoes through a lidar.

[0055] The data association module acquires the robot dog's position and attitude information based on the BeiDou positioning system and attitude sensors. Under the control of a hardware synchronization controller, the multispectral camera and the lidar simultaneously acquire the 3D spatial point cloud data and the surface condition image data. A timestamp is added to the acquired 3D spatial point cloud data and surface condition image data based on a time system. Combined with the robot dog's position and attitude information, an associated dataset is generated. By performing spectral enhancement and data correction processing on the received surface condition image data, and based on the associated dataset, the spectral information of the processed surface condition image data is associated with the corresponding 3D points in space, generating a multispectral point cloud dataset containing spatial coordinates and spectral information.

[0056] The texture mapping module generates a geometric model of the surface condition of the trace inspection site based on the three-dimensional spatial point cloud data. Based on the multispectral point cloud dataset, it uses texture mapping technology to map the color and texture information in the surface condition image data onto the geometric model of the surface condition, and outputs a visual representation of the trace inspection site in three-dimensional space.

[0057] The terminal interaction module optimizes the visualization by removing holes, cracks, and defects, and smoothing the surface of the visualization, outputting a 3D model, and transmitting the 3D model to a remote monitoring terminal via wireless communication.

[0058] Specifically, in the data acquisition module, the multispectral camera is fixed to the back of the robot dog via a shockproof bracket; based on the instructions of the hardware synchronization controller, the multispectral camera and the lidar are started simultaneously; based on the robot dog's preset path, the multispectral camera captures images of the surface condition of the trace inspection site at a set frame rate and adds timestamps to the surface condition images; combined with the robot dog's posture information at the time of capture, a metadata file containing image data, spectral band information, timestamps, and camera intrinsic parameters is output.

[0059] In this embodiment, the multispectral camera is fixed to the back of the robot dog using a shockproof bracket. Compared to traditional RGB cameras, multispectral cameras can capture spectral reflectance information of an object's surface in multiple discrete narrow bands (such as from visible light to near infrared). This information can reveal potential traces that are indistinguishable to the human eye, such as trace amounts of blood, bodily fluid residue, differences between different inks, or original information under camouflage coatings, greatly enhancing the ability to detect and identify trace evidence and compensating for the limitations of visible light imaging.

[0060] The output timestamped image data, along with point cloud data and robot dog positioning posture data, provides a spatiotemporal reference for the fusion of spectral information and 3D point cloud in the data association module, serving as the core input for generating the multispectral point cloud dataset. The preprocessed image data contains rich color and texture information, providing material for texture mapping in the texture mapping module. This allows the final 3D model to intuitively present the material distribution characteristics of the surface layer, assisting trace evidence inspectors in remote analysis. The multispectral camera, through precise spectral imaging and synchronous control, establishes the foundation for the "spectral attribute-spatial location" association at the trace evidence inspection site, providing crucial hardware support for the intelligent and precise operation of the robot dog trace evidence inspection system.

[0061] Specifically, the method for acquiring the three-dimensional spatial point cloud data in the data acquisition module is as follows:

[0062] A laser emitter emits a laser of a specific wavelength and power, and a laser receiver receives the laser reflected from the surface of an object. Based on a pre-defined task plan and the distribution of evidence at the scene, the laser controller adjusts the emission angle and scanning mode of the laser emitter.

[0063] The data collection circuit is connected to the lidar receiver, converts the received analog signal into a digital signal, and outputs three-dimensional spatial point cloud data of the trace inspection site by performing intensity correction, distance calculation and data filtering on the digital signal.

[0064] In this embodiment, the laser controller automatically adjusts the scanning mode according to the distribution of physical evidence at the scene (such as areas suspected of bloodstains): ordinary areas are scanned at a low frequency of 10Hz, and key areas are switched to high frequency scanning at 20Hz to ensure that the point cloud density of the trace area is ≥200 points / ㎡.

[0065] As the core of 3D data acquisition, lidar (LiDAR) emits laser beams and precisely measures their return time, enabling rapid acquisition of large-scale, high-density 3D point cloud data from the site, providing a foundation for constructing the macroscopic geometric framework of the site. According to the data acquisition module, the laser controller can flexibly adjust the scanning mode and angle of the laser emitter based on the distribution of evidence, achieving focused scanning of key areas.

[0066] Specifically, the data correction processing in the data association module includes geometric correction and time synchronization correction, and the correction method is as follows:

[0067] Based on the associated dataset and the intrinsic and extrinsic parameters of the multispectral camera, ground control points are deployed in the trace detection acquisition area. The surface condition image coordinates and 3D point cloud coordinates of the ground control points are extracted based on the surface condition image data and the 3D point cloud data. A geometric transformation matrix is ​​used to associate the surface condition image coordinates and 3D point cloud coordinates with the world coordinates of the ground control points, establishing a coordinate association architecture. Based on this coordinate association architecture, the surface condition image is resampled and projected using a geometric correction model to complete geometric correction.

[0068] Based on the common features in the surface condition image data and the three-dimensional spatial point cloud data, the collected surface condition image data and three-dimensional spatial point cloud data are time-synchronized and corrected by the nearest neighbor interpolation method. The time-synchronized and corrected surface condition image data and three-dimensional spatial point cloud data are then organized into a structured array containing the pixel coordinates of the surface condition image, spectral information, three-dimensional spatial point cloud coordinates, and corresponding timestamps.

[0069] Specifically, the generation of the multispectral point cloud dataset in the data association module includes:

[0070] By performing spectral enhancement and data correction processing on the received surface condition image data, a surface condition image containing reflectance information and spatial coordinates is output; the received three-dimensional spatial point cloud data is processed by denoising and filtering to retain the effective point cloud, and a continuous point cloud in the global coordinate system is generated based on the effective point cloud through registration.

[0071] Based on the corrected surface condition image, spectral reflectance information of the pixels in the surface condition image is extracted pixel by pixel. Based on the spectral reflectance information, the actual spatial ray corresponding to the pixel in the surface condition image is calculated using a projection model. The three-dimensional points in the three-dimensional spatial point cloud data are traversed, and the point closest to the actual spatial ray is marked as the matching point. The spectral information of the pixel in the surface condition image is assigned to the corresponding three-dimensional point in the three-dimensional spatial point cloud data. After point-by-point matching, a multispectral point cloud dataset containing spatial coordinates and spectral reflectance data of the three-dimensional points is formed.

[0072] Please see Figure 2 In this embodiment, a topological check is performed on the generated initial dataset to ensure that the spectral reflectance variation of adjacent point cloud data points conforms to the actual distribution pattern of ground features (such as the standard deviation of reflectance of the same object surface ≤ 5%). Abnormal points with sudden changes are then matched a second time or marked with low confidence. Combined with measured data from a standard reflectance reference board (such as a 50% gray board), the reflectance values ​​of the dataset are globally calibrated to ensure that the reflectance deviation of key bands is ≤ 1%, thereby improving the comparability of data from different regions.

[0073] Through hardware synchronization control and attitude parameter correction, the time difference between spectral information and spatial coordinates is ensured to be ≤5ms, and the spatial offset is ≤3 pixels, avoiding misattribution errors. It supports full coverage from centimeter-level fine traces (such as fibers and scratches) to meter-level scene ranges, and the point cloud density can be adjusted according to needs (e.g., single-point spacing in key areas ≤1cm, background areas ≤5cm). A reserved band expansion interface allows for the addition of ultraviolet or short-wave infrared bands as needed for trace detection, improving the identification capability of special substances (such as traces marked with fluorescent reagents).

[0074] By utilizing both location and spectral attributes, precise localization and component analysis of trace evidence targets can be achieved. For example, if a point cloud region with reflectivity characteristics matching bloodstains is found in grass, its three-dimensional coordinates (X=123.45m, Y=67.89m, Z=0.05m) can be directly obtained, providing precise guidance for on-site investigation.

[0075] It provides a spectral texture data source for texture mapping, enabling 3D models to not only present geometric shapes but also visualize material distribution through reflectance differences (e.g., using pseudo-color to encode regions with different reflectance to highlight suspicious traces). The dataset can be transmitted to remote terminals via wireless communication (5G / WiFi6), supporting offline spectral comparison (e.g., matching with known material spectral libraries) and spatial measurements, achieving "non-contact trace detection and analysis".

[0076] Specifically, the surface condition geometric model in the texture mapping module is generated as follows:

[0077] The three-dimensional spatial point cloud data is optimized by removing redundant noise points and invalid data. Based on the optimized three-dimensional spatial point cloud data, global registration of the three-dimensional spatial point cloud is performed in conjunction with the associated dataset, unifying the point clouds from different perspectives into the global coordinate system.

[0078] Based on the density and distribution of 3D spatial point cloud data, the triangulation rules are determined by the triangulation method, and the 3D spatial point cloud data is transformed into a triangular mesh model. The size and shape of the triangles are adjusted based on the triangulation rules. The triangular mesh model is then topologically optimized by removing non-manifold edges and isolated faces, and a surface condition geometric model is output.

[0079] In this embodiment, the model's spatial scale is a 1:1 replica of the scene. The accuracy of 3D distance, angle, and area measurements between any two points is consistent with the original point cloud (distance accuracy ≤ 1cm, angle ≤ 0.5°), which can be directly used for the quantitative analysis of the size of trace evidence targets (such as measuring scratch length and indentation depth). The topological structure of the mesh surface supports interactive operations, such as cropping based on rectangular and polygonal regions (focusing on suspected areas), cross-sectional cutting (observing the internal structure of the surface), and 3D roaming, meeting the observation needs of trace evidence investigators from different perspectives at the scene. The model's mesh patches have a coordinate mapping relationship with the spectral information in the multispectral point cloud dataset. In the texture mapping stage of the texture mapping module, the color and texture of the multispectral image can be accurately attached to the corresponding mesh area through UV coordinates, upgrading the model from a "pure geometric skeleton" to a "digital twin with material properties."

[0080] The surface geometry model and the multispectral point cloud dataset are deeply correlated through "coordinate mapping." Each vertex coordinate in the model corresponds to a 3D point coordinate in the multispectral point cloud dataset, ensuring spatial consistency between the two. The coverage area of ​​the mesh patches perfectly matches the distribution range of the point cloud data, providing a regional division basis for mapping spectral information from "points" to "surfaces." The model's topology provides boundary references for stitching multispectral textures, avoiding image distortion caused by geometric discontinuities during texture mapping. This correlation mechanism allows the geometric model to retain the spatial accuracy of the point cloud data while providing a structured framework for attaching spectral attributes, serving as a crucial link between "spatial geometry" and "material properties."

[0081] Please see Figure 3 Specifically, the texture mapping technology in the texture mapping module includes:

[0082] The triangular mesh model of the surface condition geometric model is assigned horizontal and vertical coordinates, and the surface of the three-dimensional mesh model is unfolded into two-dimensional planar coordinates; based on the premise of minimizing stretching deformation, the triangular facets and regions on the two-dimensional horizontal and vertical planes are mapped and paired through a mapping unfolding algorithm;

[0083] Based on the associated dataset, a spatial coordinate transformation relationship between the multispectral camera and the lidar is established; based on the projection matrix of the multispectral camera, the three-dimensional spatial projection coordinates of the multispectral image pixels are calculated, and the corresponding geometric model mesh patches are determined based on the three-dimensional spatial projection coordinates; based on the horizontal-vertical coordinates of the geometric model mesh patches, the positions of pixel color and texture information on the two-dimensional horizontal-vertical plane are determined.

[0084] Based on the matching results of the geometric model mesh patches, pixel information of the corresponding region of the surface condition image is collected and assigned to the corresponding positions in the horizontal-vertical plane.

[0085] In this embodiment, slight posture jitter may occur during the robot dog's movement, causing spatiotemporal deviation between the image and the point cloud. By using precise timestamp matching (synchronization error ≤1ms) and posture interpolation algorithms, camera extrinsic parameters are corrected in real time to ensure spatiotemporal consistency between the image and the point cloud. To address the texture stretching and deformation caused by uneven outdoor terrain, a geometry-adaptive texture sampling strategy is adopted, increasing sampling points in high-curvature areas (sampling density increased by 2-3 times) to reduce texture distortion. Furthermore, to address the uneven texture brightness caused by differences in brightness between direct sunlight and shadow areas, a lighting model in radiometric correction is used to perform brightness equalization processing on the image, ensuring that the texture brightness deviation of the same material in different lighting areas is ≤5%.

[0086] Specifically, the result of mapping using the texture mapping technique is optimized using the following method:

[0087] Feathering technology is used to similarize the texture edges of adjacent horizontal and vertical blocks. By adjusting the color values ​​of the pixels at the edges of the horizontal and vertical blocks, the color difference at the seams is reduced. A texture stretching compensation algorithm is used to correct the texture deformation in areas of abrupt curvature changes. Combined with the reflectivity information in the multispectral point cloud dataset, texture enhancement is performed on specific trace detection target areas.

[0088] Based on the accuracy verification procedure, a geometric model mesh patch is randomly selected, and the color deviation between the texture of the geometric model mesh patch and the texture of the actual surface condition image is compared. If the deviation exceeds the threshold, a correction procedure is triggered to adjust the horizontal-vertical coordinates and perform resampling and mapping.

[0089] Specifically, the method for removing holes and cracks from the visual representation in the terminal interaction module is as follows:

[0090] By traversing the triangular facets of the surface condition geometric model, the boundary edges of the triangular facets that are not shared by adjacent facets are detected. When the boundary edges form a closed loop, it is determined to be a hole region, and the coordinates of the boundary vertex, the outline shape, and the area of ​​the hole are recorded. The angle between the normal vectors of adjacent triangular facets is calculated. When the angle between the normal vectors exceeds a preset threshold and the edge distance is less than a preset distance, it is determined to be a crack region, and the start and end points and extension path of the crack are located.

[0091] Select the effective vertex closest to the boundary inside the hole boundary as the seed point, and generate new triangular patches gradually towards the hole region with the seed point as the center, based on the principle of triangulation.

[0092] Based on the detected crack region, the crack width is reduced to within a preset width by adjusting the position coordinates of the vertices on both sides of the crack; then, the boundary edges on both sides of the crack are merged into a shared edge by using an edge fusion algorithm to generate a new triangular patch to connect the crack region.

[0093] In this embodiment, cracks are eliminated and mesh continuity is restored through vertex adjustment and edge fusion. Elastic deformation correction is applied to the vertex coordinates on both sides of the crack, and the optimal vertex movement is calculated using the least squares method, reducing the crack width from the initial 0.3-1cm to ≤0.1cm. During the adjustment process, the vertex movement direction is constrained to follow the crack normal direction to avoid affecting the geometry of surrounding meshes (such as the continuity of grass texture).

[0094] An edge fusion algorithm is used to merge the boundary edges on both sides of the crack into a shared edge, remove duplicate vertices, and generate new triangular patches to connect the crack region. For example, the edge edges (V1, V2) on the left side of the crack and (V3, V4) on the right side are merged into a shared edge (V1, V3) using vertex welding technology, and a new patch (V1, V3, V5) is generated to connect the meshes on both sides, ensuring that there are no topological faults in the crack region after fusion. The repaired crack region is then locally smoothed using a Laplacian smoothing algorithm to adjust the coordinates of the surrounding vertices, ensuring that the rate of change of the normal vector between the repaired crack region and the surrounding mesh is ≤5° / cm, resulting in no visually obvious repair traces.

[0095] Specifically, the data transmission method in the terminal interaction module is as follows:

[0096] Structured datasets are generated from information obtained through trace evidence analysis;

[0097] After the robot dog establishes a connection with the remote monitoring terminal, it negotiates transmission parameters through a handshake protocol. The robot dog monitors the transmission progress, signal strength, and battery level in real time. If the transmission is interrupted, it automatically saves the transmission breakpoint information and resumes transmission after the connection is restored. If the battery level is lower than the threshold, it prioritizes the transmission of key trace information, pauses the transmission and records the status, and resumes transmission after charging.

[0098] The remote monitoring terminal receives data through dedicated client software and loads a 3D model into a 3D visualization engine; the structured dataset is associated with the spatial coordinates of the 3D model, and spectral curves, size parameters, and analysis conclusions are displayed in the trace area of ​​the 3D model.

[0099] In this embodiment, the structured dataset includes:

[0100] Spectral analysis report: Spectral reflectance curves (containing 3-10 band values) of trace targets (such as bloodstains and fibers) extracted from multispectral point cloud datasets, matching degree with standard spectral libraries (e.g., bloodstain matching degree of 92%), and inference results of material composition.

[0101] Three-dimensional spatial parameters: the three-dimensional coordinates (X,Y,Z) of the target trace, size parameters (length, width, area, with accuracy ≤1cm, 0.5cm, 0.01㎡ respectively), and spatial distance from surrounding reference objects (e.g., 1.23m from the corner of the wall).

[0102] Model association information: boundary coordinates of the model clipping area, measurement annotation layers (such as distance lines and angle annotations), and optimization and repair records (such as the location and area of ​​hole repairs).

[0103] The above information is encapsulated in JSON format, with each record containing a timestamp, data confidence level, and data collection device identifier to ensure information traceability.

[0104] Meanwhile, wireless communication automatically switches to the optimal link based on the on-site network environment:

[0105] 5G links are enabled in unobstructed outdoor scenarios, supporting the Sub-6GHz band, with a transmission rate of ≥10Mbps and network latency of ≤50ms. Data transmission stability is ensured through standalone networking mode.

[0106] Enable WiFi 6 links indoors or in WiFi coverage scenarios, support the 802.11ax protocol, operate in the 5GHz band, with a single stream rate of ≥866Mbps, and achieve concurrent transmission of multiple devices through MU-MIMO technology to avoid channel congestion.

[0107] After communication is started, the signal strength is automatically detected (≥-85dBm required). If the signal is weak, the robot dog is triggered to move to an area with a good signal, or the relay mode is enabled to enhance the signal.

[0108] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. An autonomous trace inspection system for robot dogs based on panoramic image reconstruction, characterized in that, include: The data acquisition module acquires surface condition image data of the trace inspection site in the spectral band through a multispectral camera, and collects spectral reflectance information of the ground objects at the trace inspection site; The laser radar emits a laser beam and receives the echo to obtain three-dimensional spatial point cloud data of the trace inspection site; The data association module acquires the robot dog's position and attitude information based on the BeiDou positioning system and attitude sensors. Under the control of a hardware synchronization controller, the multispectral camera and the lidar simultaneously acquire the 3D spatial point cloud data and the surface condition image data. A timestamp is added to the acquired 3D spatial point cloud data and surface condition image data based on a time system. Combined with the robot dog's position and attitude information, an associated dataset is generated. By performing spectral enhancement and data correction processing on the received surface condition image data, and based on the associated dataset, the spectral information of the processed surface condition image data is associated with the corresponding 3D points in space, generating a multispectral point cloud dataset containing spatial coordinates and spectral information. The texture mapping module generates a geometric model of the surface condition of the trace inspection site based on the three-dimensional spatial point cloud data. Based on the multispectral point cloud dataset, it uses texture mapping technology to map the color and texture information in the surface condition image data onto the geometric model of the surface condition, and outputs a visual representation of the trace inspection site in three-dimensional space. The terminal interaction module optimizes the visualization by removing holes, cracks, and defects, and smoothing the surface of the visualization, outputting a 3D model, and transmitting the 3D model to a remote monitoring terminal via wireless communication.

2. The system according to claim 1, characterized in that, In the data acquisition module, the multispectral camera is fixed to the back of the robot dog via a shockproof bracket. Based on the instructions of the hardware synchronization controller, the multispectral camera and the lidar are started simultaneously. Based on the robot dog's preset path, the multispectral camera captures images of the surface condition of the trace inspection site at a set frame rate and adds timestamps to the surface condition images. Combined with the robot dog's posture information at the time of shooting, a metadata file containing image data, spectral band information, timestamps, and camera intrinsic parameters is output.

3. The system according to claim 1, characterized in that, The method for acquiring the three-dimensional spatial point cloud data in the data acquisition module is as follows: A laser emitter emits a laser of a specific wavelength and power, and a laser receiver receives the laser reflected from the surface of an object. Based on a pre-defined task plan and the distribution of evidence at the scene, the laser controller adjusts the emission angle and scanning mode of the laser emitter. The data collection circuit is connected to the lidar receiver, converts the received analog signal into a digital signal, and outputs three-dimensional spatial point cloud data of the trace inspection site by performing intensity correction, distance calculation and data filtering on the digital signal.

4. The system according to claim 1, characterized in that, The data correction processing in the data association module includes geometric correction and time synchronization correction, and the correction method is as follows: Based on the associated dataset and the intrinsic and extrinsic parameters of the multispectral camera, ground control points are deployed in the trace detection acquisition area. The surface condition image coordinates and three-dimensional spatial point cloud coordinates of the ground control points are extracted based on the surface condition image data and the three-dimensional spatial point cloud data. The surface condition image coordinates and three-dimensional spatial point cloud coordinates are associated with the world coordinates of the ground control points through a geometric transformation matrix to establish a coordinate association architecture. Based on the coordinate association architecture, the surface condition image is resampled and projected using a geometric correction model to complete geometric correction. Based on the common features in the surface condition image data and the three-dimensional spatial point cloud data, the collected surface condition image data and three-dimensional spatial point cloud data are time-synchronized and corrected by the nearest neighbor interpolation method. The time-synchronized and corrected surface condition image data and three-dimensional spatial point cloud data are then organized into a structured array containing the pixel coordinates of the surface condition image, spectral information, three-dimensional spatial point cloud coordinates, and corresponding timestamps.

5. The system according to claim 1, characterized in that, The generation of the multispectral point cloud dataset in the data association module includes: By performing spectral enhancement and data correction processing on the received surface condition image data, a surface condition image containing reflectance information and spatial coordinates is output; the received three-dimensional spatial point cloud data is processed by denoising and filtering to retain the effective point cloud, and a continuous point cloud in the global coordinate system is generated based on the effective point cloud through registration. Based on the corrected surface condition image, spectral reflectance information of the pixels in the surface condition image is extracted pixel by pixel. Based on the spectral reflectance information, the actual spatial ray corresponding to the pixel in the surface condition image is calculated using a projection model. The three-dimensional points in the three-dimensional spatial point cloud data are traversed, and the point closest to the actual spatial ray is marked as the matching point. The spectral information of the pixel in the surface condition image is assigned to the corresponding three-dimensional point in the three-dimensional spatial point cloud data. After point-by-point matching, a multispectral point cloud dataset containing spatial coordinates and spectral reflectance data of the three-dimensional points is formed.

6. The system according to claim 1, characterized in that, The surface condition geometric model in the texture mapping module is generated in the following way: The three-dimensional spatial point cloud data is optimized by removing redundant noise points and invalid data. Based on the optimized three-dimensional spatial point cloud data, global registration of the three-dimensional spatial point cloud is performed in conjunction with the associated dataset, unifying the point clouds from different perspectives into the global coordinate system. Based on the density and distribution of three-dimensional spatial point cloud data, the division rules of triangular mesh are determined by the triangulation method, the three-dimensional spatial point cloud data is transformed into a triangular mesh model, and the size and shape of the triangles are adjusted based on the division rules. The triangular mesh model is topologically optimized by removing non-manifold edges and isolated faces, and a surface condition geometric model is output.

7. The system according to claim 1, characterized in that, The texture mapping technology in the texture mapping module includes: The triangular mesh model of the surface condition geometric model is assigned horizontal and vertical coordinates, and the surface of the triangular mesh model is unfolded into two-dimensional planar coordinates; based on the premise of minimizing stretching deformation, the triangular facets and regions on the two-dimensional horizontal and vertical planes are mapped and paired through a mapping unfolding algorithm; Based on the associated dataset, a spatial coordinate transformation relationship between the multispectral camera and the lidar is established; based on the projection matrix of the multispectral camera, the three-dimensional spatial projection coordinates of the multispectral image pixels are calculated, and the corresponding geometric model mesh patches are determined based on the three-dimensional spatial projection coordinates; based on the horizontal-vertical coordinates of the geometric model mesh patches, the positions of pixel color and texture information on the two-dimensional horizontal-vertical plane are determined. Based on the matching results of the geometric model mesh patches, pixel information of the corresponding region of the surface condition image is collected and assigned to the corresponding positions in the horizontal-vertical plane.

8. The system according to claim 7, characterized in that, The result of the texture mapping technique is optimized as follows: Feathering technology is used to similarize the texture edges of adjacent horizontal and vertical blocks. By adjusting the color values ​​of the pixels at the edges of the horizontal and vertical blocks, the color difference at the seams is reduced. A texture stretching compensation algorithm is used to correct the texture deformation in areas of abrupt curvature changes. Combined with the reflectivity information in the multispectral point cloud dataset, texture enhancement is performed on specific trace detection target areas. Based on the accuracy verification procedure, a geometric model mesh patch is randomly selected, and the color deviation between the texture of the geometric model mesh patch and the texture of the actual surface condition image is compared. If the deviation exceeds the threshold, a correction procedure is triggered to adjust the horizontal-vertical coordinates and perform resampling and mapping.

9. The system according to claim 1, characterized in that, The method for removing holes and cracks from the visual representation in the terminal interaction module is as follows: By traversing the triangular facets of the surface condition geometric model, the boundary edges of the triangular facets that are not shared by adjacent facets are detected. When the boundary edges form a closed loop, it is determined to be a hole region, and the coordinates of the boundary vertex, the outline shape, and the area of ​​the hole are recorded. The angle between the normal vectors of adjacent triangular facets is calculated. When the angle between the normal vectors exceeds a preset threshold and the edge distance is less than a preset distance, it is determined to be a crack region, and the start and end points and extension path of the crack are located. Select the effective vertex closest to the boundary inside the hole boundary as the seed point, and generate new triangular patches gradually towards the hole region with the seed point as the center, based on the principle of triangulation. Based on the detected crack region, the crack width is reduced to within a preset width by adjusting the position coordinates of the vertices on both sides of the crack; then, the boundary edges on both sides of the crack are merged into a shared edge by using an edge fusion algorithm to generate a new triangular patch to connect the crack region.

10. The system according to claim 1, characterized in that, The data transmission method in the terminal interaction module is as follows: Structured datasets are generated from information obtained through trace evidence analysis; After the robot dog establishes a connection with the remote monitoring terminal, it negotiates transmission parameters through a handshake protocol. The robot dog monitors the transmission progress, signal strength, and battery level in real time. If the transmission is interrupted, it automatically saves the transmission breakpoint information and resumes transmission after the connection is restored. If the battery level is lower than the threshold, it prioritizes the transmission of key trace information, pauses the transmission and records the status, and resumes transmission after charging. The remote monitoring terminal receives data through dedicated client software and loads a 3D model into a 3D visualization engine; the structured dataset is associated with the spatial coordinates of the 3D model, and spectral curves, size parameters, and analysis conclusions are displayed in the trace area of ​​the 3D model.

Images