Robotic dog autonomous trace detection system based on panoramic image reproduction

Abstract

The invention relates to a robot dog autonomous trace detection system based on panoramic image reproduction, and belongs to the technical field of criminal investigation trace detection. The method comprises the following steps: acquiring surface layer condition image data through a multispectral camera, and acquiring three-dimensional space point cloud data through a laser radar; combining the collected image data and point cloud data with the position and attitude information of the robot dog to generate an associated data set; carrying out optimization processing on the surface layer condition image data, and associating the processed data to three-dimensional points corresponding to spatial positions to generate a multispectral point cloud data set; generating a surface layer condition geometric model based on the three-dimensional space point cloud data, and mapping color and texture information to the model through a texture mapping technology; and optimizing the visual representation to output a 3D model, and transmitting the 3D model to a remote monitoring terminal through wireless communication. According to the invention, scene reproduction of a trace detection site is realized, and the efficiency, accuracy and scene adaptability of trace detection work are improved.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of criminal investigation and trace detection, and particularly relates to a machine dog autonomous trace detection system based on panoramic image reproduction. BACKGROUND

[0002] The trace detection scene is complex and diverse, and environmental factors greatly hinder the development of trace detection work. With the continuous improvement of the evidence accuracy requirement of the judicial system, the detection precision standard of micro traces for trace detection work is also increasingly strict. When facing a complex scene, the traditional static microscope device is difficult to flexibly adjust the position and angle, and cannot continuously and accurately microphotograph and analyze the traces at different positions and angles. Moreover, due to the uncertainty of the position of the scene traces, the traditional device is difficult to realize the full-range coverage investigation of the traces while maintaining high-precision imaging.

[0003] In recent years, the machine dog technology has developed rapidly, and its high mobility, flexibility and environmental adaptability have brought new solutions to trace detection work. The trace detection microscope gimbal is mounted on the machine dog, which can make the microscope device free from the restriction of the traditional fixed position, and realize the dynamic tracking and investigation of the scene traces. During the trace detection process, the surface of the trace evidence is often uneven, and the machine dog will vibrate and change posture during movement, which requires the microscope device to have real-time dynamic focusing capability.

[0004] The traditional fixed focusing mode cannot adapt to the changing investigation environment, which is easy to cause image blur and loss of key details, and is difficult to realize high-precision posture adjustment and focusing control, and cannot meet the requirement of microscopic photography for fine observation of small objects. The existing focusing system has slow focusing speed and low precision when facing dynamic changes of the scene, and cannot timely and accurately track the depth changes of the target object. These defects result in poor quality of the collected trace detection images, and affect the accuracy and reliability of the trace detection results. SUMMARY

[0005] To solve the above problems in the prior art, the application provides a machine dog autonomous trace detection system based on panoramic image reproduction, and the purpose of the application can be achieved through the following technical scheme. The machine dog autonomous trace detection system based on panoramic image reproduction comprises: A data acquisition module acquires surface condition image data of the trace detection scene in the spectral band through a multispectral camera, collects spectral reflectance information of the trace detection scene ground object, and emits a laser beam and receives a return wave through a laser radar to acquire three-dimensional space point cloud data of the trace detection scene. The data association module obtains position and attitude information of the robot dog based on a Beidou positioning system and an attitude sensor; the multispectral camera and the laser radar simultaneously collect the three-dimensional space point cloud data and the surface condition image data under the control of a hardware synchronization controller, add time stamps to the collected three-dimensional space point cloud data and the surface condition image data based on a time system, and generate an associated data set in combination with the position and attitude information of the robot dog; the surface condition image data received is subjected to spectral enhancement and data correction processing, the spectral information of the processed surface condition image data is associated to the three-dimensional point corresponding to the spatial position based on the associated data set, and a multispectral point cloud data set containing spatial coordinates and spectral information is generated; The texture mapping module generates a surface condition geometric model of the trace detection site according to the three-dimensional space point cloud data, maps the color and texture information in the surface condition image data to the surface condition geometric model based on the multispectral point cloud data set through a texture mapping technology, and outputs a visual representation of the trace detection site in a three-dimensional space. The terminal interaction module optimizes the visual representation by removing holes and crack defects in the visual representation and smoothing the surface of the visual representation, outputs a 3D model, and transmits the 3D model to a remote monitoring terminal through wireless communication.

[0006] As a preferred technical solution of the present application, the multispectral camera in the data acquisition module is fixed on the back of the robot dog through a shockproof support; based on the instructions of the hardware synchronization controller, the multispectral camera and the laser radar are started at the same time, based on the preset path of the robot dog, the multispectral camera photographs surface condition images of the trace detection site at a set frame rate and adds time stamps to the surface condition images, and outputs a metadata file containing image data, spectral band information, time stamps, and camera intrinsic parameters in combination with the multispectral camera attitude information at the photographing moment.

[0007] Specifically, the three-dimensional space point cloud data in the data acquisition module is obtained in the following manner: The laser radar transmitter emits laser of a specific wavelength and power, and the laser radar receiver receives the laser reflected by the surface of an object; based on a preset task plan and the distribution of trace detection site physical evidence, the laser controller adjusts the emission angle and scanning mode of the laser emitter; The data collection circuit is connected with the laser radar receiver, converts the received analog signal into a digital signal, and outputs three-dimensional space point cloud data of the trace detection site through intensity correction, distance calculation, and data filtering operations on the digital signal.

[0008] Specifically, the data correction processing in the data association module includes geometric correction and time synchronization correction, and the correction manner is as follows: Based on the associated data set and the internal and external parameters of the multi-spectral camera, ground control points are laid out in the trace detection collection area, the surface condition image coordinates and three-dimensional space point cloud coordinates of the ground control points are extracted based on the surface condition image data and the three-dimensional space point cloud data, the surface condition image coordinates and the three-dimensional space point cloud coordinates are associated to the world coordinates of the ground control points through a geometric transformation matrix, and a coordinate association architecture is established; based on the coordinate association architecture, the surface condition image is resampled and projected through a geometric correction model to complete geometric correction; Based on the common features in the surface condition image data and the three-dimensional space point cloud data, the collected surface condition image data and three-dimensional space point cloud data are time-synchronized and corrected through a nearest neighbor interpolation method, and the surface condition image data and three-dimensional space point cloud data after time-synchronized and corrected are organized and constructed into a structured array containing surface condition image pixel coordinates, spectral information, three-dimensional space point cloud coordinates and corresponding time stamps.

[0009] Specifically, the generation of the multi-spectral point cloud data set in the data association module includes: The received surface condition image data is processed through spectral enhancement and data correction to output a surface condition image containing reflectivity information and spatial coordinates; the received three-dimensional space point cloud data is processed through denoising and filtering to retain effective point clouds, and based on the effective point clouds, continuous point clouds in a global coordinate system are generated through registration; Based on the corrected surface condition image, the spectral reflectivity information of the surface condition image pixels is extracted pixel by pixel, and based on the spectral reflectivity information, the actual space rays corresponding to the surface condition image pixels are calculated through a projection model; the three-dimensional points in the three-dimensional space point cloud data are traversed, the points closest to the actual space rays are labeled as matching points, the spectral information of the surface condition image pixels is assigned to the three-dimensional points in the three-dimensional space point cloud data, and after point-by-point matching, a multi-spectral point cloud data set containing spatial coordinates and spectral reflectivity data is formed.

[0010] Specifically, the generation of the surface condition geometric model in the texture mapping module is: The three-dimensional space point cloud data is optimized by removing redundant noise points and invalid data, and based on the optimized three-dimensional space point cloud data, global registration of three-dimensional space point clouds is performed in combination with the associated data set, and point clouds of different perspectives are unified to a global coordinate system. Based on the density and distribution of the three-dimensional space point cloud data, the division rule of the triangular mesh is determined by the triangulation method, the three-dimensional space point cloud data is converted into a triangular mesh model, and the size and shape of the triangle are adjusted based on the division rule; the topological optimization of the triangular mesh model is performed by removing the non-manifold edges and isolated face pieces in the triangular mesh model, and the surface condition geometric model is output.

[0011] Specifically, the texture mapping technology in the texture mapping module includes: The triangular mesh model of the surface condition geometric model is assigned with horizontal-vertical coordinates, and the surface of the three-dimensional mesh model is unfolded into a two-dimensional plane coordinate; based on the premise of minimizing the tensile deformation, the triangular face pieces and the regions on the two-dimensional horizontal-vertical plane are mapped and paired through the mapping unfolding algorithm; Based on the correlation data set, the spatial coordinate conversion relationship between the multi-spectral camera and the laser radar is established; based on the projection matrix of the multi-spectral camera, the three-dimensional space projection coordinates of the multi-spectral image pixels are calculated, and the corresponding geometric model mesh face pieces are determined based on the three-dimensional space projection coordinates; based on the horizontal-vertical coordinates of the geometric model mesh face pieces, the positions of the pixel color and texture information on the two-dimensional horizontal-vertical plane are determined; Based on the matching results of the geometric model mesh face pieces, the pixel information of the corresponding region of the surface condition image is collected and assigned to the corresponding position of the horizontal-vertical plane.

[0012] Specifically, the result of mapping by the texture mapping technology is optimized by the method: The texture edges of adjacent horizontal-vertical blocks are processed similarly through the feathering technology, the color values of the edge pixels of the horizontal-vertical blocks are adjusted to reduce the color difference at the joint; the texture deformation of the curvature mutation area is corrected through the texture stretching compensation algorithm; the texture of the specific trace detection target area is enhanced in combination with the reflectivity information in the multi-spectral point cloud data set; Based on the accuracy verification program, the geometric model mesh face pieces are randomly selected, the color deviation of the texture of the geometric model mesh face pieces and the actual surface condition image texture is compared, and if the deviation exceeds the threshold, the correction program is triggered to adjust the horizontal-vertical coordinates and perform resampling and mapping.

[0013] Specifically, the method for removing holes and crack defects in the visualization representation in the terminal interaction module is: By traversing the triangular patches of the surface condition geometric model, the boundary edges of the triangular patches which are not shared by adjacent patches are detected, when the boundary edges form a closed loop, it is determined as a hole region, and the boundary vertex coordinates, the outline shape and the area size of the hole are recorded; the normal vector angles of adjacent triangular patches are calculated, when the normal vector angle exceeds a preset threshold and the edge distance is less than a preset distance, it is determined as a crack region, and the start and end points and the extension path of the crack are located; An effective vertex closest to the boundary inside the hole boundary is selected as a seed point, and based on the principle of triangular partitioning, new triangular patches are gradually generated towards the hole region with the seed point as the center; Based on the detected crack region, the position coordinates of the vertices on both sides of the crack are adjusted to reduce the crack width to within a preset width; and then the edge fusion algorithm is used to combine the boundary edges on both sides of the crack into shared edges to generate new triangular patches to connect the crack region.

[0014] Specifically, the transmission mode of data in the terminal interaction module is: The information generated by trace analysis is structured into a data set; After the robot dog establishes a connection with the remote monitoring terminal, transmission parameters are negotiated through a handshake protocol; the robot dog monitors the transmission progress, signal strength and battery power in real time, and if the transmission is interrupted, the transmission breakpoint information is automatically saved, and the transmission is continued after the connection is restored; if the battery power is lower than a threshold, the key trace information is preferentially transmitted, the transmission is paused and the state is recorded, and the transmission is supplemented after charging; The remote monitoring terminal receives data through a special client software and loads a 3D model in a three-dimensional visualization engine; the structured data set is associated with the spatial coordinates of the 3D model, and the spectral curve, size parameter and analysis conclusion are displayed in the trace area of the 3D model.

[0015] The beneficial effects of the present application are: The spectral reflectance of the multispectral image and the three-dimensional spatial information of the point cloud data are deeply fused to generate a multispectral point cloud data set which contains not only the three-dimensional coordinates of the ground features but also the reflectance characteristics of 3-10 wave bands. This dual data support of spatial position and material composition enables trace analysts to not only locate trace positions but also accurately identify targets such as bloodstains, fibers and tool marks through spectral comparison, solving the problem of similar color materials that cannot be identified by traditional visual imaging. Through a hardware synchronization controller and timestamp technology, strict space-time binding of multispectral data, point cloud data and robot dog posture and position information is achieved. Combined with geometric correction and time synchronization correction of ground control points, the matching error of image pixels and three-dimensional point clouds is ensured, and the analysis deviation caused by space-time misplacement of multiple data sources is avoided, providing a high-precision data basis for subsequent modeling and trace analysis. The machine dog has the ability to cross obstacles, adapt to narrow spaces and complex terrain. The multi-spectral and point cloud collection devices carried by the machine dog can reach areas that traditional manual or fixed equipment cannot reach. Combined with the preset path planning and laser radar scanning mode adjustment, it can realize all-round data collection of complex indoor and outdoor scenes, and reduce the trace omission problem caused by poor accessibility. The surface condition geometric model generated based on the point cloud data can completely restore the geometric shape and surface texture details of the scene by combining the texture mapping technology. Through model optimization and cutting measurement function, not only the micro features of key traces can be preserved, but also the accurate three-dimensional visual scene meeting the trace detection requirements can be generated, realizing the long-term retention and backtracking analysis of the scene digital twin. Using wireless communication technology, the optimized 3D model and structured trace detection information can be transmitted to the remote terminal in real time. Remote experts can realize model rotation, sectioning and trace interaction analysis through three-dimensional visualization engine, breaking through the time and space limitations of on-site investigation relying on local personnel, especially suitable for cross-regional collaborative research and judgment of major cases, improving the trace detection response speed and decision-making efficiency. Through the hole repair algorithm of region growing filling, the surface noise is smoothed by Gaussian filtering, and after topological inspection and precision verification, it is ensured that the output model has no geometric defects and retains the key trace features. The measurement function can directly obtain the trace size parameters, avoiding the subjective error of traditional manual measurement. The multi-spectral camera supports visible light and near-infrared band extension, the laser radar can adjust the scanning mode and wavelength, and the dynamic focusing control technology can adapt to different magnification microscopic needs. This modular architecture enables the system to flexibly upgrade hardware or algorithms according to the trace detection scene, with scalability to continuously adapt to the development of trace detection technology. In summary, through the deep innovation of multi-source data fusion, intelligent collection and accurate modeling, the invention significantly improves the accuracy, efficiency and scene adaptability of trace detection, providing a systematic solution for trace identification, digital preservation and remote collaborative research and judgment in complex scenes, which has important practical value and promotional significance. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to facilitate the understanding of those skilled in the art, the invention will be further described below with reference to the drawings.

[0017] Figure 1 A flowchart of a machine dog autonomous trace detection system based on panoramic image reconstruction according to the invention; Figure 2 A multi-spectral point cloud data set structure block diagram according to the invention; Figure 3 A texture mapping technology application flowchart according to the invention. DETAILED DESCRIPTION

[0018] In order to further illustrate the technical means and effects adopted by the present application to achieve the predetermined inventive purpose, the specific embodiments, structures, features and effects according to the present application are described in detail below in combination with the drawings and preferred embodiments.

[0019] Please refer to Figure 1 , the autonomous trace detection system of the robot dog based on panoramic image reconstruction, comprising: The data acquisition module acquires surface condition image data of the trace detection site in the spectral band through a multispectral camera, collects spectral reflectance information of trace detection site features; the laser radar emits a laser beam and receives the echo, and acquires three-dimensional spatial point cloud data of the trace detection site; The data association module obtains the position and attitude information of the robot dog based on the Beidou positioning system and the attitude sensor; the multispectral camera and the laser radar simultaneously collect the three-dimensional spatial point cloud data and the surface condition image data under the control of the hardware synchronization controller, add time stamps to the collected three-dimensional spatial point cloud data and the surface condition image data based on the time system, and generate an associated data set in combination with the position and attitude information of the robot dog; through spectral enhancement and data correction processing of the received surface condition image data, based on the associated data set, the spectral information of the processed surface condition image data is associated with the three-dimensional point corresponding to the spatial position, and a multispectral point cloud data set containing spatial coordinates and spectral information is generated; The texture mapping module generates a surface condition geometric model of the trace detection site according to the three-dimensional spatial point cloud data, and maps the color and texture information in the surface condition image data onto the surface condition geometric model based on the multispectral point cloud data set through texture mapping technology, and outputs a visual representation of the trace detection site in three-dimensional space; The terminal interaction module optimizes the visual representation by removing holes, cracks and defects in the visual representation and smoothing the surface of the visual representation, and outputs a 3D model, which is transmitted to a remote monitoring terminal through wireless communication.

[0020] Specifically, the multispectral camera in the data acquisition module is fixed on the back of the robot dog through a shockproof support; based on the instructions of the hardware synchronization controller, the multispectral camera and the laser radar are started at the same time, based on the preset path of the robot dog, the multispectral camera shoots the surface condition image of the trace detection site at a set frame rate and adds a time stamp to the surface condition image, in combination with the multispectral dog attitude information at the shooting time, outputs a metadata file containing image data, spectral band information, time stamp and camera internal parameter.

[0021] In this embodiment, the multispectral camera is fixed on the back of the robot dog through the shockproof bracket. Compared with the traditional RGB camera, the multispectral camera can capture the spectral reflectance information of the object surface in multiple discrete narrow wavebands (such as from visible light to near infrared). These information can reveal potential traces that the human eye cannot distinguish, such as trace amounts of blood, body fluid residues, differences in different inks, or original information under camouflage paint, greatly enhancing the ability to discover and identify trace evidence, and making up for the limitations of visible light imaging.

[0022] The output timestamped image data is associated with the point cloud data and the robot dog positioning data, providing a spatiotemporal reference for the fusion of spectral information and three-dimensional point cloud data in the data association module, and is the core input for generating multispectral point cloud data sets. The preprocessed image data contains rich color and texture information, providing materials for texture mapping in the texture mapping module, so that the final 3D model can intuitively present the material distribution characteristics of the surface layer on the scene, assisting trace detection personnel in remote research and judgment. Through precise spectral imaging and synchronous control, the multispectral camera constructs the basis for the association of "spectral properties-space location" in the trace detection scene, which is the key hardware support for realizing the intelligent and precise robot dog trace detection system.

[0023] Specifically, the acquisition method of the three-dimensional space point cloud data in the data acquisition module is: The laser radar transmitter emits laser of specific wavelength and power, and the laser radar receiver receives the laser reflected by the object surface; based on the preset task planning and the distribution of trace evidence in the scene, the laser controller adjusts the emission angle and scanning mode of the laser emitter; The data collection circuit is connected with the laser radar receiver, converts the received analog signal into digital signal, and outputs the three-dimensional space point cloud data of the trace detection scene by performing intensity correction, distance calculation, and data filtering operations on the digital signal.

[0024] In this embodiment, the laser controller automatically adjusts the scanning mode according to the distribution of trace evidence (such as suspected bloodstain area): 10Hz low frequency scanning is used in normal areas, and 20Hz high frequency scanning is switched to in key areas, to ensure that the point cloud density of trace area is ≥200 points / ㎡.

[0025] As the core of three-dimensional data acquisition, the laser radar can quickly acquire large-scale and high-density three-dimensional space point cloud data of the scene by emitting laser beams and accurately measuring their return time, providing a basis for constructing the macro geometric framework of the scene. According to the description in the data acquisition module, the laser controller can flexibly adjust the scanning mode and angle of the laser emitter according to the distribution of trace evidence, realizing key scanning of key areas.

[0026] Specifically, the data correction processing in the data association module includes geometric correction and time synchronization correction, and the correction mode is: Based on the internal and external parameters of the multi-spectral camera and the associated data set, ground control points are arranged in the trace detection collection area, the surface condition image coordinates and three-dimensional space point cloud coordinates of the ground control points are extracted based on the surface condition image data and the three-dimensional space point cloud data, the surface condition image coordinates and the three-dimensional space point cloud coordinates are associated to the world coordinates of the ground control points through a geometric transformation matrix, and a coordinate association architecture is established; based on the coordinate association architecture, the surface condition image is resampled and projected through a geometric correction model to complete geometric correction. Based on the common features in the surface condition image data and the three-dimensional space point cloud data, the collected surface condition image data and three-dimensional space point cloud data are time-synchronized through a nearest neighbor interpolation method, and the time-synchronized surface condition image data and three-dimensional space point cloud data are organized to construct a structured array containing surface condition image pixel coordinates, spectral information, three-dimensional space point cloud coordinates, and corresponding time stamps.

[0027] Specifically, the generation of the multi-spectral point cloud data set in the data association module includes: The received surface condition image data is processed through spectral enhancement and data correction to output a surface condition image containing reflectivity information and spatial coordinates; the received three-dimensional space point cloud data is processed through denoising and filtering to retain effective point clouds, and based on the effective point clouds, continuous point clouds in a global coordinate system are generated through registration; Based on the corrected surface condition image, the spectral reflectivity information of the surface condition image pixels is extracted pixel by pixel, and based on the spectral reflectivity information, the actual spatial rays corresponding to the surface condition image pixels are calculated through a projection model; the three-dimensional points in the three-dimensional space point cloud data are traversed, and the point closest to the actual spatial ray is labeled as a matching point, the spectral information of the surface condition image pixels is assigned to the three-dimensional points in the three-dimensional space point cloud data, and after point-by-point matching, a multi-spectral point cloud data set containing spatial coordinates and spectral reflectivity data is formed.

[0028] Please refer to Figure 2 In this embodiment, the generated initial data set is subjected to topological inspection to ensure that the spectral reflectivity change of adjacent point cloud data points conforms to the actual ground object distribution rule (such as the reflectivity standard deviation of the surface of the same object being ≤5%), and the abnormal points are subjected to secondary matching or marked as low confidence. Combined with the measured data of a standard reflectivity reference plate (such as a 50% gray plate), the reflectivity values of the data set are globally calibrated to ensure that the reflectivity deviation of the key waveband is ≤1%, and the comparability of the data in different regions is improved. Through hardware synchronization control and attitude parameter correction, the time difference between spectral information and spatial coordinates is ensured to be ≤5 ms, the spatial offset is ≤3 pixels, and the correlation error of "misattribution" is avoided. It supports full coverage from centimeter-level subtle traces (such as fibers and scratches) to meter-level scene range, and the point cloud density can be adjusted according to requirements (such as single-point spacing ≤1 cm in key areas and ≤5 cm in background areas). The waveband expansion interface is reserved, and the ultraviolet or short-wave infrared band can be added according to the trace detection requirements to improve the identification ability of special substances (such as fluorescent reagent marked traces). Through the "position + spectrum" dual attributes, accurate positioning and component analysis of trace detection targets are realized. For example, in the grass, the point cloud area with reflectance characteristics consistent with bloodstains can be directly obtained its three-dimensional coordinates (X=123.45m, Y=67.89m, Z=0.05m), providing accurate guidance for on-site investigation. Spectral texture data source is provided for texture mapping, so that the three-dimensional model not only presents geometric shape, but also visualizes material distribution through reflectance difference (such as using pseudo-color coding different reflectance areas to highlight suspicious traces). The dataset can be transmitted to a remote terminal through wireless communication (5G / WiFi6), supporting offline spectral comparison (such as matching with known spectral library) and spatial measurement, realizing "non-contact trace detection research and judgment".

[0029] Specifically, the generation manner of the surface condition geometric model in the texture mapping module is: The three-dimensional spatial point cloud data is optimized by removing redundant noise points and invalid data, and the global registration of three-dimensional spatial point cloud is performed based on the optimized three-dimensional spatial point cloud data and the associated data set, and the point clouds of different perspectives are unified to a global coordinate system; Based on the density and distribution of three-dimensional spatial point cloud data, the division rule of triangular mesh is determined by triangular subdivision method, the three-dimensional spatial point cloud data is converted into triangular mesh model, and the size and shape of the triangle are adjusted based on the division rule; the topological optimization of the triangular mesh model is performed by removing the non-manifold edges and isolated face pieces in the triangular mesh model, and the surface condition geometric model is output.

[0030] In this embodiment, the spatial scale of the model is reduced 1:1 on site, the three-dimensional distance, angle, and area measurement accuracy of any two points are consistent with the original point cloud (distance accuracy ≤1 cm, angle ≤0.5°), and the model can be directly used for size quantification analysis of trace detection targets (such as measuring scratch length and recess depth). The topological structure of the grid surface supports interactive operations such as rectangular and polygonal area cutting (focusing on suspected areas), profile sectioning (observing internal structures of the surface layer), and three-dimensional roaming, meeting the observation needs of trace detection personnel at different angles. The grid patches of the model have a coordinate mapping relationship with the spectral information in the multispectral point cloud dataset. In the texture mapping stage of the texture mapping module, the color and texture of the multispectral image can be accurately attached to the corresponding grid area through UV coordinates, so that the model is upgraded from a “pure geometric skeleton” to a “digital twin with material properties”. The surface condition geometric model and the multispectral point cloud dataset are deeply related through “coordinate mapping”, and each vertex coordinate of the model corresponds to a three-dimensional point coordinate in the multispectral point cloud dataset, ensuring that the spatial references of the two are consistent; the coverage range of the grid patch completely matches the distribution range of the point cloud data, providing a basis for regional division for the mapping of spectral information from “point” to “surface”; the topological structure of the model provides a boundary reference for the splicing of multispectral textures, avoiding image distortion caused by geometric discontinuity during texture mapping. This correlation mechanism not only preserves the spatial accuracy of the point cloud data, but also provides a structured framework for the attachment of spectral properties, serving as a key link between “spatial geometry” and “material properties”. See Figure 3 , specifically, the texture mapping technology in the texture mapping module includes: The triangular mesh model of the surface condition geometric model is assigned horizontal-vertical coordinates, and the surface of the three-dimensional mesh model is unfolded into a two-dimensional plane coordinate; based on the premise of minimizing tensile deformation, the triangular patches and the regions on the two-dimensional horizontal-vertical plane are mapped and paired through a mapping unfolding algorithm; Based on the associated dataset, the spatial coordinate conversion relationship between the multispectral camera and the laser radar is established; based on the projection matrix of the multispectral camera, the three-dimensional spatial projection coordinates of the multispectral image pixels are calculated, and based on the three-dimensional spatial projection coordinates, the corresponding geometric model grid patches are determined; based on the horizontal-vertical coordinates of the geometric model grid patches, the positions of the pixel color and texture information on the two-dimensional horizontal-vertical plane are determined; Based on the matching results of the geometric model grid patches, the pixel information of the corresponding region of the surface condition image is collected and assigned to the corresponding positions on the horizontal-vertical plane.

[0031] In this embodiment, slight posture jitter may occur during the movement of the robot dog, causing spatial and temporal deviation of the image and the point cloud. Through time stamp accurate matching (synchronization error ≤1 ms) and posture interpolation algorithm, the camera external parameter is corrected in real time to ensure the spatial and temporal consistency of the "image-point cloud". In the face of the texture stretching deformation caused by the uneven terrain outdoors, a texture sampling strategy based on geometric self-adaptation is adopted to increase the sampling points in the high-curvature area (the sampling density is increased by 2-3 times), thereby reducing the texture distortion. As for the brightness difference between the direct sunlight area and the shadow area, which may cause uneven brightness of the texture, a light model in the radiation correction is used to perform brightness equalization processing on the image, so that the texture brightness deviation of the same substance in different light areas is ≤5%. Specifically, the result mapped by the texture mapping technology is optimized, and the method is: The texture edges of adjacent horizontal-vertical blocks are processed similarly through the feathering technology, the color values of the edge pixels of the horizontal-vertical blocks are adjusted, and the color difference at the joint is reduced; the texture deformation of the curvature mutation area is corrected through the texture stretching compensation algorithm; and the reflectivity information in the multispectral point cloud dataset is combined to perform texture enhancement on the specific trace detection target area. Based on the accuracy verification program, the geometric model grid patches are randomly selected, the color deviation of the geometric model grid patch texture and the actual surface condition image texture is compared, if the deviation exceeds the threshold, the correction program is triggered, the horizontal-vertical coordinates are adjusted, and resampling and mapping are performed.

[0032] Specifically, the method for removing holes and crack defects in the visualization representation in the terminal interaction module is: By traversing the triangular patches of the surface condition geometric model, the boundary edges of the triangular patches that are not shared by adjacent patches are detected, when the boundary edges form a closed loop, it is determined as a hole region, and the boundary vertex coordinates, outline shape and area size of the hole are recorded; the included angle of the normal vectors of adjacent triangular patches is calculated, when the included angle of the normal vectors exceeds the preset threshold and the edge distance is less than the preset distance, it is determined as a crack region, and the start and end points and the extension path of the crack are located; An effective vertex closest to the boundary inside the hole boundary is selected as a seed point, based on the principle of triangular partitioning, new triangular patches are gradually generated towards the hole region with the seed point as the center; Based on the detected crack region, the position coordinates of the vertices on both sides of the crack are adjusted to reduce the crack width to within the preset width; and then the edge fusion algorithm is used to combine the boundary edges on both sides of the crack into shared edges to generate new triangular patches to connect the crack region.

[0033] In this embodiment, the cracks are eliminated by vertex adjustment and edge fusion to restore the continuity of the mesh. The vertex coordinates on both sides of the crack are corrected by elastic deformation. The optimal movement of the vertex is calculated based on the least squares method to reduce the crack width from the initial 0.3-1 cm to ≤0.1 cm. During the adjustment process, the movement direction of the vertex is constrained along the normal direction of the crack to avoid affecting the geometric shape of the surrounding mesh (such as the continuity of the grass texture). The edge fusion algorithm is used to merge the boundary edges on both sides of the crack into a shared edge, delete the duplicate vertex, and generate a new triangular patch to connect the crack area. For example, the edge on the left side of the crack (V1, V2) and the edge on the right side of the crack (V3, V4) are merged into a shared edge (V1, V3) by vertex welding technology, and a new patch (V1, V3, V5) is generated to connect the two sides of the mesh, ensuring that there is no topological fault in the fused crack area. The repaired crack area is subjected to local smoothing processing, and the Laplace smoothing algorithm is used to adjust the coordinates of the surrounding vertices to make the normal vector change rate of the crack repair area and the surrounding mesh ≤5° / cm, and there is no obvious repair mark in the visual.

[0034] Specifically, the transmission mode of data in the terminal interaction module is: Generate a structured data set from the information generated by trace analysis; After the robot dog establishes a connection with the remote monitoring terminal, the transmission parameters are negotiated through a handshake protocol. The robot dog monitors the transmission progress, signal strength, and battery level in real time. If the transmission is interrupted, it automatically saves the transmission breakpoint information and continues the transmission after the connection is restored. If the battery level is below the threshold, the key trace information is transmitted first, the transmission is paused, and the status is recorded. After charging, the transmission is resumed; The remote monitoring terminal receives data through a dedicated client software and loads the 3D model in a three-dimensional visualization engine. The structured data set is associated with the spatial coordinates of the 3D model, and the spectral curve, size parameter, and analysis conclusion are displayed in the trace area of the 3D model.

[0035] In this embodiment, the structured data set formed includes: Spectral analysis report: Extract the spectral reflectance curve (including 3-10 waveband values) of the trace target (such as bloodstains, fibers) in the multi-spectral point cloud data set, the matching degree with the standard spectral library (such as bloodstain matching degree 92%), and the material composition inference result. Three-dimensional space parameters: Three-dimensional coordinates (X, Y, Z) of the target trace, size parameters (length, width, area, accuracy ≤1 cm, 0.5 cm, 0.01 m2), and spatial distance from the surrounding reference (such as 1.23 m from the corner of the wall). Model association information: Boundary coordinates of the model cutting area, measurement annotation layers (such as distance lines, angle annotations), and optimization repair records (such as hole repair position and area). The above information is encapsulated in JSON format, and each record contains a timestamp, data confidence, and device identification, ensuring traceability of the information.

[0036] At the same time, the wireless communication automatically switches the optimal link according to the field network environment: In the outdoor unshielded scene, the 5G link is enabled, supporting the Sub-6GHz frequency band, and the transmission rate is configured to be greater than or equal to 10 Mbps, and the network delay is less than or equal to 50 ms, and the independent networking mode is used to ensure the stability of data transmission.

[0037] In the indoor or WiFi coverage scene, the WiFi6 link is enabled, supporting the 802.11ax protocol, and working in the 5GHz frequency band, and the single flow rate is greater than or equal to 866Mbps, and the MU-MIMO technology is used to realize multi-device concurrent transmission, avoiding channel congestion. After the communication is started, the signal strength is automatically detected (the requirement is greater than or equal to-85dBm), if the signal is weak, the robot dog is triggered to move to an area with good signal, or a relay mode is enabled to enhance the signal.

[0038] The above is only the preferred embodiment of the present application, and does not limit the present application in any form. Although the preferred embodiment of the present application has been disclosed as above, it is not intended to limit the present application. Any person skilled in the art can make some changes or modifications to the above disclosed technical content without departing from the scope of the present application, and any equivalent embodiments with equivalent changes and modifications are still within the scope of the present application.

Claims

1. An autonomous trace inspection system for robot dogs based on panoramic image reconstruction, characterized in that, include: The data acquisition module acquires surface condition image data of the trace inspection site in the spectral band through a multispectral camera, and collects spectral reflectance information of the ground objects at the trace inspection site; The laser radar emits a laser beam and receives the echo to obtain three-dimensional spatial point cloud data of the trace inspection site; The data association module acquires the robot dog's position and attitude information based on the BeiDou positioning system and attitude sensors. Under the control of a hardware synchronization controller, the multispectral camera and the lidar simultaneously acquire the 3D spatial point cloud data and the surface condition image data. A timestamp is added to the acquired 3D spatial point cloud data and surface condition image data based on a time system. Combined with the robot dog's position and attitude information, an associated dataset is generated. By performing spectral enhancement and data correction processing on the received surface condition image data, and based on the associated dataset, the spectral information of the processed surface condition image data is associated with the corresponding 3D points in space, generating a multispectral point cloud dataset containing spatial coordinates and spectral information. The texture mapping module generates a geometric model of the surface condition of the trace inspection site based on the three-dimensional spatial point cloud data. Based on the multispectral point cloud dataset, it uses texture mapping technology to map the color and texture information in the surface condition image data onto the geometric model of the surface condition, and outputs a visual representation of the trace inspection site in three-dimensional space. The terminal interaction module optimizes the visualization by removing holes, cracks, and defects, and smoothing the surface of the visualization, outputting a 3D model, and transmitting the 3D model to a remote monitoring terminal via wireless communication.

2. The system according to claim 1, characterized in that, In the data acquisition module, the multispectral camera is fixed to the back of the robot dog via a shockproof bracket. Based on the instructions of the hardware synchronization controller, the multispectral camera and the lidar are started simultaneously. Based on the robot dog's preset path, the multispectral camera captures images of the surface condition of the trace inspection site at a set frame rate and adds timestamps to the surface condition images. Combined with the robot dog's posture information at the time of shooting, a metadata file containing image data, spectral band information, timestamps, and camera intrinsic parameters is output.

3. The system according to claim 1, characterized in that, The method for acquiring the three-dimensional spatial point cloud data in the data acquisition module is as follows: A laser emitter emits a laser of a specific wavelength and power, and a laser receiver receives the laser reflected from the surface of an object. Based on a pre-defined task plan and the distribution of evidence at the scene, the laser controller adjusts the emission angle and scanning mode of the laser emitter. The data collection circuit is connected to the lidar receiver, converts the received analog signal into a digital signal, and outputs three-dimensional spatial point cloud data of the trace inspection site by performing intensity correction, distance calculation and data filtering on the digital signal.

4. The system according to claim 1, characterized in that, The data correction processing in the data association module includes geometric correction and time synchronization correction, and the correction method is as follows: Based on the associated dataset and the intrinsic and extrinsic parameters of the multispectral camera, ground control points are deployed in the trace detection acquisition area. The surface condition image coordinates and three-dimensional spatial point cloud coordinates of the ground control points are extracted based on the surface condition image data and the three-dimensional spatial point cloud data. The surface condition image coordinates and three-dimensional spatial point cloud coordinates are associated with the world coordinates of the ground control points through a geometric transformation matrix to establish a coordinate association architecture. Based on the coordinate association architecture, the surface condition image is resampled and projected using a geometric correction model to complete geometric correction. Based on the common features in the surface condition image data and the three-dimensional spatial point cloud data, the collected surface condition image data and three-dimensional spatial point cloud data are time-synchronized and corrected by the nearest neighbor interpolation method. The time-synchronized and corrected surface condition image data and three-dimensional spatial point cloud data are then organized into a structured array containing the pixel coordinates of the surface condition image, spectral information, three-dimensional spatial point cloud coordinates, and corresponding timestamps.

5. The system according to claim 1, characterized in that, The generation of the multispectral point cloud dataset in the data association module includes: By performing spectral enhancement and data correction processing on the received surface condition image data, a surface condition image containing reflectance information and spatial coordinates is output; the received three-dimensional spatial point cloud data is processed by denoising and filtering to retain the effective point cloud, and a continuous point cloud in the global coordinate system is generated based on the effective point cloud through registration. Based on the corrected surface condition image, spectral reflectance information of the pixels in the surface condition image is extracted pixel by pixel. Based on the spectral reflectance information, the actual spatial ray corresponding to the pixel in the surface condition image is calculated using a projection model. The three-dimensional points in the three-dimensional spatial point cloud data are traversed, and the point closest to the actual spatial ray is marked as the matching point. The spectral information of the pixel in the surface condition image is assigned to the corresponding three-dimensional point in the three-dimensional spatial point cloud data. After point-by-point matching, a multispectral point cloud dataset containing spatial coordinates and spectral reflectance data of the three-dimensional points is formed.

6. The system according to claim 1, characterized in that, The surface condition geometric model in the texture mapping module is generated in the following way: The three-dimensional spatial point cloud data is optimized by removing redundant noise points and invalid data. Based on the optimized three-dimensional spatial point cloud data, global registration of the three-dimensional spatial point cloud is performed in conjunction with the associated dataset, unifying the point clouds from different perspectives into the global coordinate system. Based on the density and distribution of three-dimensional spatial point cloud data, the division rules of triangular mesh are determined by the triangulation method, the three-dimensional spatial point cloud data is transformed into a triangular mesh model, and the size and shape of the triangles are adjusted based on the division rules. The triangular mesh model is topologically optimized by removing non-manifold edges and isolated faces, and a surface condition geometric model is output.

7. The system according to claim 1, characterized in that, The texture mapping technology in the texture mapping module includes: The triangular mesh model of the surface condition geometric model is assigned horizontal and vertical coordinates, and the surface of the three-dimensional mesh model is unfolded into two-dimensional planar coordinates; based on the premise of minimizing stretching deformation, the triangular facets and regions on the two-dimensional horizontal and vertical planes are mapped and paired through a mapping unfolding algorithm; Based on the associated dataset, a spatial coordinate transformation relationship between the multispectral camera and the lidar is established; based on the projection matrix of the multispectral camera, the three-dimensional spatial projection coordinates of the multispectral image pixels are calculated, and the corresponding geometric model mesh patches are determined based on the three-dimensional spatial projection coordinates; based on the horizontal-vertical coordinates of the geometric model mesh patches, the positions of pixel color and texture information on the two-dimensional horizontal-vertical plane are determined. Based on the matching results of the geometric model mesh patches, pixel information of the corresponding region of the surface condition image is collected and assigned to the corresponding positions in the horizontal-vertical plane.

8. The system according to claim 7, characterized in that, The result of the texture mapping technique is optimized as follows: Feathering technology is used to similarize the texture edges of adjacent horizontal and vertical blocks. By adjusting the color values ​​of the pixels at the edges of the horizontal and vertical blocks, the color difference at the seams is reduced. A texture stretching compensation algorithm is used to correct the texture deformation in areas of abrupt curvature changes. Combined with the reflectivity information in the multispectral point cloud dataset, texture enhancement is performed on specific trace detection target areas. Based on the accuracy verification procedure, a geometric model mesh patch is randomly selected, and the color deviation between the texture of the geometric model mesh patch and the texture of the actual surface condition image is compared. If the deviation exceeds the threshold, a correction procedure is triggered to adjust the horizontal-vertical coordinates and perform resampling and mapping.

9. The system according to claim 1, characterized in that, The method for removing holes and cracks from the visual representation in the terminal interaction module is as follows: By traversing the triangular facets of the surface condition geometric model, the boundary edges of the triangular facets that are not shared by adjacent facets are detected. When the boundary edges form a closed loop, it is determined to be a hole region, and the coordinates of the boundary vertex, the outline shape, and the area of ​​the hole are recorded. The angle between the normal vectors of adjacent triangular facets is calculated. When the angle between the normal vectors exceeds a preset threshold and the edge distance is less than a preset distance, it is determined to be a crack region, and the start and end points and extension path of the crack are located. Select the effective vertex closest to the boundary inside the hole boundary as the seed point, and generate new triangular patches gradually towards the hole region with the seed point as the center, based on the principle of triangulation. Based on the detected crack region, the crack width is reduced to within a preset width by adjusting the position coordinates of the vertices on both sides of the crack; then, the boundary edges on both sides of the crack are merged into a shared edge by using an edge fusion algorithm to generate a new triangular patch to connect the crack region.

10. The system according to claim 1, characterized in that, The data transmission method in the terminal interaction module is as follows: Structured datasets are generated from information obtained through trace evidence analysis; After the robot dog establishes a connection with the remote monitoring terminal, it negotiates transmission parameters through a handshake protocol. The robot dog monitors the transmission progress, signal strength, and battery level in real time. If the transmission is interrupted, it automatically saves the transmission breakpoint information and resumes transmission after the connection is restored. If the battery level is lower than the threshold, it prioritizes the transmission of key trace information, pauses the transmission and records the status, and resumes transmission after charging. The remote monitoring terminal receives data through dedicated client software and loads a 3D model into a 3D visualization engine; the structured dataset is associated with the spatial coordinates of the 3D model, and spectral curves, size parameters, and analysis conclusions are displayed in the trace area of ​​the 3D model.

Images