A three-dimensional human body modeling method and system for underground space rescue

Abstract

This invention discloses a 3D human body modeling method for underground space rescue, comprising: acquiring multi-view images of the accident area in the underground space environment and recording the corresponding camera pose information; identifying human body regions in the images and obtaining key points of human body joints; estimating human body shape parameters and posture parameters using a parametric human body model to obtain an initial 3D mesh model of the human body; constructing a standard human body posture space, mapping 3D points in the observation space to the standard human body posture space to obtain standard spatial coordinates; encoding spatial features of the standard spatial coordinates, inputting the spatial encoded features and observation direction into a neural radiation field model to predict the volume density and color information of spatial points; optimizing the neural radiation field model by minimizing the difference between the rendered image and the actual acquired image, and outputting a 3D model of the human body target in the underground space. This invention can achieve efficient reconstruction of the 3D structure and posture information of trapped personnel in complex underground rescue environments.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional reconstruction technology, specifically to a three-dimensional human body modeling method and system for underground space rescue. Background Technology

[0002] With the acceleration of urbanization, underground spaces (such as subway tunnels, mines, underground utility tunnels, and parking lots) have become an important part of urban operations. However, when these spaces encounter sudden disasters such as collapses, fires, or explosions, they often exhibit extreme environmental characteristics such as complex structures, lack of lighting, dense smoke, and multiple obstructions. In such confined scenarios, quickly and accurately obtaining the three-dimensional spatial location and posture information of trapped personnel is a core prerequisite for formulating precise rescue strategies, assessing the condition of the injured, and avoiding secondary injuries. Detailed three-dimensional human body modeling can not only intuitively display the location of trapped personnel, but also restore their limb posture (such as whether they are buried or whether their limbs are compressed), providing rescuers with crucial decision-making information.

[0003] Currently, traditional rescue detection methods mainly rely on thermal imagers, life detection radars, or handheld cameras. While these methods can achieve preliminary localization of human targets, the output information is mostly limited to a two-dimensional plane or simple point cloud distribution, lacking a detailed three-dimensional description of human posture and the spatial relationship with the surrounding environment. Existing computer vision-based 3D reconstruction technologies include some schemes that attempt to utilize Structure from Motion (SfM) or Multi-View Stereo (MVS) techniques for scene reconstruction. However, these traditional geometric reconstruction methods heavily rely on environmental texture and lighting conditions. In the low-light, weak-texture backgrounds commonly found in underground spaces, they are prone to feature point matching failures, leading to reconstruction holes or decreased accuracy. Furthermore, these methods primarily target static environmental geometry; for non-rigid targets like the human body with variable postures, they struggle to effectively capture their detailed limb structures and morphological features.

[0004] In recent years, the rise of Neural Radiance Fields (NeRF) technology has brought breakthroughs to high-fidelity 3D reconstruction. NeRF, through implicit neural network representation, can learn the continuum density and color distribution of a scene from multi-view images, generating highly realistic 3D models with strong robustness to environments lacking texture. Therefore, introducing NeRF into the field of human body modeling is considered a key path to solving the problems of poor reconstruction quality and loss of detail in traditional methods. Theoretically, using NeRF technology, it is possible to recover human body models containing clothing folds and fine geometry from sparse viewpoints.

[0005] However, directly applying NeRF to human reconstruction in underground space rescue scenarios still faces severe challenges, with existing technologies exhibiting significant bottlenecks: First, there are issues with learning efficiency and pose sensitivity. NeRF typically requires a large amount of viewpoint data and a long training period. In underground rescue scenarios, data acquisition is often limited by sparse viewpoints and varying human poses. Humans at accident scenes have complex poses and are situated in non-standard coordinate systems. NeRF is extremely sensitive to pose changes, and direct modeling can easily lead to feature learning chaos due to pose inconsistencies, making convergence difficult. Second, there is a conflict between computational resources and real-time requirements. Underground spaces are usually large, and performing high-resolution NeRF modeling on the entire scene would result in enormous memory overhead and a long training time. Existing fully connected neural networks or regular grid encoding cannot achieve high resolution with limited computational resources, failing to meet the stringent requirements of rescue missions.

[0006] To address the aforementioned technical bottlenecks, current technologies have not yet proposed a solution that can balance modeling accuracy, efficiency, and environmental adaptability. In particular, there are still technological gaps in how to standardize human poses to improve the stability of NeRF learning and how to achieve efficient spatial feature encoding with limited computing resources. Summary of the Invention

[0007] To address the aforementioned problems, this invention proposes a 3D human body modeling method and system for underground space rescue. This invention introduces a parametric human body model (such as SMPL) to construct a standard human body space. By transforming the skeletal posture, observation points are mapped to this standard space, effectively eliminating the interference of posture changes on NeRF feature learning and solving the convergence problem under sparse viewpoints. Secondly, a multi-resolution hash encoding based on MurmurHash is used to replace traditional position encoding. This hierarchical feature grid significantly reduces storage overhead, achieving efficient and high-fidelity reconstruction with limited resources. This invention can quickly recover the 3D structure and posture information of trapped personnel in complex underground environments, possessing advantages such as high modeling efficiency, strong expressive power, and adaptability to extreme environments.

[0008] To achieve the above objectives, this invention provides a three-dimensional human body modeling method for underground space rescue, the method comprising: Collect multi-view images of the accident area in an underground space environment and record the corresponding camera pose information; Preprocess the multi-view images to identify human body regions in the images and obtain key points of human joints. Use the parametric human body model to estimate human body shape parameters and posture parameters to obtain an initial three-dimensional mesh model of the human body. Construct a standard human posture space, and map three-dimensional points in the observation space to the standard human posture space according to the posture transformation relationship of human skeletal joints to obtain standard space coordinates; Spatial feature encoding is performed on standard spatial coordinates to obtain spatial encoded features; The spatial encoding features and the viewing direction are input into the neural radiation field model to predict the volume density and color information of spatial points. Based on the camera pose information, samples are taken along the camera ray, and a rendered image is generated by combining the volume rendering method. The neural radiation field model is optimized by minimizing the difference between the rendered image and the real acquired image, and a three-dimensional model of the human target in the underground space is output.

[0009] More preferably, the multi-view images are a sequence of multi-view images acquired by moving a mobile acquisition device; during the acquisition process, the mobile acquisition device is controlled to move within the accident area and acquire image frames at preset time intervals to form the multi-view image sequence; Using the positioning system of a mobile acquisition device or synchronous positioning and mapping technology, the camera pose information corresponding to each frame of image is recorded, and the camera pose information includes the camera position and rotation angle. Denoising, brightness enhancement, and distortion correction are performed on multi-view image sequences to generate a standardized multi-view image dataset.

[0010] Further preferably, the process of obtaining the initial three-dimensional human body mesh model includes: processing the preprocessed multi-view image using a human detection algorithm to identify the human target region; extracting human joint key point information from the human target region using a human key point detection method, wherein the human joint key point information includes skeletal point coordinates; introducing an SMPL parametric human model, using the extracted human joint key points as constraints, optimizing the objective function by minimizing the projection error of the human key points, and calculating human posture parameters and human shape parameters, wherein the human posture parameters include joint rotation angles, and the human shape parameters are used to characterize the human body shape; generating the initial three-dimensional human body mesh model based on the human posture parameters and human shape parameters, and obtaining the human skeleton transformation matrix.

[0011] Further preferably, the process of obtaining standard spatial coordinates includes: defining a standard spatial coordinate system for the human body in a preset standard posture, and using the standard spatial coordinate system as a unified reference system for feature learning; calculating the skeletal transformation matrix from the observation space to the human body standard posture space based on the human body shape parameters and posture parameters, wherein the skeletal transformation matrix is ​​used to describe the rotation and translation relationships of human body joints in different postures; for three-dimensional points in the observation space, calculating the corresponding coordinates of the three-dimensional points in the human body standard posture space based on the posture transformation matrix corresponding to each skeletal joint and the linear skinning weights corresponding to the three-dimensional points; and outputting the corresponding coordinates as standard spatial coordinates to the spatial feature encoding step.

[0012] Further preferably, the process of obtaining spatial coding features includes: using a spatial representation method based on multi-resolution hash coding to divide the human standard pose space into multiple scales and construct a multi-layer spatial resolution structure; at each spatial resolution, dividing the human standard pose space into a voxel grid and determining the corresponding voxel index according to the standard spatial coordinates; mapping the voxel index at each spatial resolution to a feature table of a fixed size to obtain the corresponding spatial feature vector; and concatenating or fusing the spatial feature vectors obtained at different spatial resolutions to form the final spatial coding features.

[0013] More preferably, the step of mapping the voxel index at each spatial resolution to a fixed-size feature table to obtain the corresponding spatial feature vector includes: inputting the voxel index at each spatial resolution into the MurmurHash function to obtain the corresponding hash value; determining the hash position of the voxel index in the fixed-size feature table based on the hash value and the size of the feature table corresponding to the spatial resolution; querying the corresponding spatial feature vector from the fixed-size feature table based on the hash position; and concatenating the spatial feature vectors obtained from different spatial resolutions to obtain spatial encoded features.

[0014] More preferably, the step of inputting spatial coding features and observation direction into the neural radiation field model to predict the volume density and color information of spatial points includes: establishing a neural network model and using the neural network model as the neural radiation field model; using the spatial coding features and observation direction obtained from standard spatial coordinates as input to the neural radiation field model; performing a nonlinear transformation on the spatial coding features and observation direction through the neural radiation field model to predict the volume density and color information of the corresponding spatial points; and outputting the volume density and color information to the volume rendering step so that the volume rendering step can accumulate colors based on multiple sampling points on the camera ray to obtain the corresponding image pixel color values.

[0015] More preferably, the step of sampling along the camera ray based on camera pose information and generating a rendered image using a volume rendering method, and optimizing the neural radiation field model by minimizing the difference between the rendered image and the real-world captured image, includes: determining the camera ray corresponding to a pixel in the real-world captured image based on the camera pose information; performing discrete sampling along the camera ray in space to obtain a series of spatial sampling points; predicting the volume density and color information of the spatial sampling points using the neural radiation field model, and calculating the light transmittance based on the volume density; accumulating the light transmittance and color information using the volume rendering formula to obtain the pixel color value corresponding to the camera ray; generating a rendered image based on multiple pixel color values, and calculating the difference loss between the rendered image and the real-world captured image; and optimizing the network parameters of the neural radiation field model and the parameters in the spatial feature encoding step using the backpropagation algorithm based on the difference loss, until the loss function converges.

[0016] More preferably, the output of a three-dimensional model of a human target in the underground space includes: after the neural radiation field model is trained, generating a three-dimensional model of the underground space environment and the human target, and displaying the three-dimensional model of the human target in a three-dimensional visualization manner, so that the human structure can be observed from any angle.

[0017] The present invention also provides a three-dimensional human body modeling system for underground space rescue, the system including a data acquisition module, a human body posture estimation module, a standard spatial mapping module, a spatial coding module, a neural radiation field modeling module, and a volume rendering optimization module; The data acquisition module is used to acquire multi-view images of the accident area in the underground space environment, record the corresponding camera pose information, and generate a standardized multi-view image dataset after preprocessing the multi-view images. The human pose estimation module is used to receive a standardized multi-view image dataset, identify human regions in the image and obtain key points of human joints, estimate human shape parameters and pose parameters using a parameterized human model, and obtain an initial three-dimensional mesh model of the human body. The standard space mapping module is used to construct a standard human posture space. Based on the posture transformation relationship of human skeletal joints, it maps three-dimensional points in the observation space to the standard human posture space to obtain standard space coordinates. The spatial encoding module is used to encode the spatial features of standard spatial coordinates to obtain spatial encoded features; The neural radiation field modeling module is used to input spatial coding features and observation direction into the neural radiation field model to predict the volume density and color information of spatial points; The volume rendering optimization module is used to sample along the camera ray based on the camera pose information and generate a rendered image by combining the volume rendering method. It optimizes the neural radiation field model by minimizing the difference between the rendered image and the real acquired image, and outputs a three-dimensional model of the human target in the underground space.

[0018] The beneficial effects of the three-dimensional human body modeling method and system provided by this invention are as follows: First, multi-view image data of the accident area is acquired in the underground space environment using unmanned system cluster equipment, and the corresponding camera pose information is recorded. Then, human joint information is obtained using human detection and keypoint recognition methods, and human posture and shape parameters are estimated through a parametric human model to obtain an initial 3D human model. Based on this, a standard human space is constructed, and 3D points in the observation space are mapped to the standard space coordinate system, thereby eliminating the influence of changes in human posture. Furthermore, a multi-resolution hashing encoding method based on MurmurHash is used to feature-encode the spatial coordinates, and the encoded features are input into a neural radiation field model to predict the volume density and color information of spatial points. The 3D structure of the human body is then reconstructed using a volume rendering method. This method can efficiently recover the 3D structure and posture information of trapped personnel in complex underground environments, providing intuitive and reliable 3D information support for underground space accident rescue. It has advantages such as high modeling efficiency, strong expressive power, and adaptability to complex environments. Attached Figure Description

[0019] To more clearly illustrate the technical solution of the present invention, the accompanying drawings are used to further explain the present invention.

[0020] Figure 1 This is a flowchart of a three-dimensional human body modeling method for underground space rescue according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of a three-dimensional human body modeling system for underground space rescue in an embodiment of the present invention; Figure 3 A schematic diagram of the human body standard spatial mapping network structure based on neural radiation fields; Figure 4 This is a schematic diagram illustrating the results of human body reconstruction in an underground space environment using the present invention. Figure 5 This is a schematic diagram illustrating the effect of multi-view human body reconstruction under specific actions using the present invention; Figure 6 This is a comparison diagram of the effects of the method of the present invention and traditional human body reconstruction methods; Figure 7 This is a diagram comparing the reconstruction indicators using different methods. Detailed Implementation

[0021] To make the technical solution of this invention clearer, the three-dimensional human body modeling method and system for underground space rescue proposed in this invention will be described in detail below with reference to specific embodiments. It should be noted that this embodiment is only used to illustrate the technical solution of this invention and is not intended to limit the scope of protection of this invention.

[0022] This invention provides a 3D human body modeling method for underground space rescue. This method is applicable to complex underground environments such as underground tunnels, mines, underground utility tunnels, and underground parking lots. It utilizes multi-view images, human pose estimation, spatial mapping of standard human poses, spatial feature encoding, neural radiation field modeling, and volume rendering optimization to form a complete human body modeling process from data acquisition to 3D visualization output. Figure 1 As shown, the method mainly includes the following steps: Step 1, acquiring multi-view images of the accident area in the underground space environment and recording the corresponding camera pose information; Step 2, preprocessing the multi-view images, identifying human body regions in the images and obtaining key points of human body joints, estimating human body shape parameters and posture parameters using a parametric human body model, and obtaining an initial three-dimensional mesh model of the human body; Step 3, constructing a standard human body posture space, mapping three-dimensional points in the observation space to the standard human body posture space according to the posture transformation relationship of human skeletal joints, and obtaining standard spatial coordinates; Step 4, performing spatial feature encoding on the standard spatial coordinates to obtain spatial encoded features; Step 5, inputting the spatial encoded features and observation direction into the neural radiation field model to predict the volume density and color information of spatial points; Step 6, sampling along the camera ray according to the camera pose information, and generating a rendered image by combining a volume rendering method, optimizing the neural radiation field model by minimizing the difference between the rendered image and the real acquired image, and outputting a three-dimensional model of the human target in the underground space.

[0023] Steps one through six are sequentially linked, with the output of the previous step serving as the input for the next. This process enables the recovery of the three-dimensional structure and posture information of a trapped human target in underground environments characterized by low light, weak texture, limited viewing angles, and complex human postures.

[0024] Example 1: Multi-view image acquisition and preprocessing.

[0025] In this embodiment, multi-view images of the accident area are first acquired in an underground space environment using a mobile acquisition device. The mobile acquisition device can be at least one of a drone, a mobile robot, or an unmanned rescue vehicle. The mobile acquisition device is equipped with a camera; preferably, the camera can be a low-light camera or a wide-angle camera to adapt to the complex environment of the underground space, such as weak textures, low light, and limited field of view.

[0026] Specifically, in underground spaces such as mines or tunnels, mobile acquisition devices are controlled to move within the accident area and acquire image frames at preset time intervals, forming a multi-view image sequence. By moving the mobile acquisition devices within the underground space, image data of the accident area from different perspectives can be obtained, thus providing multi-view visual input for subsequent 3D human body modeling.

[0027] While acquiring images, the positioning system of the mobile acquisition device or Simultaneous Localization and Mapping (SLAM) technology is used to record the camera pose information corresponding to each frame of the image. The camera pose information includes the camera position and rotation angle, which is used to subsequently determine the camera rays and establish the correspondence between the real acquired image and the spatial sampling points.

[0028] Furthermore, the acquired multi-view image sequences are transmitted to the processing system, where they undergo preprocessing including denoising, brightness enhancement, and distortion correction to generate a standardized multi-view image dataset. This preprocessing reduces the impact of factors such as low light levels, smoke, and lens distortion in underground spaces on subsequent human detection, key point extraction, and neural radiation field modeling.

[0029] Example 2: Human body detection, key point recognition and initial 3D mesh model generation.

[0030] After obtaining a standardized multi-view image dataset, human region recognition and human joint key point extraction are performed on the multi-view images to extract the prior of human three-dimensional structure from two-dimensional images.

[0031] Specifically, human detection algorithms are used to process the preprocessed multi-view images to identify human target regions in the images. These human detection algorithms can employ YOLO or similar algorithms to determine the target region of a trapped human body within the image.

[0032] After identifying the target human body region, human keypoint detection methods are used to extract key joint information from the target human body region. These key joint detection methods can include OpenPose or similar techniques. The key joint information includes skeletal point coordinates, such as the two-dimensional image coordinates of key human joints like the head, shoulders, elbows, wrists, hips, knees, and ankles.

[0033] Subsequently, a parametric human body model is introduced to estimate human posture. Preferably, the parametric human body model is an SMPL parametric human body model. Using the extracted human joint key points as constraints, the objective function is optimized by minimizing the projection error of the human key points to calculate human posture parameters and human shape parameters. The human posture parameters include joint rotation angles, and the human shape parameters characterize human body shape.

[0034] After obtaining the human posture and shape parameters, an initial 3D human mesh model is generated based on these parameters. This initial 3D human mesh model serves as the geometric prior for subsequent neural radiation field learning. Simultaneously, the human skeleton transformation matrix is ​​obtained based on the optimized posture parameters, and this matrix is ​​output to the subsequent standard space mapping step.

[0035] Example 3: Construction of standard human posture space and mapping of observation space points.

[0036] In this embodiment, a standard human pose space is constructed, and three-dimensional points in the observation space are mapped to the standard human pose space. This step addresses the difficulty of feature learning caused by the variability of human poses, enabling the human structure under different observation poses to be represented in a unified space, such as... Figure 3 The diagram shows the human body standard space mapping network structure of this invention. This network achieves accurate transformation from observation space coordinates to standard space coordinates, balancing the efficiency of linear pose transformation with the accuracy of nonlinear correction. The network takes the coordinates X of the three-dimensional sampling point in the observation space as input, and first passes through the skeleton pose transformation module. The linear skinning pose transformation is completed, initially mapping the observation space points to the standard space, and intermediate transformation results are obtained. To correct the local deformation errors present in traditional linear skinning transformation, intermediate results are simultaneously input into a multilayer perceptron network. The nonlinear coordinate residual correction is learned by this network. Finally, the linear transformation result is fused with the nonlinear correction to obtain accurate standard spatial coordinates. This provides a unified standard coordinate input for subsequent spatial feature encoding.

[0037] Specifically, a standard spatial coordinate system (canonical space) is defined for the human body in a preset standard pose, and this standard spatial coordinate system is used as a unified reference system for feature learning. The preset standard pose can be a T-pose. In this paper, both the human standard space and the human standard pose space refer to the standard spatial coordinate system for the human body in a preset standard pose, which is used as a unified reference system for learning human structural features.

[0038] Based on the human shape and posture parameters obtained in the human pose estimation step, a skeletal transformation matrix (T) is calculated from the observation space to the standard human pose space. This skeletal transformation matrix describes the rotational and translational relationships of human joints in different poses. Through this skeletal transformation matrix, the transformation relationship between human points in the observation space and corresponding points in the standard human pose space can be established.

[0039] For any three-dimensional point x in the observation space, its corresponding coordinates in the standard human pose space are calculated using linear skinning weights. The three-dimensional points in the observation space mentioned here specifically refer to spatial sampling points obtained by discrete sampling along the camera ray, covering the human target and its surrounding area to be modeled, and are not limited to the mesh vertices of the SMPL parametric model. This ensures that a uniform spatial transformation can be performed on all points to be modeled on the human surface and its surrounding space. The specific mapping process can be represented as follows: ; in, Represents standard spatial coordinates, This represents the pose transformation matrix corresponding to the k-th joint. This represents the skin weight of the point relative to the corresponding joint, where K is the number of joints in the human skeleton. This step maps human points from different poses to a standardized human pose space, eliminating the interference of pose changes on feature learning, and outputs the mapped standard space coordinates to the subsequent spatial feature encoding step.

[0040] For this sampling point, the linear skin weights are not only predefined for SMPL mesh vertices. For spatial sampling points that are not mesh vertices, we interpolate the predefined linear skin weights of the SMPL mesh vertices based on the relative position of the sampling point in the SMPL mesh: First, we query the SMPL mesh face closest to the sampling point. Based on the centroid coordinates of the sampling point in the face, we interpolate the skin weights of the three vertices of the face to obtain the skin weights of each joint corresponding to the sampling point, ensuring that sampling points that are not mesh vertices can also obtain accurate pose transformation weights.

[0041] Example 4: Spatial Feature Coding.

[0042] In this embodiment, spatial feature encoding is performed on standard spatial coordinates to obtain spatial encoded features. This step is used to improve spatial representation efficiency and provide efficient spatial feature input for the neural radiation field model.

[0043] Specifically, in the process of modeling the human neural radiation field, the standard human posture space is first divided into multiple scales. Let a certain point in the space be: ; Subsequently, a spatial representation method based on multi-resolution hash coding was adopted to divide the human standard pose space into multiple scales and construct a multi-layer spatial resolution structure.

[0044] Step 1: Multi-resolution mesh generation. Construct a multi-layered spatial resolution structure to hierarchically divide the standard human pose space. At the l-th spatial resolution layer, divide the 3D space into layers with a resolution of [missing information]. The voxel grid. The voxel coordinates corresponding to this spatial point can be represented as: ; in, Point The integer index of the voxel is used, and the three components correspond to the voxel grid numbers on the x, y, and z axes, respectively. This index can be used to determine which voxel the point belongs to in the voxel grid of the i-th layer, providing a query index for subsequent hash mapping.

[0045] Step 2, Hash Mapping and Feature Lookup. To reduce storage overhead, this implementation does not directly store the complete voxel grid. Instead, it maps voxel indices to a fixed-size feature table using a hash function. Specifically, the MurmurHash function can be used for hash mapping: ; in, Represents the MurmurHash function, This indicates the size of the l-th level hash table. This represents the hash position after voxel index mapping. It should be noted that the spatial feature vectors in the fixed-size feature table are all trainable parameters. During model training, these parameters, along with the network parameters of the neural radiation field model, are iteratively updated using the backpropagation algorithm based on the rendering loss. This ensures that the spatial features can adapt to the structural features of the current human target, improving modeling accuracy.

[0046] This hash mapping allows us to retrieve the corresponding spatial feature vector from a fixed-size feature table. ; in, This represents the feature table of the l-th layer. Represents the l-th layer with respect to point Corresponding spatial features.

[0047] Step 3, Feature Fusion. The spatial feature vectors extracted from each layer of the multi-resolution structure are concatenated or fused to form the final spatial coding feature vector. Within the multi-resolution coding framework, spatial features at different scales are combined to obtain the final spatial feature representation: ; Where L represents the number of spatial coding layers, This indicates a feature concatenation operation. Through the multi-resolution hash encoding process described above, multi-scale spatial details can be captured while reducing storage overhead, providing spatially encoded features for subsequent neural radiation field modeling.

[0048] Example 5: Modeling of neural radiation fields.

[0049] After obtaining the spatial encoding features, they are input into a neural network to establish a neural radiation field model, thereby predicting the volume density and color information of spatial points.

[0050] Specifically, a neural network model is established and used as a neural radiation field model. This neural network model can typically be an MLP (Multi-Level Processing). Spatial encoding features obtained from standard spatial coordinates are then used. The observation direction d is used as the input node of the neural radiation field model. Here, x represents the standard spatial coordinates, and d represents the observation direction. The observation direction d is calculated from the camera pose information, specifically the camera ray direction corresponding to the pixel. This direction is calculated in the observation space without requiring standard spatial coordinate transformation and is directly used as the model input to represent the current observation viewpoint, supporting viewpoint-related color prediction.

[0051] The neural radiation field model performs a nonlinear transformation on spatial encoded features and viewing direction to predict the volume density and color information of corresponding spatial points. The network predicts the volume density and color information of spatial points through nonlinear transformation. The neural radiation field function is expressed as: ; in, Let c represent volume density and 'c' represent color information. The volume density and color information do not directly form a complete rendered image; instead, they are output to the volume rendering step, whereby the volume rendering step accumulates colors based on multiple sampling points on the camera ray to obtain the corresponding image pixel color values.

[0052] In this embodiment, parameter initialization of the neural radiation field model is also required. After initializing the network weights, the neural radiation field model receives error feedback from the volume rendering step and optimizes its parameters through backpropagation.

[0053] Example 6: Volume rendering optimization.

[0054] In this embodiment, the volume rendering step is responsible for converting the spatial point volume density and color information predicted by the neural radiation field model into a visual image, and training the entire model through error feedback closed loop.

[0055] Step 1, Light Sampling. Based on the camera pose information, determine the camera ray corresponding to the pixels in the actual captured image, and then sample along the camera ray. Discrete sampling is performed in three-dimensional space to obtain a series of spatial sampling points. This is the starting point of the camera ray; Let be the direction vector of the camera ray; This is the distance parameter of the ray. This step establishes the correspondence between camera pixels, camera rays, and 3D spatial sampling points.

[0056] Step two, pixel value generation. Using the volume density and color information of spatial sampling points predicted by the neural radiation field model, the light transmittance and final pixel color value are calculated using the volume rendering formula: ; in, This represents the transmittance during light propagation. Regarding the volume rendering formula above, the specific meanings of each symbol are as follows: r represents the camera ray; The final pixel color value corresponding to the camera ray is the rendered color of the pixel calculated through volume rendering; N is the total number of spatial sampling points obtained by discrete sampling along the camera ray. The sorting index of the sampling points is the volume density of the i-th sampling point; The spatial distance between the i-th sampling point and the next adjacent sampling point; The weight coefficient for the i-th sampling point; represents the color information of the i-th sampling point; j is the sampling point index in the transmittance calculation process; , Let be the volume density of the j-th sampling point and the adjacent sampling interval, respectively.

[0057] By accumulating light transmittance and color information using volume rendering formulas, the pixel color values ​​corresponding to camera rays can be obtained. A rendered image is then generated based on these multiple pixel color values.

[0058] Step 3, Backpropagation and Optimization. Calculate the difference loss between the rendered image and the real-world captured image, and train the neural network parameters by minimizing this difference loss. The loss function can be expressed as: ; Where p is the pixel index, used to traverse all pixels in the actual captured image; The rendered color value of the p-th pixel predicted by the model; This is the actual color value of the p-th pixel.

[0059] Based on the aforementioned difference loss, the backpropagation algorithm is used to optimize the network parameters of the neural radiation field model and the parameters in the spatial feature encoding step. In other words, error feedback is used not only to update the network parameters in the neural radiation field model but also to update the relevant parameters in the spatial encoding module, enabling the spatial encoded features and the neural radiation field model to jointly adapt to the real-world acquired images.

[0060] Step four, model convergence. Repeat the rendering and optimization process described above until the loss function converges, obtaining a trained 3D human body structure model. After training, output the 3D models of the underground space environment and the human target.

[0061] Example 7: 3D model output and visualization.

[0062] After the neural radiation field model is trained, a three-dimensional model of the underground space environment and human targets is generated based on the trained neural radiation field model.

[0063] The final output 3D model is based on a trained implicit neural radiation field model. This model supports rendering from any viewpoint, and high-precision human body mesh models, point cloud models, or volume data can be extracted from this implicit model using the Marching Cubes algorithm to adapt to different visualization and analysis needs. Among them, the underground space environment model is obtained by modeling the human target synchronously during the modeling process: In the data sampling and model training stage, in addition to the human target area, the system simultaneously samples and models the spatial points of the background area of ​​the underground space. The system completes the learning of the volume density and color information of the environmental area through the same neural radiation field framework, and finally obtains the three-dimensional environmental model that is unified with the human target in the same spatial coordinate system.

[0064] Furthermore, displaying a 3D model of the human body target using 3D visualization allows rescuers to observe the human structure from any perspective. The 3D visualization results allow for observation of the trapped person's overall posture, limb spatial position, and spatial relationship between the human target and its surrounding environment, thus assisting rescuers in determining the trapped person's posture and spatial relationship with the surrounding environment.

[0065] In this embodiment, the 3D model output includes not only the 3D structure of the human target itself, but also relevant spatial information from the underground environment. By displaying the underground environment and the 3D model of the human target in a unified manner, intuitive and reliable 3D information support can be provided for underground accident rescue.

[0066] To verify the effectiveness, modeling accuracy, and environmental adaptability of the three-dimensional human body modeling method for underground space rescue of the present invention, this embodiment designs a comparative experiment to conduct performance verification. The experimental process and results are analyzed as follows.

[0067] The experimental group used the modeling method proposed in this invention to achieve three-dimensional human body modeling through the entire process of data acquisition, human pose estimation, standard spatial mapping, multi-resolution hash coding, neural radiation field modeling, and volume rendering optimization.

[0068] The control group consisted of a traditional method and the original photographs as a comparison. The traditional method could be a method that applies HumanNeRF to human body modeling. The control group used the same dataset as the experimental group for modeling.

[0069] The method of this invention achieves the following effect in the overall reconstruction of the human body in underground space: Figure 4 As shown, Figure 4 The lower middle section shows the actual captured image, while the upper section shows the 3D reconstruction visualization generated using the method of this invention. It can be clearly seen that the reconstruction result completely restores the overall posture of the trapped human body, without issues such as overall contour distortion or spatial position shift, meeting the visual requirements of underground rescue for overall scene and human body modeling.

[0070] To verify the robustness of the method of this invention in reconstructing human postures from different perspectives, a typical non-standard posture simulating the limb bending of a trapped person was selected, and the reconstruction effect from different observation perspectives was demonstrated. The results are as follows: Figure 5 As shown. Figure 5 The reconstructed images from multiple perspectives can restore the bending angles of human joints, the spatial placement of limbs, and details such as clothing folds and limb outlines are clearly discernible. Even from non-frontal perspectives such as side views and top views, there are no obvious problems such as missing local structures or posture distortion, indicating that the method of the present invention can still achieve multi-perspective reconstruction of human posture under sparse perspectives.

[0071] The visual effects of the method of this invention are compared with those of traditional reconstruction methods, and the results are as follows: Figure 6 As shown, Figure 6 The left side shows the reconstruction result using the traditional HumanNeRF method, while the right side shows the reconstruction result using the method of this invention. Traditional methods often result in blurred local contours, loss of limb details, and poor integration with the background environment. In contrast, the method of this invention reconstructs human limbs with complete details and clear contours, demonstrating the superior visual reconstruction quality of this invention.

[0072] like Figure 7 As shown, the evaluation metrics are a comprehensive evaluation combining general objective metrics in the field of 3D reconstruction and subjective metrics specific to the rescue scenario. These evaluation metrics include Peak Signal-to-Noise Ratio (PSNR), Structural Similarity (SSIM), and Learned Perceptual Patch Similarity (LPIPS). Higher PSNR and SSIM values ​​indicate a higher similarity between the reconstructed image and the real image, while lower LPIPS values ​​indicate better visual perception of the reconstruction. As can be seen from the figure, the evaluation metrics of the reconstruction method of this invention are significantly higher than those of the traditional HumanNeRF method.

[0073] Combination Figure 7 Quantitative indicators and Figure 4 , Figure 5 , Figure 6 The visual effect verification shows that the three-dimensional human body modeling method proposed in this invention can generate a high-fidelity three-dimensional human body model, accurately restore the posture of trapped personnel, meet the technical requirements of three-dimensional human body modeling for underground space accident rescue, and has practical engineering application value.

[0074] Based on the above-described method embodiments, the present invention also provides a three-dimensional human body modeling system for underground space rescue. For example... Figure 2 As shown, the system includes a data acquisition module, a human pose estimation module, a standard spatial mapping module, a spatial coding module, a neural radiation field modeling module, and a volume rendering optimization module. These modules are connected sequentially to form a complete pipeline from data input to 3D visualization.

[0075] The data acquisition module, serving as the system's input, is used to acquire multi-view images of the accident area in the underground space environment and record the corresponding camera pose information. After preprocessing the multi-view images, a standardized multi-view image dataset is generated. Specifically, the data acquisition module can utilize a mobile acquisition device to complete the data acquisition. This mobile acquisition device can be a drone, mobile robot, or unmanned rescue vehicle, and can be equipped with low-light or wide-angle cameras. During the acquisition process, the mobile acquisition device is controlled to move within the accident area and acquire image frames at preset time intervals, forming a multi-view image sequence. Simultaneously, the positioning system or simultaneous localization and mapping (SMR) technology of the mobile acquisition device is used to record the camera pose information corresponding to each image frame. Subsequently, the multi-view image sequence undergoes denoising, brightness enhancement, and distortion correction processing to generate a standardized multi-view image dataset, which is then output to the human pose estimation module.

[0076] The human pose estimation module receives a standardized multi-view image dataset, identifies human regions in the images and obtains key points of human joints, estimates human shape and pose parameters using a parametric human model, and obtains an initial 3D human mesh model. Specifically, the human pose estimation module processes the preprocessed image sequence using a human detection algorithm to identify human target regions and extracts key point information of human joints using a human key point detection method; further, it introduces the SMPL parametric human model, using the extracted human key points as constraints, optimizes the objective function by minimizing projection error, calculates human pose and shape parameters, and generates an initial 3D human mesh model based on the optimized parameters, while simultaneously obtaining the human skeleton transformation matrix.

[0077] The standard space mapping module is used to construct a standard human posture space. Based on the posture transformation relationships of the human skeletal joints, it maps three-dimensional points in the observation space to the standard human posture space to obtain standard space coordinates. Specifically, the standard space mapping module defines a standard space coordinate system for the human body in a preset standard posture and uses this standard space coordinate system as a unified reference system for feature learning. Based on the human body shape parameters and posture parameters, it calculates the skeletal transformation matrix from the observation space to the standard human posture space, and uses the posture transformation matrix corresponding to each skeletal joint and the linear skinning weights corresponding to the three-dimensional points to calculate the corresponding coordinates of the three-dimensional points in the observation space in the standard human posture space.

[0078] The spatial encoding module is used to encode spatial features of standard spatial coordinates to obtain spatial encoded features. Specifically, the spatial encoding module adopts a spatial representation method based on multi-resolution hash encoding to divide the human standard pose space into multiple scales and construct a multi-layer spatial resolution structure. At each spatial resolution, the human standard pose space is divided into a voxel grid, and the corresponding voxel index is determined according to the standard spatial coordinates. Furthermore, the voxel index at each spatial resolution is input into the MurmurHash function to obtain the corresponding hash value. The hash position is determined according to the hash value and the size of the feature table corresponding to the spatial resolution, and the corresponding spatial feature vector is queried from the feature table of a fixed size. Finally, the spatial feature vectors obtained at different spatial resolutions are concatenated or fused to form the final spatial encoded features.

[0079] The neural radiation field modeling module is used to input spatial encoding features and viewing direction into the neural radiation field model to predict the volume density and color information of spatial points. Specifically, the neural radiation field modeling module establishes a neural network model and uses this neural network model as the neural radiation field model; it uses the spatial encoding features and viewing direction obtained from standard spatial coordinates as input to the neural radiation field model; it predicts the volume density and color information of the corresponding spatial points through nonlinear transformation, and outputs the volume density and color information to the volume rendering optimization module.

[0080] The volume rendering optimization module is used to sample along the camera ray based on camera pose information and generate a rendered image using a volume rendering method. It optimizes the neural radiation field model by minimizing the difference between the rendered image and the actual captured image, outputting a 3D model of the human target in the underground space. Specifically, the volume rendering optimization module determines the camera ray corresponding to the pixels in the actual captured image based on the camera pose information, performs discrete sampling along the camera ray in space to obtain a series of spatial sampling points; uses the neural radiation field model to predict the volume density and color information of the spatial sampling points, and calculates the light transmittance based on the volume density; accumulates the light transmittance and color information using the volume rendering formula to obtain the pixel color value corresponding to the camera ray; generates a rendered image based on multiple pixel color values ​​and calculates the difference loss between the rendered image and the actual captured image; based on the difference loss, it uses the backpropagation algorithm to optimize the network parameters of the neural radiation field model and the parameters in the spatial feature encoding step until the loss function converges. After the model training is completed, it outputs a 3D model of the underground space environment and the human target, and displays the 3D model of the human target in a 3D visualization manner, enabling observation of the human structure from any perspective.

[0081] In summary, this invention, through the collaborative processing of multi-view image acquisition, human pose estimation, human standard pose spatial mapping, spatial feature encoding, neural radiation field modeling, and volume rendering optimization, can achieve three-dimensional modeling of human targets in complex underground spatial environments, providing three-dimensional information support for underground accident rescue.

[0082] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention. Any equivalent substitutions or obvious modifications made by those skilled in the art based on the description and claims of the present invention without departing from the technical solutions of the present invention should fall within the scope of protection of the present invention.

Claims

1. A three-dimensional human body modeling method for underground space rescue, characterized in that, include: Collect multi-view images of the accident area in an underground space environment and record the corresponding camera pose information; Preprocess the multi-view images to identify human body regions in the images and obtain key points of human joints. Use the parametric human body model to estimate human body shape parameters and posture parameters to obtain an initial three-dimensional mesh model of the human body. Construct a standard human posture space, and map three-dimensional points in the observation space to the standard human posture space according to the posture transformation relationship of human skeletal joints to obtain standard space coordinates; Spatial feature encoding is performed on standard spatial coordinates to obtain spatial encoded features; The spatial encoding features and the viewing direction are input into the neural radiation field model to predict the volume density and color information of spatial points. Based on the camera pose information, samples are taken along the camera ray, and a rendered image is generated by combining the volume rendering method. The neural radiation field model is optimized by minimizing the difference between the rendered image and the real acquired image, and a three-dimensional model of the human target in the underground space is output.

2. The three-dimensional human body modeling method for underground space rescue according to claim 1, characterized in that, The multi-view images are a sequence of multi-view images acquired by moving a mobile acquisition device; during the acquisition process, the mobile acquisition device is controlled to move within the accident area and acquire image frames at preset time intervals to form the multi-view image sequence. Using the positioning system of a mobile acquisition device or synchronous positioning and mapping technology, the camera pose information corresponding to each frame of image is recorded, and the camera pose information includes the camera position and rotation angle. Denoising, brightness enhancement, and distortion correction are performed on multi-view image sequences to generate a standardized multi-view image dataset.

3. The three-dimensional human body modeling method for underground space rescue according to claim 1, characterized in that, The process of obtaining the initial 3D human body mesh model includes: processing the preprocessed multi-view images using a human detection algorithm to identify the human target region; extracting human joint key point information from the human target region using a human key point detection method, the human joint key point information including skeletal point coordinates; introducing an SMPL parametric human model, using the extracted human joint key points as constraints, optimizing the objective function by minimizing the projection error of the human key points, and calculating human posture parameters and human shape parameters, the human posture parameters including joint rotation angles, and the human shape parameters used to characterize the human body shape; generating the initial 3D human body mesh model based on the human posture parameters and human shape parameters, and obtaining the human skeleton transformation matrix.

4. The three-dimensional human body modeling method for underground space rescue according to claim 1, characterized in that, The process of obtaining standard spatial coordinates includes: defining a standard spatial coordinate system for the human body in a preset standard posture, and using the standard spatial coordinate system as a unified reference system for feature learning; calculating the skeletal transformation matrix from the observation space to the human standard posture space based on the human body shape parameters and posture parameters, wherein the skeletal transformation matrix is ​​used to describe the rotation and translation relationships of human joints in different postures; for a three-dimensional point in the observation space, calculating the corresponding coordinates of the three-dimensional point in the human standard posture space based on the posture transformation matrix corresponding to each skeletal joint and the linear skinning weight corresponding to the three-dimensional point; and outputting the corresponding coordinates as standard spatial coordinates to the spatial feature encoding step.

5. The three-dimensional human body modeling method for underground space rescue according to claim 1, characterized in that, The process of obtaining spatial coding features includes: using a spatial representation method based on multi-resolution hash coding to divide the human standard pose space into multiple scales and construct a multi-layer spatial resolution structure; at each spatial resolution, dividing the human standard pose space into a voxel grid and determining the corresponding voxel index according to the standard spatial coordinates; mapping the voxel index at each spatial resolution to a feature table of a fixed size to obtain the corresponding spatial feature vector; and concatenating or fusing the spatial feature vectors obtained at different spatial resolutions to form the final spatial coding features.

6. The three-dimensional human body modeling method for underground space rescue according to claim 5, characterized in that, The step of mapping the voxel index at each spatial resolution to a fixed-size feature table to obtain the corresponding spatial feature vector includes: inputting the voxel index at each spatial resolution into the MurmurHash function to obtain the corresponding hash value; determining the hash position of the voxel index in the fixed-size feature table based on the hash value and the size of the feature table corresponding to that spatial resolution; querying the corresponding spatial feature vector from the fixed-size feature table based on the hash position; and concatenating the spatial feature vectors obtained from different spatial resolutions to obtain spatial encoded features.

7. The three-dimensional human body modeling method for underground space rescue according to claim 1, characterized in that, The step of inputting spatial coding features and observation direction into a neural radiation field model to predict the volume density and color information of spatial points includes: establishing a neural network model and using the neural network model as the neural radiation field model; using spatial coding features and observation direction obtained from standard spatial coordinates as input to the neural radiation field model; performing a nonlinear transformation on the spatial coding features and observation direction through the neural radiation field model to predict the volume density and color information of the corresponding spatial points; and outputting the volume density and color information to the volume rendering step so that the volume rendering step can accumulate colors based on multiple sampling points on the camera ray to obtain the corresponding image pixel color values.

8. The three-dimensional human body modeling method for underground space rescue according to claim 1, characterized in that, The process of sampling along camera rays based on camera pose information and generating a rendered image using a volume rendering method, optimizing the neural radiation field model by minimizing the difference between the rendered image and the actual captured image, includes: determining the camera ray corresponding to a pixel in the actual captured image based on camera pose information; performing discrete sampling along the camera ray in space to obtain a series of spatial sampling points; predicting the volume density and color information of the spatial sampling points using the neural radiation field model, and calculating the light transmittance based on the volume density; accumulating the light transmittance and color information using a volume rendering formula to obtain the pixel color value corresponding to the camera ray; generating a rendered image based on multiple pixel color values ​​and calculating the difference loss between the rendered image and the actual captured image; and optimizing the network parameters of the neural radiation field model and the parameters in the spatial feature encoding step using a backpropagation algorithm based on the difference loss until the loss function converges.

9. The three-dimensional human body modeling method for underground space rescue according to claim 1, characterized in that, Output a 3D model of a human target in underground space, including: after the neural radiation field model is trained, generate a 3D model of the underground space environment and the human target, and display the 3D model of the human target in a 3D visualization manner, so that the human structure can be observed from any perspective.

10. A three-dimensional human body modeling system for underground space rescue, characterized in that, It includes a data acquisition module, a human pose estimation module, a standard spatial mapping module, a spatial coding module, a neural radiation field modeling module, and a volume rendering optimization module; The data acquisition module is used to acquire multi-view images of the accident area in the underground space environment, record the corresponding camera pose information, and generate a standardized multi-view image dataset after preprocessing the multi-view images. The human pose estimation module is used to receive a standardized multi-view image dataset, identify human regions in the image and obtain key points of human joints, estimate human shape parameters and pose parameters using a parameterized human model, and obtain an initial three-dimensional mesh model of the human body. The standard space mapping module is used to construct a standard human posture space. Based on the posture transformation relationship of human skeletal joints, it maps three-dimensional points in the observation space to the standard human posture space to obtain standard space coordinates. The spatial encoding module is used to encode the spatial features of standard spatial coordinates to obtain spatial encoded features; The neural radiation field modeling module is used to input spatial coding features and observation direction into the neural radiation field model to predict the volume density and color information of spatial points; The volume rendering optimization module is used to sample along the camera ray based on the camera pose information and generate a rendered image by combining the volume rendering method. It optimizes the neural radiation field model by minimizing the difference between the rendered image and the real acquired image, and outputs a three-dimensional model of the human target in the underground space.

Images