A three-dimensional graph convolution-based case scene structured modeling method

Abstract

This invention discloses a structured crime scene modeling method based on 3D graph convolution, comprising the following steps: collecting data to generate a multi-source crime scene input dataset; constructing a 3D spatial graph of the crime scene, establishing graph edges based on the constraint set of structural relationships between nodes, and generating the 3D spatial graph; initializing node features and edge weights to generate an initial graph structure; performing 3D graph convolution processing on the initial graph structure, constraining the convolutional neighborhoods of nodes, performing neighborhood-based convolution operations on the features of nodes within the neighborhoods, and generating an intermediate feature map; performing graph pooling operations on the intermediate feature map to generate a compressed structural representation of the crime scene; and decoding and modeling the compressed structural representation to obtain a structured 3D scene representation. This invention constructs a structured crime scene model based on 3D graph convolution, realizing the fusion of multi-source data and criminal investigation relationships, and possesses the advantages of high semantic fidelity, complete structural representation, and strong modeling reliability.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional scene reconstruction technology, and in particular to a structured modeling method for crime scenes based on three-dimensional graph convolution. Background Technology

[0002] Existing 3D modeling techniques for crime scenes mainly rely on image reconstruction, laser mapping, and point cloud fusion to reconstruct the spatial structure of the scene. However, they can usually only produce 3D models at the geometric level, which are difficult to reflect the spatial logic, occlusion relationships, contact chain information, and temporal correlations between physical evidence elements, scene components, and human traces. Traditional 3D reconstruction methods are mostly based on geometric structures and do not have the ability to organize multi-source scene data in a unified manner and construct a computable relational structure, which cannot support subsequent intelligent reasoning and correlation analysis.

[0003] In deep learning-based scene analysis research, existing graph neural networks generally adopt common graph construction and convolution methods, lacking structural relationship constraint mechanisms for criminal investigation scenes. The selection method of convolutional neighborhood does not match the relationship of physical evidence. The pooling process is prone to destroying key spatial, contact chain and occlusion semantics, making it difficult for the model to maintain the real structure of the crime scene. Existing technologies cannot yet achieve structured fusion and three-dimensional semantic modeling of multi-source scene data, nor can they form a structured three-dimensional scene expression for case analysis. Summary of the Invention

[0004] One objective of this invention is to propose a structured crime scene modeling method based on three-dimensional graph convolution. This invention constructs a structured crime scene model based on three-dimensional graph convolution, realizes the fusion of multi-source data and criminal investigation relationships, and has the advantages of high semantic fidelity, complete structural expression and strong modeling reliability.

[0005] According to an embodiment of the present invention, a structured modeling method for crime scenes based on three-dimensional graph convolution includes the following steps: Collect multi-source raw data from the crime scene to generate a multi-source on-site input dataset; A three-dimensional spatial map of the crime scene is constructed based on a multi-source on-site input dataset. The physical evidence elements, scene components and human traces in the crime scene are used as graph nodes, and graph edges are established according to the set of constraints on the structural relationships between nodes. The set of constraints on the structural relationships of the crime scene includes spatial distance relationships, occlusion relationships, contact chain relationships and temporal correlation relationships, thus generating a three-dimensional spatial map. Initialize node features and edge weights in the 3D spatial graph to generate an initial graph structure; The initial graph structure is subjected to 3D graph convolution processing. The convolutional neighborhood of the nodes is constrained and filtered according to the edge weight of the graph edge. The convolution operation based on neighborhood aggregation is performed on the node features within the neighborhood, the node features are updated, and an intermediate feature map is generated. Graph pooling is performed on the intermediate feature map, and the pooling aggregation process is subject to structure preservation constraints based on the set of constraints on the structural relationships of the crime scene, thereby generating a compressed structural representation of the crime scene. By decoding and modeling the compressed structural representation, a structured three-dimensional scene representation of the crime scene is obtained.

[0006] Optionally, the generation of the multi-source on-site input dataset specifically includes: Collect image data, video sequence data, depth point cloud data, laser mapping coordinate data, and physical evidence annotation data from the crime scene; Time references were standardized for image data, video sequence data, depth point cloud data, laser mapping coordinate data, and physical evidence annotation data, and data from different sources were sorted according to a unified crime scene time reference. Spatial reference unification is performed on image data, video sequence data, depth point cloud data, laser mapping coordinate data, and physical evidence annotation data, and the spatial location identifiers, acquisition direction identifiers, and device location identifiers corresponding to the data are made consistent. Preprocessing operations are performed on image data, video sequence data, depth point cloud data, laser mapping coordinate data, and physical evidence annotation data that have achieved unified time and spatial references. Field values ​​of various types of data are normalized according to preset data standardization standards. Data that has undergone time-reference unification, spatial-reference unification, and preprocessing is structured and combined according to data source, acquisition time, acquisition location, and data type to generate a multi-source field input dataset.

[0007] Optionally, the generation of the three-dimensional spatial map specifically includes: Extract physical evidence elements, scene components, and human traces from the crime scene from the multi-source on-site input dataset, perform unified node generation processing, and generate a set of graph nodes for a three-dimensional spatial map of the crime scene; Based on the spatial distance relationship in the set of structural relationship constraints at the crime scene, read the spatial position field of any two graph nodes in the graph node set, perform association checks based on the spatial distance judgment conditions between the spatial position fields, establish graph edges corresponding to the spatial distance relationship for graph node pairs that satisfy the spatial distance relationship, and record them as spatial distance relationship graph edges; Based on the occlusion relationship in the structural relationship constraint set of the crime scene, read the occlusion feature fields of any two graph nodes in the graph node set, perform association checks based on the occlusion judgment conditions corresponding to the occlusion feature fields, establish graph edges corresponding to the occlusion relationship for graph node pairs that satisfy the occlusion relationship, and record them as occlusion relationship graph edges. Based on the contact chain relationship in the structural relationship constraint set of the crime scene, read the contact chain field of any two graph nodes in the graph node set, perform association check according to the contact chain judgment condition corresponding to the contact chain field, establish graph edges corresponding to the contact chain relationship for graph node pairs that satisfy the contact chain relationship, and record them as contact chain relationship graph edges. Based on the temporal association relationships in the structural relationship constraint set of the crime scene, read the time fields of any two graph nodes in the graph node set, perform association checks based on the temporal judgment conditions corresponding to the time fields, establish graph edges corresponding to the temporal association relationships for graph node pairs that satisfy the temporal association relationships, and record them as temporal association relationship graph edges; All graph edges generated based on spatial distance relationships, occlusion relationships, contact chain relationships, and temporal correlation relationships are combined in the order of their creation to generate a graph edge set for the three-dimensional spatial graph of the crime scene. By combining the graph node set and the graph edge set, a three-dimensional spatial graph of the crime scene is generated.

[0008] Optionally, the generation of the initial graph structure specifically includes: For each node in the 3D spatial map of the crime scene, its spatial location field, category field, and identifier field are read. The spatial location field, category field, and identifier field are then arranged in a uniform order to generate node feature data corresponding to each node. The node feature data corresponding to all graph nodes are combined according to the record order of the graph node set to generate a node feature set; For each edge in the 3D spatial graph of the crime scene, read its corresponding relationship type field, and organize the spatial distance relationship field, occlusion relationship field, contact chain relationship field and temporal association relationship field according to the edge recording order to generate edge weight data corresponding to each edge. Combine the edge weight data corresponding to all graph edges according to the record order of the graph edge set to generate an edge weight set; Align the node feature set item by item according to the recording order of the graph node set, and align the edge weight set item by item according to the recording order of the graph edge set; The initial graph structure is generated by combining the graph node set, graph edge set, node feature set, and edge weight set according to a fixed input format.

[0009] Optionally, the generation of the intermediate feature map specifically includes: Read the initial node features of each graph node from the initial graph structure according to the record order of the graph node set; Read the edge weight of each graph edge from the initial graph structure according to the record order of the graph edge set; Based on the edge weights corresponding to the graph edges, the graph nodes that can be associated with each graph node are filtered, and the graph nodes whose edge weights satisfy the convolutional neighborhood filtering conditions are determined as the convolutional neighborhood nodes of that graph node. For each graph node, the initial node features of its convolutional neighbor nodes are read in the order of the convolutional neighborhood records, and these features are combined to generate the feature combination result. The feature combination result of the convolutional neighborhood nodes is jointly processed with the initial node features of the graph node itself to generate a node feature set; For each graph node, the node feature set corresponding to that node is updated according to the record order of the graph node set to generate an updated node feature set. For each graph node, according to the recording order of the graph node set, the update node feature corresponding to the graph node is written into the node feature field of the graph node, and the node feature field of the graph node is matched one-to-one with the update node feature. For each graph edge set, read the indices of the two graph nodes corresponding to each graph edge according to the record order of the graph edge set, and establish an association between the updated node features of the corresponding index in the updated node feature set and the graph edge, keeping the node features referenced by the graph edge in the same order as the node relationships in the initial graph structure; According to the recording order of the graph node set and the graph edge set, the updated node feature set is written into the node feature field corresponding to the graph node set, the graph edge set is registered while maintaining its original order, and the two are combined into a graph structure after convolution processing. For all graph nodes in the graph structure after convolution, read their updated node feature fields according to the record order of the graph node set, and establish a correspondence with the graph edge set to generate an intermediate feature map composed of the updated node feature set, the graph node set, and the graph edge set.

[0010] Optionally, the generation of the compressed structure representation specifically includes: Read the updated node features of each graph node from the intermediate feature map according to the record order of the graph node set, and retain the read updated node features as the basic feature data for pooling processing; Read the set of crime scene structural relationship constraints corresponding to each graph edge from the intermediate feature graph according to the record order of the graph edge set, and use the spatial distance relationship, occlusion relationship, contact chain relationship and temporal association relationship as the relationship fields used for structure preservation during the pooling process; Based on the set of structural relationship constraints at the crime scene, the graph nodes are divided into pooled regions. Graph nodes that simultaneously satisfy the conditions of the set of structural relationship constraints at the crime scene are assigned to the same pooled region, so that each pooled region contains several graph nodes that satisfy the structural relationship constraints. For each graph node in the pooling region, read its updated node features and aggregate the updated node features in the pooling region according to the preset feature aggregation rules to generate the pooling node features corresponding to each pooling region. The structural relationships between pooling regions are reconstructed. Based on the graph edge set record in the intermediate feature graph, the relationship fields between different pooling regions are read in sequence. The spatial distance relationship, occlusion relationship, contact chain relationship and temporal association relationship between graph nodes belonging to two pooling regions are summarized according to the corresponding relationship of the pooling regions. The summarized relationship fields are registered as the structural relationship between pooling regions. The structural relationship between pooled node features and pooled regions is combined according to the recording order of pooled regions to construct a pooled graph structure. The constructed graph structure is then organized to generate a compressed structural representation of the crime scene.

[0011] Optionally, the modeling specifically includes: Read the pooling node features corresponding to each pooling region from the compressed structural representation of the crime scene according to the recording order of the pooling regions; The structural relationships between each pooling region are read from the compressed structural representation of the crime scene in the order of the recorded structural relationships between the pooling regions; Based on the pooling node features, feature expansion processing is performed on each pooling region, and the pooling node features corresponding to each pooling region are expanded according to the region recording order to generate extended node features. The connection method between pooling regions is reconstructed based on the structural relationship. The structural relationship between pooling regions is registered according to the recording order of the pooling regions, and the structural relationship between different pooling regions is established as the connection relationship of the decoding graph structure. The connection relationship between the extended node features and the decoding graph structure is combined in a fixed order, and the combined decoding graph structure is then subjected to feature recombination processing. The recombined decoded graph structure is organized according to the recording order of the pooled regions to generate a structured 3D scene representation of the crime scene.

[0012] The beneficial effects of this invention are: This invention constructs a three-dimensional spatial graph based on multi-source raw data from the crime scene, uniformly representing physical evidence elements, scene components, and personnel traces as graph nodes. Graph edges are constructed using spatial distance relationships, occlusion relationships, contact chain relationships, and temporal correlation relationships, enabling a unified structured representation of the geometric information and criminal investigation semantic information of the crime scene. This invention overcomes the limitations of traditional 3D reconstruction techniques, which can only restore geometric shapes and cannot reflect criminal investigation relationships. By structurally initializing node features and edge weights, multi-source data can be accurately processed by the computational model in both spatial and semantic dimensions. This provides an atomically and scene-level consistent input structure for subsequent graph convolution feature extraction, effectively solving the problems of difficulty in fusing multi-source data and the inability to uniformly model semantic relationships in existing technologies.

[0013] This invention further introduces a 3D graph convolution mechanism based on edge weight constraints to achieve semantic filtering of the convolutional neighborhood for criminal investigation. This allows key criminal investigation structural relationships such as space, occlusion, contact chains, and temporal sequence to directly influence the feature aggregation process, thereby ensuring that the semantic relationships are not weakened or lost during feature updates. Through a structure-preserving pooling method, the core structural relationships of the crime scene can be continuously maintained during node aggregation and region compression, ensuring that the compressed structural representation still possesses complete spatial logic and case analysis-related information. In addition, this invention reconstructs regional-level node features and structural relationships by decoding and modeling the compressed structural representation, restoring the structured 3D scene of the crime scene while ensuring semantic consistency and structural integrity. This allows the output results to directly serve case analysis, path inference, evidence association, and scene reconstruction. The method of this invention has significant advantages such as high semantic fidelity, stable structural expression, good multi-source information fusion effect, and strong interpretability, significantly improving the practical value and reasoning ability of 3D crime scene modeling in criminal investigation scenarios. Attached Figure Description

[0014] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of a structured crime scene modeling method based on 3D graph convolution proposed in this invention; Figure 2 This is a schematic diagram illustrating the structure of a three-dimensional spatial graph of a crime scene, based on a three-dimensional graph convolution-based structured modeling method for crime scenes proposed in this invention. Detailed Implementation

[0015] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0016] refer to Figure 1 and Figure 2 A structured crime scene modeling method based on 3D graph convolution includes the following steps: Collect multi-source raw data from the crime scene to generate a multi-source on-site input dataset; A three-dimensional spatial map of the crime scene is constructed based on a multi-source on-site input dataset. The physical evidence elements, scene components and human traces in the crime scene are used as graph nodes, and graph edges are established according to the set of constraints on the structural relationships between nodes. The set of constraints on the structural relationships of the crime scene includes spatial distance relationships, occlusion relationships, contact chain relationships and temporal correlation relationships, thus generating a three-dimensional spatial map. Initialize node features and edge weights in the 3D spatial graph to generate an initial graph structure; The initial graph structure is subjected to 3D graph convolution processing. The convolutional neighborhood of the nodes is constrained and filtered according to the edge weight of the graph edge. The convolution operation based on neighborhood aggregation is performed on the node features within the neighborhood, the node features are updated, and an intermediate feature map is generated. Graph pooling is performed on the intermediate feature map, and the pooling aggregation process is subject to structure preservation constraints based on the set of constraints on the structural relationships of the crime scene, thereby generating a compressed structural representation of the crime scene. By decoding and modeling the compressed structural representation, a structured three-dimensional scene representation of the crime scene is obtained.

[0017] In this embodiment, the generation of the multi-source on-site input dataset specifically includes: Collect image data from the crime scene, organize the image files generated from each shooting location at the crime scene according to the shooting order, and record the shooting time, shooting location information, and acquisition device identification for each image; Collect video sequence data from the crime scene, divide the continuously captured video sequence into video frames according to time sequence, and record the shooting time, shooting device location and frame number information of each video frame, so that the image data and video sequence data maintain a corresponding relationship in terms of acquisition time; Collect depth point cloud data from the crime scene, organize the point cloud scanning results obtained by the depth acquisition device according to the order of point cloud blocks, record the acquisition time, acquisition device attitude, point cloud density and point cloud coverage of each depth point cloud, and maintain the spatial correspondence between the depth point cloud data and image data and video sequence data. Collect laser mapping coordinate data at the crime scene, organize the three-dimensional mapping coordinates obtained by the laser mapping equipment according to the mapping point order, and record the three-dimensional coordinate value, mapping time, mapping orientation and equipment position of each mapping point to maintain a unified spatial reference between the laser mapping coordinate data and the depth point cloud data. Collect evidence annotation data from the crime scene, organize the annotation results of evidence location, evidence type and evidence number formed during the on-site investigation in the order of evidence number, record the annotation time, annotation source, annotation location and annotation content of each evidence annotation, and associate the evidence annotation data with image data and video sequence data accordingly; Time reference is unified for image data, video sequence data, depth point cloud data, laser mapping coordinate data and physical evidence annotation data. Data from different sources are sorted according to a unified crime scene time benchmark to ensure consistency of multi-source raw data in the time dimension. The unified time reference specifically includes: reading the collection time field from various types of data, aligning the collection time field item by item according to the collection source to obtain the original time value of each data; performing time offset processing on all original time values ​​according to the crime scene time benchmark to obtain the time alignment value of each data; and registering the time alignment value of all data according to the unified crime scene time benchmark, so that the multi-source original data adopts a consistent time reference system in the time dimension. Spatial reference unification is performed on image data, video sequence data, depth point cloud data, laser mapping coordinate data and physical evidence annotation data. The spatial location identifiers, acquisition direction identifiers and device location identifiers corresponding to the data are made consistent, so that multi-source raw data from different sources adopt a unified three-dimensional reference system in the spatial dimension. The unified spatial reference is as follows: the three-dimensional coordinate field of each mapping point is read from the laser mapping coordinate data, and the three-dimensional coordinate field is used as a unified spatial reference benchmark; the spatial position fields in image data, video sequence data, depth point cloud data and physical evidence annotation data are processed by coordinate mapping to obtain the spatial mapping results of various types of data; the spatial mapping results are registered according to the three-dimensional reference benchmark so that all data adopt a unified three-dimensional position reference system in the spatial dimension. Preprocessing operations are performed on image data, video sequence data, depth point cloud data, laser mapping coordinate data, and physical evidence annotation data that have achieved unified time and spatial references. Field values ​​of various types of data are normalized according to preset data standardization standards. Data that has undergone time-reference unification, spatial-reference unification, and preprocessing is structured and combined according to data source, acquisition time, acquisition location, and data type to generate a multi-source field input dataset.

[0018] In this embodiment, the generation of the three-dimensional spatial map specifically includes: Physical evidence elements, scene components, and human traces from the crime scene are extracted from the multi-source crime scene input dataset. A unified node generation process is then performed. The spatial location, category, and identification fields corresponding to the physical evidence elements, the spatial location, type, and identification fields corresponding to the scene components, and the spatial location, trace type, and trace identification fields corresponding to the human traces in the multi-source crime scene input dataset are read separately. The spatial location, category, type, and identification fields are then combined according to the recording order to generate a set of graph nodes for a three-dimensional spatial map of the crime scene. This set of graph nodes completely represents the physical evidence elements, scene components, and human traces at the crime scene. The node generation specifically includes: when generating a graph node set from extracted evidence elements, scene components, and personnel traces, reading the spatial location field, category field, type field, or identifier field of each type of data, organizing the fields according to the record order to obtain the node field data corresponding to the graph node; combining the node field data according to the node record order as the field composition of the graph node set, so that the graph node set can completely represent the spatial and category information of the evidence elements, scene components, and personnel traces at the crime scene; Based on the spatial distance relationship in the set of structural relationship constraints at the crime scene, read the spatial position field of any two graph nodes in the graph node set, perform association checks based on the spatial distance judgment conditions between the spatial position fields, establish graph edges corresponding to the spatial distance relationship for graph node pairs that satisfy the spatial distance relationship, and record them as spatial distance relationship graph edges; The generation of the spatial distance relationship graph edge specifically includes: reading the spatial position field of any two graph nodes in the graph node set; splitting the spatial position field according to the recording order to obtain the spatial position data value of the first graph node and the spatial position data value of the second graph node; then performing item-by-item difference processing on the two spatial position data values ​​to obtain spatial difference data; performing value accumulation processing on the spatial difference data according to a fixed accumulation order to obtain the spatial position difference result between the two graph nodes; comparing the spatial position difference result with the spatial distance relationship judgment threshold to obtain the spatial distance judgment result value; when the spatial distance judgment result value meets the spatial distance relationship judgment condition, recording the spatial distance relationship between the two graph nodes and generating the spatial distance relationship graph edge. Based on the occlusion relationship in the structural relationship constraint set of the crime scene, read the occlusion feature fields of any two graph nodes in the graph node set, perform association checks based on the occlusion judgment conditions corresponding to the occlusion feature fields, establish graph edges corresponding to the occlusion relationship for graph node pairs that satisfy the occlusion relationship, and record them as occlusion relationship graph edges. The generation of the occlusion relationship graph edges specifically includes: reading the occlusion feature fields of any two graph nodes, comparing the occlusion description values ​​in the occlusion feature fields item by item to obtain occlusion comparison result values; comparing the occlusion comparison result values ​​according to the occlusion judgment conditions corresponding to the occlusion relationship to obtain occlusion relationship judgment values; when the occlusion relationship judgment value meets the occlusion relationship judgment conditions, recording the occlusion relationship between the two graph nodes and generating occlusion relationship graph edges. Based on the contact chain relationship in the structural relationship constraint set of the crime scene, read the contact chain field of any two graph nodes in the graph node set, perform association check according to the contact chain judgment condition corresponding to the contact chain field, establish graph edges corresponding to the contact chain relationship for graph node pairs that satisfy the contact chain relationship, and record them as contact chain relationship graph edges. The generation of the contact chain relationship graph edge specifically includes: reading the contact chain field of any two graph nodes, comparing the contact association values ​​in the contact chain field according to the recording order to obtain the contact chain comparison value; comparing the contact chain comparison value with the judgment condition corresponding to the contact chain relationship to obtain the contact chain relationship judgment value; when the contact chain relationship judgment value meets the contact chain relationship judgment condition, recording the contact chain relationship between the two graph nodes and generating the contact chain relationship graph edge. Based on the temporal association relationships in the structural relationship constraint set of the crime scene, read the time fields of any two graph nodes in the graph node set, perform association checks based on the temporal judgment conditions corresponding to the time fields, establish graph edges corresponding to the temporal association relationships for graph node pairs that satisfy the temporal association relationships, and record them as temporal association relationship graph edges; The generation of the temporal association graph edge specifically includes: reading the time field of any two graph nodes, performing time difference processing on the time record values ​​in the time field to obtain a temporal difference; comparing the temporal difference with the judgment condition corresponding to the temporal relationship to obtain a temporal association judgment value; when the temporal association judgment value meets the temporal relationship judgment condition, recording the temporal association relationship between the two graph nodes and generating a temporal association graph edge. All graph edges generated based on spatial distance relationships, occlusion relationships, contact chain relationships, and temporal correlation relationships are combined in the order of their creation to generate a graph edge set of the three-dimensional spatial graph of the crime scene, so that the graph edge set completely includes all graph edge types in the set of structural relationship constraints of the crime scene; By combining the graph node set and the graph edge set, a three-dimensional spatial graph of the crime scene is generated.

[0019] In this embodiment, the generation of the initial graph structure specifically includes: For each node in the 3D spatial map of the crime scene, its spatial location field, category field, and identifier field are read. The spatial location field, category field, and identifier field are then arranged in a uniform order to generate node feature data corresponding to each node. The node feature data corresponding to all graph nodes are combined according to the recording order of the graph node set to generate a node feature set, so that the node feature set can completely represent the spatial location attribute, category attribute and identification attribute of the graph node. For each edge in the 3D spatial graph of the crime scene, read its corresponding relationship type field, and organize the spatial distance relationship field, occlusion relationship field, contact chain relationship field and temporal association relationship field according to the edge recording order to generate edge weight data corresponding to each edge. All edge weights corresponding to the graph edges are combined according to the recording order of the graph edge set to generate an edge weight set, so that the edge weight set can completely represent the spatial distance relationship, occlusion relationship, contact chain relationship and temporal relationship corresponding to the graph edges; Align the node feature set item by item according to the recording order of the graph node set, and align the edge weight set item by item according to the recording order of the graph edge set, so that the node feature set and edge weight set are consistent with the graph node set and graph edge set in index order, respectively. The graph node set, graph edge set, node feature set, and edge weight set are combined according to a fixed input format to generate an initial graph structure composed of the node set, edge set, node feature set, and edge weight set.

[0020] In this embodiment, the generation of the intermediate feature map specifically includes: Read the initial node features of each graph node from the initial graph structure according to the record order of the graph node set; Read the edge weight of each graph edge from the initial graph structure according to the record order of the graph edge set, and use the edge weight to limit the association strength between graph nodes; Based on the edge weights corresponding to the graph edges, the graph nodes that can be associated with each graph node are filtered, and the graph nodes whose edge weights satisfy the convolutional neighborhood filtering conditions are determined as the convolutional neighborhood nodes of that graph node. The filtering process specifically includes: reading the edge weight field of each graph node's associative graph edges; comparing the spatial distance relationship value, occlusion relationship value, contact chain relationship value, and temporal association relationship value in the edge weight field in a fixed order to obtain the edge weight comparison value; judging the edge weight comparison value according to the convolutional neighborhood filtering conditions corresponding to each relationship to obtain the convolutional neighborhood judgment result; when the judgment result meets the convolutional neighborhood filtering conditions, the corresponding graph node is taken as a convolutional neighborhood node; For each graph node, the initial node features of its convolutional neighbor nodes are read in the order of the convolutional neighborhood records, and these features are combined to generate the feature combination result. The feature combination result of the convolutional neighbor nodes is jointly processed with the initial node features of the graph node itself to generate a set of node features for performing feature updates on the graph node. For each graph node, the node feature set corresponding to that node is updated according to the record order of the graph node set to generate an updated node feature set. The feature update process specifically includes: for each graph node, merging the feature combination result of the convolutional neighbor nodes with the initial node features of the graph node according to a fixed field order to obtain the node feature update value; writing the node feature update value into the node feature field of the graph node to generate the updated node feature set; For each graph node, according to the recording order of the graph node set, the update node feature corresponding to the graph node is written into the node feature field of the graph node, and the node feature field of the graph node is matched one-to-one with the update node feature. For each graph edge set, read the indices of the two graph nodes corresponding to each graph edge according to the record order of the graph edge set, and establish an association between the updated node features of the corresponding index in the updated node feature set and the graph edge, keeping the node features referenced by the graph edge in the same order as the node relationships in the initial graph structure; According to the recording order of the graph node set and the graph edge set, the updated node feature set is written into the node feature field corresponding to the graph node set, the graph edge set is registered while maintaining its original order, and the two are combined into a graph structure after convolution processing. For all graph nodes in the graph structure after convolution, read their updated node feature fields according to the record order of the graph node set, and establish a correspondence with the graph edge set to generate an intermediate feature map composed of the updated node feature set, the graph node set, and the graph edge set.

[0021] In this embodiment, the generation of the compressed structure representation specifically includes: Read the updated node features of each graph node from the intermediate feature map according to the record order of the graph node set, and retain the read updated node features as the basic feature data for pooling processing; Read the set of crime scene structural relationship constraints corresponding to each graph edge from the intermediate feature graph according to the record order of the graph edge set, and use the spatial distance relationship, occlusion relationship, contact chain relationship and temporal association relationship as the relationship fields used for structure preservation during the pooling process; Based on the set of structural relationship constraints at the crime scene, the graph nodes are divided into pooled regions. Graph nodes that simultaneously satisfy the conditions of the set of structural relationship constraints at the crime scene are assigned to the same pooled region, so that each pooled region contains several graph nodes that satisfy the structural relationship constraints. The pooling region division specifically includes: reading the spatial distance relationship field, occlusion relationship field, contact chain relationship field, and temporal association relationship field of each graph node, comparing the fields in a fixed order to obtain a structural relationship comparison value; filtering the structural relationship comparison value according to the division conditions in the structural relationship constraint set of the crime scene to obtain the region division result; when the region division result meets the division conditions, registering the corresponding graph node to the same pooling region; For each graph node in the pooling region, read its updated node features and aggregate the updated node features in the pooling region according to the preset feature aggregation rules to generate the pooling node features corresponding to each pooling region. The structural relationships between pooling regions are reconstructed. Based on the graph edge set record in the intermediate feature graph, the relationship fields between different pooling regions are read in sequence. The spatial distance relationship, occlusion relationship, contact chain relationship and temporal association relationship between graph nodes belonging to two pooling regions are summarized according to the corresponding relationship of the pooling regions. The summarized relationship fields are registered as the structural relationship between pooling regions. The reconstruction of the structural relationships between pooling regions specifically includes: reading the spatial distance relationship field, occlusion relationship field, contact chain relationship field, and temporal association relationship field between graph nodes belonging to different pooling regions one by one to obtain node-level structural relationship values; summarizing the node-level structural relationship values ​​according to the pooling region correspondence to obtain region-level structural relationship values; and registering the region-level structural relationship values ​​as the structural relationships between pooling regions. The structural relationship between pooled node features and pooled regions is combined according to the recording order of pooled regions to construct a pooled graph structure. The constructed graph structure is then organized to generate a compressed structural representation of the crime scene.

[0022] In this embodiment, the modeling specifically includes: The pooling node features corresponding to each pooling region are read from the compressed structural representation of the crime scene in the order of the recorded pooling regions, and used as the initial feature data for decoding modeling. The structural relationships between each pooling region are read from the compressed structural representation of the crime scene in the order of the recorded structural relationships between the pooling regions, and used as the structural basis for region connection in decoding modeling; Based on the pooling node features, feature expansion processing is performed on each pooling region. The pooling node features corresponding to each pooling region are expanded according to the region recording order to generate extended node features for decoding. The connection method between pooling regions is reconstructed based on the structural relationship. The structural relationship between pooling regions is registered according to the recording order of the pooling regions, and the structural relationship between different pooling regions is established as the connection relationship of the decoding graph structure. The connection relationships between the extended node features and the decoded graph structure are combined in a fixed order. The combined decoded graph structure is then subjected to feature reorganization processing to ensure that the extended node features and structural relationships maintain a consistent index order in the decoded graph structure. The recombined decoded graph structure is organized according to the recording order of the pooled regions to generate a structured 3D scene representation of the crime scene.

[0023] Example 1: To verify the feasibility of this invention in practice, it was applied to the 3D modeling and information analysis of a crime scene. The crime scene was located inside a closed building, including a main room and connected auxiliary spaces. Multiple pieces of physical evidence, obvious signs of movement, and partially obscured structures were distributed within the scene. Due to the complex spatial structure of the scene, the inconsistency of multi-source data, and the various investigative relationships among the physical evidence, traditional 3D modeling methods based on image reconstruction or point cloud fusion could not accurately display the spatial logic, obscuring relationships, contact chains, and temporal correlations between the evidence. This made it difficult to obtain structured clues for subsequent analysis. This invention, by constructing a structured crime scene modeling method based on 3D graph convolution, was fully validated in this scenario.

[0024] During the on-site investigation, staff used handheld acquisition devices to photograph the interior of the room from multiple angles, generating image and video data. Simultaneously, they performed depth scanning of the spatial structure to obtain depth point cloud data, and used laser mapping equipment to collect coordinate data for key locations within the room. Investigators also marked the location, type, and identification information of physical evidence on-site, creating evidence annotation data. Since all collected data came from different sources, had inconsistent collection times, and varied location references, time and spatial unification were necessary. This invention, during implementation, reads the time field of all collected data and performs offset correction using the crime scene time reference, ensuring a unified reference for time records from different devices. Subsequently, using the spatial three-dimensional reference system provided by the laser mapping coordinates as a unified benchmark, the spatial positions of images, videos, depth point clouds, and evidence annotation data are aligned through mapping relationships, thus giving all data a unified coordinate system in the spatial dimension. After unification processing, the multi-source data is structured and combined according to time, space, and category to form a multi-source on-site input dataset.

[0025] In this input dataset, physical evidence elements, scene components, and human traces are extracted as graph nodes, each containing a spatial location field, a category field, and an identifier field. Subsequently, fields related to spatial distance, occlusion information, contact chain associations, and temporal information are read item by item from multi-source scene data. Through comparison, judgment, and conditional filtering, structural relationships between each pair of nodes are generated. If two nodes meet the proximity condition in spatial location, a spatial distance relationship is established; if there is a condition corresponding to occlusion information between nodes, an occlusion relationship is established; if there is a contact chain connection between nodes, a contact chain relationship is established; if the two have a definite chronological order in the temporal field, a temporal relationship is established. All of the above relationships are recorded as graph edges, enabling the spatial structure of the crime scene and criminal investigation semantics to be expressed simultaneously, thus forming a three-dimensional spatial graph of the crime scene.

[0026] To enable the model to effectively compute nodes and relationships, this invention reads the spatial location, category, and identifier fields of each node and organizes them into a node feature set. Simultaneously, it organizes the relationship type field of each graph edge into an edge weight set, ensuring the initial graph structure is built on a unified field foundation. Subsequently, this initial graph structure undergoes 3D graph convolution processing. During convolution, based on the relationship fields corresponding to the graph edges, the associative nodes of each graph node are filtered, retaining only neighboring nodes that satisfy the structural relationship constraints as convolutional neighbors. The convolution operation combines the node features of the convolutional neighbors with the node's own features to participate in feature updates, allowing each node to ultimately obtain updated features containing relationship information. After convolution processing, the updated node features are rewritten into the graph node set, and the graph edges are combined in their original order to generate an intermediate feature graph.

[0027] Due to the complex structure and numerous nodes at the crime scene, directly using intermediate feature maps for 3D scene representation would be redundant. This invention employs a structure-preserving pooling method to divide nodes into regions, grouping nodes that simultaneously satisfy spatial distance, occlusion, contact chain, and temporal correlation conditions into the same pooling region. Node features within each pooling region are read and aggregated into pooled node features. Simultaneously, the structural relationships between pooling regions are reconstructed by reading relationship values ​​from multiple node-level relationship fields and summarizing them according to the corresponding relationships within the pooling regions to form region-level structural relationships. Subsequently, the pooled node features and region-level structural relationships are combined in a predetermined order to form a compressed structural representation of the crime scene. This representation fully preserves the global logic of the crime scene's structural relationships and is more concise, making it suitable for subsequent scene decoding.

[0028] In the scene decoding and modeling stage, this invention reads pooled node features item by item from the compressed structural representation and expands the features to restore the features of each pooled region to the features of the decoding node. At the same time, decoding edges are constructed according to the structural relationship between regions, so that the relationship between each region after pooling is expressed in a structured way. Then, all decoding node features and edge records are combined into a decoding graph structure, and finally a structured three-dimensional scene representation of the crime scene is generated. In this scene representation, the spatial proximity, occlusion layers, contact chain associations and temporal sequence between physical evidence are presented in a hierarchical and structured way, which solves the technical bottleneck that traditional three-dimensional reconstruction methods cannot express criminal investigation semantics.

[0029] As can be seen from the above implementation process, when faced with complex crime scenes involving multi-source data, mixed spatial structures, and concealed investigative relationships, this invention can unify heterogeneous data such as images, point clouds, laser mapping, and physical evidence annotations into a computable graph structure. Through graph convolution and structure-preserving pooling mechanisms, investigative relationships are fully preserved during the modeling process. In practical applications, the resulting structured 3D scene representation allows investigators to more clearly understand the structural logic between physical evidence, assisting in determining movement trajectories, occlusion layers, and contact chains, thus providing strong support for case analysis. This invention effectively solves the technical problem that traditional 3D modeling cannot express investigative semantics, significantly improving the structured representation capability of crime scene information and the reliability of on-site reasoning.

[0030] Table 1. Performance Comparison of the Invention Method and Traditional 3D Crime Scene Modeling Methods

[0031] As can be seen from the data in Table 1, the method of this invention has achieved stable and practical improvements over traditional 3D modeling methods in all key indicators. First, in terms of the integrity of spatial structure, the traditional method has a value of 74.2%, while the method of this invention reaches 82.6%, an improvement of more than 8 percentage points. This difference indicates that after aligning multi-source information, this invention, by introducing node feature initialization and convolutional neighborhood filtering with structural relationship constraints, can better preserve the spatial hierarchy during the convolution process, thereby reducing the structural collapse and local misalignment problems common in traditional methods.

[0032] In terms of accuracy in restoring occlusion relationships, the traditional method achieves 61.3%, while this invention reaches 76.8%, an improvement of more than 15 percentage points. This is one of the most significant improvements among all indicators. The core reason for this difference is that the traditional method does not explicitly model the occlusion relationship, while this invention uses the occlusion relationship as a graph edge record field in the graph construction stage to participate in the convolutional neighborhood selection. This makes the generation process of convolutional features structurally sensitive to the occlusion level, thereby enabling more accurate restoration of occlusion information.

[0033] Regarding the consistency of contact chain inference, the traditional method yields 54.7%, while this invention achieves 72.4%, an improvement of nearly 18 percentage points. This improvement mainly stems from the fact that this invention explicitly uses contact chain relationships as graph edges to participate in convolutional neighborhood selection. In other words, the contact chains that have occurred between nodes no longer depend solely on geometric position for judgment, but are used as contribution factors of the convolutional structure, allowing contact chain associations to be preserved and strengthened during feature update.

[0034] In terms of the degree of preservation of temporal correlation, the traditional method is only 47.9%, while the present invention improves it to 69.1%, an increase of more than 21 percentage points, which is the most significant improvement. The traditional method can hardly express temporal sequence in the geometric reconstruction framework. However, the present invention participates in the modeling by using the time field as the structural relationship of the graph edge, so that the convolution operation can directly preserve the temporal order logic between nodes, avoiding the loss of temporal relationship in the data fusion process.

[0035] The consistency of multi-source data fusion was improved from 68.5% to 81.7%, an increase of nearly 13 percentage points. This shows that the integration effect of the present invention on heterogeneous data in the two steps of time reference unification and spatial reference unification is significantly better than the traditional approach. Traditional methods usually rely on post-registration, while the present invention incorporates multi-source data into the same three-dimensional reference system in the mapping stage, making the fusion result more stable.

[0036] In terms of structural semantic completeness, the traditional method achieves 58.6%, while the present invention achieves 79.2%, an improvement of more than 20 percentage points. This indicates that the traditional method is prone to losing semantic information when simplifying the structure, while the present invention relies on the structure-preserving pooling mechanism to ensure that the pooling stage does not destroy the four key relationships of space, occlusion, contact chain and temporal sequence, thus obtaining a more complete semantic expression.

[0037] In terms of interpretability, the score improved from 5.2 to 7.6, and the reconstruction stability score increased from 5.9 to 8.1. This indicates that the present invention has significantly improved the logical coherence of the 3D scene expression and the repeatability of the generation effect. In particular, the improvement in reconstruction stability exceeded 2 points, reflecting that structure-preserving pooling and decoding modeling have a significant enhancing effect on the final expression quality.

[0038] As can be seen from the above data, the performance improvement of this invention does not come from single-point optimization, but from a complete structured modeling mechanism: explicit encoding of relations in the graph, structural constraints in convolution, semantic preservation in pooling, and structural reconstruction in decoding. These designs together enable this invention to achieve higher semantic fidelity and structural stability than traditional methods in actual crime scene 3D modeling.

[0039] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A structured modeling method for crime scenes based on 3D graph convolution, characterized in that, Includes the following steps: Collect multi-source raw data from the crime scene to generate a multi-source on-site input dataset; A three-dimensional spatial map of the crime scene is constructed based on a multi-source on-site input dataset. The physical evidence elements, scene components and human traces in the crime scene are used as graph nodes, and graph edges are established according to the set of constraints on the structural relationships between nodes. The set of constraints on the structural relationships of the crime scene includes spatial distance relationships, occlusion relationships, contact chain relationships and temporal correlation relationships, thus generating a three-dimensional spatial map. Initialize node features and edge weights in the 3D spatial graph to generate an initial graph structure; The initial graph structure is subjected to 3D graph convolution processing. The convolutional neighborhood of the nodes is constrained and filtered according to the edge weight of the graph edge. The convolution operation based on neighborhood aggregation is performed on the node features within the neighborhood, the node features are updated, and an intermediate feature map is generated. Graph pooling is performed on the intermediate feature map, and the pooling aggregation process is subject to structure preservation constraints based on the set of constraints on the structural relationships of the crime scene, thereby generating a compressed structural representation of the crime scene. By decoding and modeling the compressed structural representation, a structured three-dimensional scene representation of the crime scene is obtained.

2. The method for structured modeling of crime scenes based on 3D graph convolution as described in claim 1, wherein, The generation of the multi-source on-site input dataset specifically includes: Collect image data, video sequence data, depth point cloud data, laser mapping coordinate data, and physical evidence annotation data from the crime scene; Time references were standardized for image data, video sequence data, depth point cloud data, laser mapping coordinate data, and physical evidence annotation data, and data from different sources were sorted according to a unified crime scene time reference. Spatial reference unification is performed on image data, video sequence data, depth point cloud data, laser mapping coordinate data, and physical evidence annotation data, and the spatial location identifiers, acquisition direction identifiers, and device location identifiers corresponding to the data are made consistent. Preprocessing operations are performed on image data, video sequence data, depth point cloud data, laser mapping coordinate data, and physical evidence annotation data that have achieved unified time and spatial references. Field values ​​of various types of data are normalized according to preset data standardization standards. Data that has undergone time-reference unification, spatial-reference unification, and preprocessing is structured and combined according to data source, acquisition time, acquisition location, and data type to generate a multi-source field input dataset.

3. The method for structured modeling of crime scenes based on 3D graph convolution as described in claim 1, wherein, The generation of the three-dimensional spatial map specifically includes: Extract physical evidence elements, scene components, and human traces from the crime scene from the multi-source on-site input dataset, perform unified node generation processing, and generate a set of graph nodes for a three-dimensional spatial map of the crime scene; Based on the spatial distance relationship in the set of structural relationship constraints at the crime scene, read the spatial position field of any two graph nodes in the graph node set, perform association checks based on the spatial distance judgment conditions between the spatial position fields, establish graph edges corresponding to the spatial distance relationship for graph node pairs that satisfy the spatial distance relationship, and record them as spatial distance relationship graph edges; Based on the occlusion relationship in the structural relationship constraint set of the crime scene, read the occlusion feature fields of any two graph nodes in the graph node set, perform association checks based on the occlusion judgment conditions corresponding to the occlusion feature fields, establish graph edges corresponding to the occlusion relationship for graph node pairs that satisfy the occlusion relationship, and record them as occlusion relationship graph edges. Based on the contact chain relationship in the structural relationship constraint set of the crime scene, read the contact chain field of any two graph nodes in the graph node set, perform association check according to the contact chain judgment condition corresponding to the contact chain field, establish graph edges corresponding to the contact chain relationship for graph node pairs that satisfy the contact chain relationship, and record them as contact chain relationship graph edges. Based on the temporal association relationships in the structural relationship constraint set of the crime scene, read the time fields of any two graph nodes in the graph node set, perform association checks based on the temporal judgment conditions corresponding to the time fields, establish graph edges corresponding to the temporal association relationships for graph node pairs that satisfy the temporal association relationships, and record them as temporal association relationship graph edges; All graph edges generated based on spatial distance relationships, occlusion relationships, contact chain relationships, and temporal correlation relationships are combined in the order of their creation to generate a graph edge set for the three-dimensional spatial graph of the crime scene. By combining the graph node set and the graph edge set, a three-dimensional spatial graph of the crime scene is generated.

4. The method for structured modeling of crime scenes based on 3D graph convolution as described in claim 1, wherein, The generation of the initial graph structure specifically includes: For each node in the 3D spatial map of the crime scene, its spatial location field, category field, and identifier field are read. The spatial location field, category field, and identifier field are then arranged in a uniform order to generate node feature data corresponding to each node. The node feature data corresponding to all graph nodes are combined according to the record order of the graph node set to generate a node feature set; For each edge in the 3D spatial graph of the crime scene, read its corresponding relationship type field, and organize the spatial distance relationship field, occlusion relationship field, contact chain relationship field and temporal association relationship field according to the edge recording order to generate edge weight data corresponding to each edge. Combine the edge weight data corresponding to all graph edges according to the record order of the graph edge set to generate an edge weight set; Align the node feature set item by item according to the recording order of the graph node set, and align the edge weight set item by item according to the recording order of the graph edge set; The initial graph structure is generated by combining the graph node set, graph edge set, node feature set, and edge weight set according to a fixed input format.

5. A method for structured modeling of crime scenes based on 3D graph convolution as described in claim 1, wherein, The generation of the intermediate feature map specifically includes: Read the initial node features of each graph node from the initial graph structure according to the record order of the graph node set; Read the edge weight of each graph edge from the initial graph structure according to the record order of the graph edge set; Based on the edge weights corresponding to the graph edges, the graph nodes that can be associated with each graph node are filtered, and the graph nodes whose edge weights satisfy the convolutional neighborhood filtering conditions are determined as the convolutional neighborhood nodes of that graph node. For each graph node, the initial node features of its convolutional neighbor nodes are read in the order of the convolutional neighborhood records, and these features are combined to generate the feature combination result. The feature combination result of the convolutional neighborhood nodes is jointly processed with the initial node features of the graph node itself to generate a node feature set; For each graph node, the node feature set corresponding to that node is updated according to the record order of the graph node set to generate an updated node feature set. For each graph node, according to the recording order of the graph node set, the update node feature corresponding to the graph node is written into the node feature field of the graph node, and the node feature field of the graph node is matched one-to-one with the update node feature. For each graph edge set, read the indices of the two graph nodes corresponding to each graph edge according to the record order of the graph edge set, and establish an association between the updated node features of the corresponding index in the updated node feature set and the graph edge, keeping the node features referenced by the graph edge in the same order as the node relationships in the initial graph structure; According to the recording order of the graph node set and the graph edge set, the updated node feature set is written into the node feature field corresponding to the graph node set, the graph edge set is registered while maintaining its original order, and the two are combined into a graph structure after convolution processing. For all graph nodes in the graph structure after convolution, read their updated node feature fields according to the record order of the graph node set, and establish a correspondence with the graph edge set to generate an intermediate feature map composed of the updated node feature set, the graph node set, and the graph edge set.

6. A method for structured modeling of crime scenes based on 3D graph convolution as described in claim 1, wherein, The generation of the compressed structure representation specifically includes: Read the updated node features of each graph node from the intermediate feature map according to the record order of the graph node set, and retain the read updated node features as the basic feature data for pooling processing; Read the set of crime scene structural relationship constraints corresponding to each graph edge from the intermediate feature graph according to the record order of the graph edge set, and use the spatial distance relationship, occlusion relationship, contact chain relationship and temporal association relationship as the relationship fields used for structure preservation during the pooling process; Based on the set of structural relationship constraints at the crime scene, the graph nodes are divided into pooled regions. Graph nodes that simultaneously satisfy the conditions of the set of structural relationship constraints at the crime scene are assigned to the same pooled region, so that each pooled region contains several graph nodes that satisfy the structural relationship constraints. For each graph node in the pooling region, read its updated node features and aggregate the updated node features in the pooling region according to the preset feature aggregation rules to generate the pooling node features corresponding to each pooling region. The structural relationships between pooling regions are reconstructed. Based on the graph edge set record in the intermediate feature graph, the relationship fields between different pooling regions are read in sequence. The spatial distance relationship, occlusion relationship, contact chain relationship and temporal association relationship between graph nodes belonging to two pooling regions are summarized according to the corresponding relationship of the pooling regions. The summarized relationship fields are registered as the structural relationship between pooling regions. The structural relationship between pooled node features and pooled regions is combined according to the recording order of pooled regions to construct a pooled graph structure. The constructed graph structure is then organized to generate a compressed structural representation of the crime scene.

7. A method for structured modeling of crime scenes based on 3D graph convolution as described in claim 1, wherein, The modeling specifically includes: Read the pooling node features corresponding to each pooling region from the compressed structural representation of the crime scene according to the recording order of the pooling regions; The structural relationships between each pooling region are read from the compressed structural representation of the crime scene in the order of the recorded structural relationships between the pooling regions; Based on the pooling node features, feature expansion processing is performed on each pooling region, and the pooling node features corresponding to each pooling region are expanded according to the region recording order to generate extended node features. The connection method between pooling regions is reconstructed based on the structural relationship. The structural relationship between pooling regions is registered according to the recording order of the pooling regions, and the structural relationship between different pooling regions is established as the connection relationship of the decoding graph structure. The connection relationship between the extended node features and the decoding graph structure is combined in a fixed order, and the combined decoding graph structure is then subjected to feature recombination processing. The recombined decoded graph structure is organized according to the recording order of the pooled regions to generate a structured 3D scene representation of the crime scene.

Images